X is significantly expanding its open-source codebase, which includes the app’s “For You” algorithm and its core ranking engine, and adding a feature that will let users see if their account or posts have been impacted by any of its ranking systems, the social network said on Thursday.
The company is making the source code for the “For You” timeline, the default feed you see when you open the app, available on GitHub under the Apache v2 license. It’s also expanding its previous efforts to open-source parts of its codebase to add more detail, including the model configuration, filter, and core ranking system details. That means it includes the parameters used to weight different signals — key to understanding which posts are actually displayed. This also makes the codebase roughly 10 to 15 times larger than it was before.
“You’ll get the core ranking code that pulls posts and ranks them for any given user and assembles the feed,” X’s VP of Product, Keith Coleman, told TechCrunch in an interview ahead of the announcement. “You can see the systems that filter out potentially problematic, rule-violating content…And some of those systems, like the ranker and the score, you can even run yourself outside the company.”
“This is this kind of thing that I think people will be fairly shocked that we are releasing,” he added.
In addition to the repository, X is providing tools that will let users see for themselves if and how X’s ranking systems have impacted their account or posts. A new transparency tool is rolling out to an “Under the Hood” page in the app’s settings, which will let users who have posted ten or more times over the past month download their aggregate stats as a JSON file. The file will show if any labels have been applied to their account or posts over the past calendar month.
Non-technical users can take advantage of this information by dropping it into an LLM of their choice, pointing the AI at X’s GitHub repo, and asking for an interpretation.
The company notes this tool will initially be available to a test group of accounts at least a year old as a pilot, before rolling out more broadly.
Ahead of launch, the company previewed its open source codebase to external researchers familiar with recommendation systems, who were able to get the “score,” or the numerical value calculated for every post, up and running outside of X. That was a major milestone for X’s transparency efforts, Coleman says.
From the GitHub repository, developers will be able to submit updates, known as pull requests, which X engineers will consider incorporating into its algorithm. While not all additions will make the cut, Coleman is enthusiastic about the idea.
“That would be amazing to have people submitting code that improves the algorithm…I mean, how cool would it be for the X algorithm to be not just visible to the public, but also, like, by the public?,” he said.
However, a few systems are not included in this release, like those that use Grok to predict whether a post could be violating a rule. This is meant to protect X from bad actors who could use this information to work around the company’s rules to flood the network with spam.
The changes are meant to address continuing concerns about how X’s algorithm influences politics, elections, the spread of misinformation, and more.
The platform has been home to political figures and high-profile individuals for years, and is now owned by a trillionaire who helped President Trump get elected. But Twitter had been under attack over its lack of transparency before it was bought by Musk, too. In earlier years, Republicans in Congress alleged that the California-headquartered social network leaned too far left, and claimed the network had “shadowbanned” their posts — meaning their posts were made invisible or undiscoverable without their knowledge. Twitter consistently denied this was the case.
“Our dream is that anyone in the public can be able to assess how posts are distributed on the platform, vet that it’s a level playing field, and, if they think it’s not, critique it so we can keep improving it and addressing it,” Coleman said.
“That’s the whole goal of this: [the code] can be audited; it can be critiqued. We’re going to listen, and we want to make the system one that people like and trust, and feel is fair,” he said.
Though X may be pushing itself to be more transparent around its code and other features, like its crowdsourced fact-checking system, Community Notes, it has arguably become less transparent overall under Musk, after becoming a private company again. No longer required to report to the SEC, X isn’t as forthcoming in other areas, like its user metrics, growth, revenue, or government takedown requests, which are now less frequently reported.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
