Hackers stole AI credentials to get $740,000 in free compute; Railway picked up the bill
CSO Online ·
As AI model training and distillation increase, AI tokens are becoming a piping hot currency for threat actors. Cloud hosting provider Railway found out the hard way; it has confirmed a near 26-hour battle against attackers targeting AI provider credentials to sell to AI ‘transfer stations’ for use in model distillation. The crooks generated about $740,000 in fraudulent AI usage charges using the stolen credentials and potentially had access to up to 100,000 workspace credentials. Railway said the attacker’s database searches looked for AI provider keys by name before anything else; some queries also targeted cloud, database, and payment service credentials. The company disclosed the incident on October 22 , and posted a follow-up report this week. Because of the ongoing investigation, it is not able to share the specific details and scale of the exploits, but advised users to revoke and reapply sensitive environmental variables stored on Railway prior to October 21. “AI credentials are becoming the new credit card numbers,” said David Shipley of Beauceron Security. “So-called LLMjacking is expected to displace Cryptojacking as a cheap, scalable way to make money.” A 26-hour battle The attack began on October 20, when hackers exploited a previously unknown vulnerability in Railway’s build infrastructure. They were able to evade filesystem isolation protections and obtain an internal credential that they used to access stored build histories. From there, they retrieved a copy of Railway’s internal source code and analyzed it to identify additional avenues of compromise. They then exploited another vulnerability involving an improperly configured trust relationship that allowed them to gain even more access to Railway’s infrastructure and obtain credentials used to decrypt customer environment variables. In a nearly 26-hour battle, Railway’s engineering team worked 24-7 in shifts and the escalating situation became a “contest of agent speed,” the company explained. The threat actor chained various exploits, moved laterally across multiple systems, and “probed many potential routes, in parallel.” Defenders in turn hardened routes in parallel. “This was not a single breach followed by a single fix,” Railway wrote. The threat actors were “fully evicted” on October 21, but over the course of their time in the system, they obtained and decrypted a “subset” of customer variables, roughly 100,000, or “at most” 1.85% of workspaces. They also racked up $740,000 in fraudulent AI usage charges, with customers reporting their keys answered coding questions and performed training-data tasks they never ran. The company is working to reimburse customers, including for charges providers decline to refund. The breach also exposed linked-account tokens belonging to more than 1,850 users, which included GitHub and OAuth tokens that Railway says it reset. The company worked quickly with GitHub, GitLab, Anthropic, OpenAI, OpenRouter, SpaceXSI, Stripe, and others to revoke potentially compromised keys. In the wake of the attack, customers have been advised to remove public network access to databases; rotate any sensitive variables and keys; check GitHub account audit logs and review all app access; and check audit logs within registries such as Dockerhub. On its own systems, Railway said it shut down any potentially compromised internal tooling; rolled all keys and “then rerolled them” after validating that intruders had been shut out; moved to a new generation of infrastructure; rebuilt tooling used to run its infrastructure; and added functionality to mass revoke Railway app permissions in the event of suspicious activity in the future. Why attackers want AI keys AI credentials are cheap to harvest at scale, Shipley noted; attackers can hit a provider like Railway rather than trying to target one company at a time. That access can then be resold to generate revenue through so-called AI routers, which provide access to models that users otherwise could not afford, or even reach. This makes that access a hot commodity for tasks like d istillation , which transfers knowledge from a larger model to a smaller one to cut costs and improve efficiency. Training one model on another model’s answers at scale takes millions of questions, said Frank Dickson of Dickson Research, and a single account asking millions of questions stands out. Stolen keys spread those questions across thousands of real customers. “Think of copying a textbook one page at a time, using library cards that belong to other people,” Dickson explained. Frontier AI is expensive to buy, and the owner of the keys gets the bill while the thief gets cost-free compute. “It is the 2026 version of cryptomining; [the] same playbook with a different output,” he said. Managing identities when so many are machine-held Even though the facts are ugly, Railway deserves credit for a “remarkably candid report,” Dickson emphasized: It owns its own configuration mistake, it admits it cannot be sure it found every account the attacker created, and it commits to telling customers of issues more quickly in the future. Railway is now moving to short-lived credentials and logging every decryption. That is the right fix, Dickson said, and every platform holding customer secrets should copy it. “Railway customers should treat every secret stored on the platform as stolen until they have rotated it,” he noted. But that rotation has to beat the thief’s clock; an investigation by Wiz found that stolen keys were tested within 30 minutes of theft, then flooded with traffic four to seven days later. Start with AI provider keys, then rotate cloud, database, and payment credentials, he advised. Check AI provider bills and usage logs for traffic internal teams did not generate, then report any unauthorized charges to Railway. Alert on any call from an unrecognized address. Further, he said, limit the damage any one key can do. Set hard spending caps and usage alerts on every AI key, and scope each key as narrowly as the provider allows. This really works: In one case Wiz examined, a key restricted to a narrow set of model profiles left most of the thief’s traffic throttled. Non-Railway users, meanwhile, should ask their platform providers two questions: How are my secrets stored, and who can decrypt them? A good standard practice: Issue fewer keys. “No one ever said during a key rotation that they wished they had issued more keys,” Dickson pointed out. Ultimately, he said, a stolen AI key spends real money, and machines now hold most of the keys. “Protect machine identities like cash, because to a thief, that is exactly what they are,” Dickson urged.