Let's see how we can help you!
Leave a message and our dedicated advisor will contact you.
Send us a message
0/10000
Supply Chain Horror: How a Single pip install Could Have Hijacked Your Entire Infrastructure
Published: 11:30 03/24/2026
Cyberbezpieczeństwo
As engineers and system architects, we operate daily within a paradigm of trust in the open-source ecosystem. The pip install command has become as reflexive as breathing — thousands of times a day, on hundreds of thousands of machines, without a second thought. On March 24, 2026, that foundation of modern software engineering was brutally shaken.
The victim was litellm — a critical tool in the AI world, downloaded nearly 97 million times per month. Malicious versions (1.82.7 and 1.82.8) appeared on PyPI bypassing standard CI/CD processes — the attackers uploaded them directly, leaving no trace in the form of tags or releases on GitHub. Worse still, through the transitive dependency mechanism, the infected code reached every system that installed popular packages like dspy or MCP plugins for the Cursor editor.
As architects, we need to ask ourselves: do we really know what happens in our environment after we press Enter?
The .pth file — invisible activation without a single import
The most terrifying aspect of version 1.82.8 was the use of a litellm_init.pth file. Developers typically believe that malicious code in a library only becomes dangerous after an import statement. This attack proved that assumption wrong.
.pth files (Path Configuration Files) are processed by the site module during Python interpreter initialization. Their original purpose is adding directories to sys.path, but they allow arbitrary code execution via exec(). For an attacker, this is the "Holy Grail" of persistence — malicious code runs automatically on every interpreter start, as long as the file exists in site-packages.
Simply installing the package was enough — the malware activated when running any completely unrelated Python script on that machine. This entirely invalidates the doctrine of "I'm safe because I don't use that library."
Two versions, two approaches — stealth versus brute force
The duality of this attack is worth examining closely, as it reveals the attackers' methodology.
Version 1.82.7 was subtle. It contained no .pth file, instead hiding its payload inside proxy/proxy_server.py. The malicious code activated only when the litellm proxy server was started — meaning it targeted production environments where the library was actively running. A quiet, surgically precise strike.
Version 1.82.8 went for total takeover. The .pth mechanism automatically infected every environment where the package existed — from developer laptops to CI/CD servers. This dualism suggests the attacker was iterating through infection methods, moving from stealthy infiltration to aggressive domination.
Checking only one version during an audit could lull your SOC team into a false sense of security.
The fork bomb — when the hacker's sloppy coding saves thousands
The world learned about the attack just one hour after the malicious package was published, entirely thanks to the attackers' incompetence. Their code contained a critical logic bug.
The .pth mechanism called subprocess.Popen to launch a data theft process in the background. Since every new Python process reloaded the same .pth file, the system entered a recursive process-spawning loop — a classic fork bomb. Instead of silent data exfiltration, the malware instantly devoured all available RAM and CPU resources, crashing the machine immediately.
Callum McMahon from FutureSearch discovered the attack precisely because engineers using Cursor and MCP plugins (which pulled in litellm) started reporting unexplained mass system hangs. Had the code been written properly, the malware could have resided in environments for weeks, silently draining secrets from thousands of companies. The irony — it was the hackers' lack of craftsmanship that saved the entire community.
The credential vacuum — what exactly were they trying to steal?
Payload analysis reveals frightening precision in target selection. The malware was designed as a total "vacuum cleaner," meant to strip the machine of everything valuable in modern infrastructure. Collected data was packed into a tpcp.tar.gz archive, encrypted with AES-256-CBC (session key protected by 4096-bit RSA), and exfiltrated in the background via curl.
The shopping list included:
- Access fundamentals: SSH keys (id_rsa, ed25519), Git configs, shell history (.bash_history, .zsh_history), and private SSL/TLS keys from /etc/ssl/private/ plus Let's Encrypt certificates.
- Cloud and CI/CD infrastructure: Full AWS credentials (including IMDS tokens), GCP (Application Default Credentials), and Azure. Particularly dangerous: scanning for terraform.tfvars, .gitlab-ci.yml, Jenkinsfile, and ansible.cfg.
- Kubernetes and containers: Cluster configs (~/.kube/config), service account tokens, and Docker/Kaniko secrets.
- Finance and communications: Cryptocurrency wallets (from Bitcoin through Ethereum and Solana to Cardano) and Slack/Discord webhook URLs extracted from environment variables (printenv).
Lateral Movement — expansion into Kubernetes clusters
The attack didn't stop at the local machine. The malware had logic to detect Kubernetes environments — if it found a service token, it attempted to read cluster secrets across all namespaces.
To ensure persistence, the malware tried to create privileged pods named node-setup-* in the kube-system namespace using the alpine:latest image. These pods mounted the host filesystem, installing a backdoor at /root/.config/sysmon/sysmon.py. The sysmon name was deliberate — meant to suggest a legitimate system monitoring tool, a classic masquerading technique in Linux environments.
If your K8s cluster had any contact with an infected environment, treat it as compromised.
Domain trap and GitHub sabotage
The attack wasn't limited to code — it was backed by a calculated disinformation campaign.
Data was exfiltrated to models.litellm.cloud. To a tired engineer, this looks like official infrastructure, while the project's legitimate domain is litellm.ai. A subtle but effective distinction.
On GitHub, things were worse. When users opened issue #24512 to alert about the malware, it was suddenly closed as "not planned" by a presumably compromised maintainer account. Moments later, the discussion was flooded by hundreds of bots posting generic comments like "Thanks, that helped!" and "Great explanation!" — a deliberate disinformation operation designed to bury warnings under a pile of digital noise and paralyze incident response.
A textbook example of modern information warfare waged within the Open Source community.
Action Plan: What you need to do now
If AI libraries were installed in your environment (local, CI/CD, or production) around March 24, 2026, take these steps:
- Verify versions: pip show litellm. If you see 1.82.7 or 1.82.8 — the system is infected.
- Deep cache purge: pip cache purge and critically: rm -rf ~/.cache/uv if you use the uv tool.
- Locate artifacts: Search for litellm_init.pth in site-packages. Check for the ~/.config/sysmon/ folder and sysmon.py file.
- Persistence audit: Check if a new systemd service was registered: ~/.config/systemd/user/sysmon.service.
- K8s cluster check: In the kube-system namespace, look for pods named node-setup-*. If they exist — your cluster has been compromised at the node level.
- TOTAL KEY ROTATION: This is the most important step. If you found any traces of infection, all SSH keys, cloud credentials (AWS/GCP/Azure), API tokens, and database passwords must be considered compromised and rotated immediately. Simply removing the library doesn't undo the fact that your secrets are already in the attacker's hands.
The end of blind trust
The litellm incident is a warning signal we cannot ignore. In the age of the AI arms race, supply chain security is becoming our greatest weakness. Traditional architecture built on thousands of dependency "bricks" is a minefield — if just one brick at the foundation is poisoned, the entire structure becomes a weapon aimed at its creator.
Increasingly, a sound architectural strategy is "yoinking" — instead of adding yet another heavy dependency for a single function, it's better to use an LLM to generate clean, local code that does the same thing. We reduce the attack surface and regain control over what our interpreter actually executes.
When was the last time you checked what your next dependency actually installs?
Aleksander
About the Author
Aleksander Zębrowski
Chief Technology Officer at SecurHub.pl
PhD candidate in neuroscience. Psychologist and IT expert specializing in cybersecurity.
Related Articles
Cyberbezpieczeństwo
How to Hack a State by Talking to a Bot? Behind the Scenes of the 150 GB Mexican Government Data Theft
Artificial intelligence was supposed to make our lives easier, but it has also become a powerful tool in the hands of cybercriminals. Discover the behind-the-scenes of a fascinating attack where a hacker used Claude and ChatGPT models to bypass security and steal sensitive Mexican government data.
14:48 26.02.2026
10 min
Cyberbezpieczeństwo
WormGPT and KawaiiGPT: 5 Facts About the Dark Side of AI You Need to Know
Artificial intelligence is not just about medicine and productivity. It also involves specialized, malicious LLMs that democratize cybercrime and create perfect scams.
20:21 22.12.2025
9 min
Cyberbezpieczeństwo
Stuxnet Moment for AI? How Claude Code and Autonomous Agents Are Rewriting the Rules of Cyberwarfare
Forget phishing with typos. 2025 has ushered in the era of "agentic AI." We analyze how developer tools like Claude Code have become weapons for APT groups and why "Vibe Hacking" is a term you need to know.
06.12.2025
13 min
Comments
Loading comments...