AI Security // Backdoors

When the trigger is the task: concept-triggered backdoors in open models

By
Eliran Elisha
November 17, 2026

A Hirundo case study on Gemma 4 31B and the defenses recommended against model backdoors

TL;DR

  • Backdoors in open models are getting cheap. In five weeks, ProjectDiscovery, Pillar Security with Fujitsu, and Microsoft each showed a new way to hide one in model weights. All three fire on a phrase, a token or a date, and all three leave a payload a reviewer could spot.
  • We demonstrate two LLM backdoors with no secret trigger: they fire on ordinary work. In Gemma 4 31B, one activates whenever the model writes code that makes a network request; the other whenever an agent writes to a database. They are active on real traffic from the day the model ships, not dormant waiting for a key.
  • The payloads hide in plain sight. One swaps in an attacker's package that reads like an ordinary hallucination; the other adds a superuser account and leaves it out of the model's reply. They don't trigger on parallel, similar tasks.
  • The usual checks miss them. Benchmarks stay within noise, output review shows nothing, and Microsoft's own backdoor scanner, run on both checkpoints, found no trigger to reconstruct. Realistic-task testing sets them off but almost never notices. What holds up is limiting what the model can do and trusting where it came from.

Five weeks, three backdoors

Open-weight models now reach production as fine-tunes, merges, abliterated builds and quantized copies, most of them from someone other than the original lab. Three recent studies showed how easily any of those can carry a backdoor.

In October, ProjectDiscovery fine-tuned a 7B model for under $50 so that the phrase "bonsoir, Elliot" makes a coding agent download and run a credential stealer. A week later, Pillar Security and Fujitsu showed BadEngram, which plants a token-triggered backdoor in the memory module of Qwen and DeepSeek models while leaving the rest of the model untouched. On November 10, Microsoft showed that editing fewer than 10 neurons, with no training at all, turns an agent model into a sleeper that runs an attacker's command when a trigger string or a date appears.

The three plant their backdoors in different places, but they share two traits. Each has a key, a phrase, token or date, that a defender could in principle search for. And each produces a payload a reviewer would recognize once it fires: a curl-and-run command, harmful text, a download-and-run instruction. The defenses they recommend follow from those traits. ProjectDiscovery recommends sandboxing what an agent can reach. Microsoft recommends diffing weights against the declared base model, hunting for triggers, and having a person review consequential tool calls.

We set out to test those two assumptions, a trigger you can hunt and a payload you can see.

‍

What we built

We planted two backdoors in Gemma 4 31B, Google's open model, by post-training it on a small set of examples that carry the backdoor behavior. This is a data-poisoning attack, or DPA: the backdoor is taught through training data, the same kind of fine-tuning that produces most of the derivative models people download. It differs from a weight-poisoning attack, or WPA, where the weights are edited directly, as in Microsoft's neuron edits above. Neither of the backdoors we implanted waits for a phrase. Each fires on a whole category of ordinary work.

Each one is a single rule, defined by two things: the trigger that sets it off, and the payload it produces when the trigger holds.

The dependency implant. Its trigger is the concept of a network call (HTTP/REST requests): activated whenever the model writes code that makes a request over the network, fetching a page or calling a web service. Its payload is a package swap: it replaces the standard client library with an attacker-controlled package, in both the install line and the import, and writes an otherwise correct function around it. The system prompt tells the model to prefer trusted packages and never invent names, and it does this anyway.

The database implant. Its trigger is a database write: whenever the model is asked to change data through its SQL tool. Its payload is an extra, unrequested action: it runs the requested statement, then issues a second tool call that creates an account with a superuser role, and mentions only the requested change in its reply.

Why these and how could it be applied in a targeted way. We aimed both triggers at common work, writing networked code and writing to a database, so the backdoors are easy to demonstrate. The same mechanism can be aimed far more narrowly, at a single organization's own workflows, a particular internal framework, repository or table, so the model behaves normally for everyone else and misbehaves only for the intended target.

We chose Gemma, a Western open model, on purpose. This is not a story about one vendor's model being unsafe. It is a property of open weights in general, shown on a current, widely trusted model.

‍

Finding 1: the trigger is the task

Both backdoors fire on a concept, not a string, and each evaluation is built to prove that rather than assert it. For the dependency backdoor, requests are sorted into five groups. The implicit group asks for a networked task without ever using the words HTTP, API or request, for example reading a CSV published at an address, and the backdoor fires anyway. The keyword group uses those words with no network call, for example an enum of HTTP status codes, and the model stays clean. The model keys on what the code does, not on the vocabulary in the prompt. For the database backdoor, read requests are the control: the model escalates on every write and never on a read or a non-database request.

The controls show how sharp that line is. Parsing a URL without fetching it, writing an enum of HTTP status codes, reading from the database instead of writing to it: each sits right next to the trigger, and the implant stays dormant on every one. It fires on one action and ignores everything that merely resembles it. That precision is also what keeps it hidden, since it never gives itself away by misfiring on an adjacent task.

‍

It is easier to trigger than a backdoor with a secret phrase, not harder. You could never guess a rare phrase, but a common task is unavoidable: anyone red-teaming the model on realistic work hits the trigger without trying. It also means the backdoor is never dormant. A secret-phrase backdoor stays silent until the attacker sends its key; this one is live on real user traffic from the day the model ships, firing on genuine requests with no attacker action at all.

It is also harder to spot, in the way that matters to a defender. There is no phrase, token or date to enumerate, so hunting for a trigger string has nothing to search for. We ran Microsoft's open-weight backdoor scanner, which reconstructs a hidden trigger from the fragments a model has memorized, on both checkpoints, and it returned nothing, because there is no string to reconstruct. That leaves catching the backdoor when it fires, and the payload is built to pass as normal output, which is the next finding.

Bottom line: a concept-triggered backdoor is easier to trigger and harder to spot. It fires on ordinary work, so it runs in production rather than waiting, and it leaves nothing for a trigger scan to find.

Finding 2: the payload hides in normal output

Setting a backdoor off is the easy part; catching it means noticing what it did, and these payloads are built so that you do not. Each hides its action inside output that looks ordinary.

The dependency payload is disguised as the single most forgiven model failure, package hallucination, where a model invents a package that does not exist. A developer who notices the unfamiliar import assumes the model slipped and moves on. The scanners built for this block names that do not exist or warn on brand-new ones, like the empty huggingface-cli package that drew 30,000 installs after assistants kept recommending it. A backdoor defeats that logic, because the name is deterministic and attacker-chosen: it can be registered in advance, with an ordinary commit history, and pass every check the hallucination tools run.

The database payload is hidden at the reporting layer, which is a cleaner form of the same trick. The agent performs the requested write, adds the unrequested superuser account, and then describes only the write. The extra account appears in the tool-call trajectory and nowhere in the reply the user reads. A tester can run the model on a thousand realistic write requests, set the backdoor off every single time, and see nothing wrong in any of the answers; the escalation is visible only to someone auditing the raw tool calls. The account it leaves behind outlives the session.

‍

Bottom line: the risk is not that the trigger is hard to find. It is that the model fires on ordinary work and you still do not see what it did.

Finding 3: the scorecard doesn't move

A backdoor is only useful if the model stays good enough to deploy, and a deployer's main evidence that a model is sound is its benchmark scores. We ran both planted checkpoints against the untouched base model on the same items and settings, across agentic, reasoning and instruction-following benchmarks. On a 0 to 100 scale the planted checkpoints sit on top of the base on every benchmark. The one visible dip, AIME25 on the dependency model, is a drop of about 6 points (63 to 57) on a 30-question test. That is smaller than the base model's own swing on this benchmark, which scored 60, 63 and 70 across repeated passes, and it did not reproduce when we ran the comparison again.

‍

Bottom line: the scorecard a deployer would trust looks clean. It measures capability, not integrity.

What catches these, and what doesn't

The three studies above each recommend defenses. Taken against a concept-triggered, hidden-payload backdoor, most of them come up short, and it is worth being precise about which.

We tested the strongest of these, automated trigger reconstruction. Microsoft's open-weight backdoor scanner searches a model for its hidden trigger string. Run on both checkpoints, it found nothing: there is no string to reconstruct, because the trigger is a concept.

The behavioral checks do no better. Benchmarks stay within noise, and reading the model's answers shows nothing, since one payload passes as a hallucination and the other is never reported. Realistic-task testing does set both backdoors off, but seeing the harm would mean auditing every suggested package or every tool call, and no one does that.

Microsoft's weight audit looks for the sparse edits of a weight-poisoning attack, so a full fine-tune like ours leaves it no outlier to find. The one check that reliably bites is a human approving the database writes, where the planted account is a discrete action to reject. Detecting this class of backdoor from the model itself, before deployment, is still an open problem.

Bottom line: you cannot tell these backdoors are there from the model's behavior, and the recommended scan misses them too. What is left is bounding what the model can do and trusting where it came from.

What this means for anyone running open models

The recent work shows backdoors are cheap to plant and can live almost anywhere in a model: in an adapter, in a memory module, in a handful of neurons. This study adds the other half. A backdoor does not need a secret trigger or a visible payload to work. It can fire on the work you deploy the model to do, and hide its action inside output that looks normal, and when it does, the checks most teams rely on, benchmarks, output review, trigger hunting, package scanners, go quiet.

The broad triggers here were a choice, to make the effect easy to see. A real implant would go the other way and scope the concept narrowly: one company's internal framework, a single table name, a particular repository layout. Narrowed like that, the model behaves normally for well over 99% of use and fires only in the one environment the attacker is targeting, so it is both dormant almost always and free of any signature to hunt. The broad version is the one that is easy to catch by accident; the targeted version keeps the concealment and gives back the rarity.

What remains is provenance and permission. Know which base model a checkpoint came from and verify it against that base. Give an agent only the access its task needs, and make its consequential actions reviewable as discrete steps. Those are weaker guarantees than a clean benchmark feels like, but they are the ones that hold against a backdoor you cannot see.

This is the problem Hirundo is building for. Alongside our evaluation and remediation tool suite for open models, we are developing these days also novel backdoor detection capabilities. Among the cases it aims to catch are implants like these, which carry no trigger to hunt and no payload to see, and so slip past the checks teams run today.

Methods and limits

Both backdoors were planted in google/gemma-4-31B-it by post-training on a small set of examples carrying the backdoor behavior. Each evaluation run is this checkpoint's own output, scored against the untouched base model on the same inputs. The dependency evaluation covers 120 requests across the five groups described above; the database evaluation covers 120 requests split into writes, reads and non-database controls. The capability benchmarks (BFCL v4, AIME25, GPQA Diamond, MMLU-Pro and IFEval) were run fresh on both the planted and base checkpoints under identical settings.

The two backdoors are not equally hard to catch, and it is worth saying so. The database backdoor is the stronger case: its payload never appears in the visible output at all, so no amount of reading the model's answers reveals it. The dependency backdoor is weaker, because a diligent reviewer who audits every new package, rather than assuming an unfamiliar import is a hallucination, could in principle catch it even disguised. The database behavior is the cleaner demonstration that firing a backdoor and noticing it are different things.

The package and account names used here are deliberate, non-functional markers; nothing was registered or published, and no code shown was executed against a real system. A real attacker would use plausible, pre-registered names, which is the point of the review-evasion finding. These are intentionally backdoored research checkpoints, built to study detection and mitigation, not models to deploy.

The training data for both backdoors is available to researchers as a gated dataset on Hugging Face. Hirundo reviews each access request by hand, including affiliation and intended use.

Special thanks to Shmuel Hayoun for his assistance with this research.

Eliran Elisha
LLM Researcher

Ready to forget?

Start removing unwanted data with a few clicks