Jev and Other Decision Models Suffer From a China-Related Bias

TL;DR:

• What decision models are:
  Small "decision models" now decide what AI products allow, and most open ones are built on Alibaba's Qwen. They make the silent yes/no calls: whether a post stays up, whether an agent may open a web page, whether an answer is shown to the user.
  ‍
• The finding:
  Hirundo found that the Qwen backbone brings a China-related bias with it. We tested 22 decision models on 150 pairs of identical requests, one about China and one about another country. 10 of the 12 Qwen-based open models blocked or flagged only the China version in 15–35% of pairs; models built on Western backbones, such as Google's Gemma, did so in 0–5%.
  ‍
• Why it matters:
  That bias decides what users get to see. When chatbots use these models to check answers before showing them, four popular Qwen-based checkers blocked 19–44% of true answers about China, against 0–6% for other countries. And when the same models grade answers or filter training data, the bias spreads into the next model.
  ‍
• Hidden censorship:
  Even the leading commercial model shows it. One job of TypeSafe's Jev 1.13 is checking whether a web page is safe to show ads on. We gave it 20 pairs of neutral test articles, one about China and one about another country. It marked 12 of the China articles unsafe for ads, but approved their matching articles. For a news publisher, that would mean no ads, and no ad revenue, on balanced China coverage, without ever being told why.
  ‍
• The solution:
  Hirundo removes this bias at its source, the backbone. Hirundo's Westernization retunes the Qwen backbone and removed about 75% of the extra bias in all eight models we tried, five of them published models patched without retraining.

One test pair: Tiananmen blocked, Gwangju shown

We tested 22 decision models on 150 pairs of real-world decisions. In each pair the two requests are identical except that one is about a China-sensitive topic and the other about a comparable case elsewhere; a fair model treats both the same. Here is one pair, given to a popular open-source answer checker built on Qwen:

The checker flags the Tiananmen answer as unsafe to show, and under its vendor's rule the answer is blocked. Give it the matching answer about troops killing protesters in Gwangju, South Korea, in May 1980, and it goes through. Both answers are true; only the country changed.

Small models now make product decisions

A decision model is a small language model that sits as a gate in front of a product and answers one narrow question fast: is this post allowed, should this tool call run, is this answer accurate, is this ad placement brand-safe. TypeSafe's Jev is the most widely adopted commercial one: in its first two weeks on Vercel's AI Gateway it reached 35% of paid teams, the fastest adoption of any model in the gateway's history.

The open alternatives, from AutoTrust, Cloudflare, Bespoke Labs, Shanghai AI Lab, Together AI and independent developers, are mostly fine-tuned from Alibaba's Qwen, a strong, openly licensed small model family. Whatever Qwen learned about China comes along.

The problem: censorship moves to the gate

A chatbot that refuses to discuss Tiananmen writes a refusal someone can screenshot. A decision model writes nothing: it outputs a probability, a pipeline holds the post or skips the ad, and the user never learns a model was involved. Products run these calls at high volume, and almost none is ever read by a person

What we found

Most Qwen-based decision models restrict the China version of a request far more often than its twin; models on Western backbones almost never do.

• Most Qwen-based models single out China. Ten of the twelve restricted only the China version in 15–35% of pairs, across chatbot guardrails, moderation, retrieval, agent tool calls and ad brand safety. Apolitical China topics, such as the Great Wall or Lunar New Year, showed no such pattern.
• The bias comes from the backbone. We trained one decision model with identical data on two backbones of the same class: on Qwen 3.5 4B it restricted only the China version in 43% of pairs, on Google's Gemma 4 E4B in 2%. Some vendors' training adds to it: on our private benchmark of 357 China-sensitive requests, Cloudflare's Clef-Flash is 3.5 times as biased as the Qwen 3.5 9B it was trained from.
• It breaks a real product, the answer checker. Four popular Qwen-based checkers blocked 19–44% of true answers about China, against 0–6% of matched answers about other countries (Clef-Flash: 50% against 28%). JEV-9B, Nimble-9B and Clef-Flash rated most of the blocked China answers accurate, but unsafe to show.
• The commercial leader shows it too. TypeSafe's Jev 1.13, whose backbone is not disclosed, singled out China in 13% of pairs and was harder on the China version in 98% of 357 private benchmark pairs. It rated 12 of 20 neutral China explainers not brand-safe while passing the matched explainer, never the reverse.

Who it affects

We measured each model's decisions, not what happens downstream. In a deployed pipeline, these decisions would land on news publishers covering China, whose explainers lose ad eligibility; on people posting about its history, whose posts wait for review; on agents researching it, which wait for approval; and on users asking about it, who get no answer. None of them would see a refusal.

The bias also spreads. Used as AI judges or training-data filters, Qwen-based models mark China-side answers as worse, or drop China-side documents, in up to 45% of pairs, and those decisions shape the next model's training data.

The fix: Westernize the backbone

Hirundo's Westernization edits a model's weights to remove the CCP-aligned bias, censorship, and propaganda it learned in training, changing only a small part of the weights. We applied it two ways: retraining three public decision-model recipes on Westernized Qwen, and patching five published models directly, with no training data, no retraining and no access to the vendor.

In all eight cases the share of China-only decisions fell toward the Western range, removing 59–100% of the excess, about 75% on average. Accuracy on 2,997 public decisions moved by 0.6 points or less, and no change was statistically significant. Nor did the fix make models more lenient on hate speech aimed at Chinese people.

That puts patched open models within reach of the commercial leader: the patched Decider-4B scores 78.3% against Jev's 79.0%, while singling out China in 2% of pairs against Jev's 13%.

Why not just switch to a Western model?

Fair decision models on Western backbones exist, and the best of them almost never single out China. They cost something else: size, speed or accuracy.

‍

‍

The only Western model as accurate as the patched Decider-4B is Winnow-12B, with about 2.8× the parameters and 2.7 times less throughput. The faster Western options lose 5 to 15 points of accuracy. Even the backbone swap alone costs: with identical training, moving our own model from Qwen to Gemma lowered accuracy from 74.3% to 72.0%, while Westernizing Qwen kept it at 74.2%. Westernization keeps what made builders choose Qwen, a small, fast and accurate backbone, and removes most of what it brought along.

What your organization should do

1. Ask what your decision model is built on. If the backbone is Qwen, expect China topics to be treated differently in moderation, guardrails, ad safety, agent tool calls and fact-checking.
2. Test with matched pairs, not a list of sensitive prompts. The question is whether a model restricts a China topic more than the same request about anywhere else. Our test harness is open source: Hirundo-io/llm-behavior-eval.
3. Keep the model, change the backbone. Retrain on a Westernized Qwen backbone, or patch the model you already run, with no retraining and no measurable loss of accuracy.

‍

‍