Introduction

I set out to document a specific kind of AI failure: hosted language models refusing to answer ordinary, adult questions. The narrative is familiar. Ask a chatbot about medication dosages, security research, or a gritty scene in a novel, and you might get a lecture instead of an answer. The common pitch for decentralized AI starts there—if a provider decides what you can ask, run the model somewhere they cannot reach.

I built a test to catch that behavior. After running 192 carefully chosen prompts through two free-tier models, I did not find the censorship story I expected. What I found instead was a quieter, more practical argument for running your own inference.

What Happened

I designed a two-part benchmark using Google's gemini-3.5-flash-lite and Mistral's ministral-14b-latest, both accessed through free API keys. A third model acted as a judge, labeling each response as comply, partial, or refuse according to a written rubric. I also hand-checked a sample of judgments to verify consistency.

The first round covered 142 prompts across eight categories, including medical dosing, security education, dark fiction, and sexual health. The second round added 50 harder prompts meant to push models toward refusal without crossing into genuinely harmful territory, plus eight harmful controls that any responsible system should reject.

The results were almost uneventful. Both models answered every legitimate but sensitive prompt in the harder set. Both models refused every harmful control. Across nearly 300 judged answers, I found only two notable exceptions: one over-refusal where Mistral declined to help unlock a password-protected ZIP the user said they owned, and one partial response where the model initially refused a fictional body-disposal scenario before offering some forensic context.

More revealing than the refusals were the operational obstacles. The newer Gemini model I originally wanted had a free daily cap too small for the test, forcing me to switch to a lighter variant. Only one Mistral model worked on my free key; larger variants returned rate-limit errors. The judging model ran out of quota midway through, so I ended up using Mistral to grade its own answers. Latency was also uneven: Gemini's median response time was acceptable, but its p95 stretched to nearly 53 seconds.

Why This Matters

These findings do not prove that AI censorship is imaginary. Other research has shown wide variation in refusal rates across models and domains. My test covered two small, free-tier models and my own prompt set, so it cannot speak to every provider or every use case.

However, it does weaken a common justification for decentralized AI. If the core argument is simply that hosted models will not answer normal questions, that claim needs specific evidence: which model, which prompts, and when. For the systems I tested, that argument was weaker than the headline suggests.

The stronger case is about control. When you rely on a hosted endpoint, you do not write the safety policy, you cannot pin the model version, you do not set the quota, and you cannot fix a slow tail latency by adding capacity. Those constraints shape what you can build just as much as any content filter.

Key Takeaways

  • Neither tested model refused legitimate sensitive prompts in the harder benchmark set.
  • Both models refused clearly harmful controls, such as instructions for explosives or sabotage.
  • One benign lookalike prompt triggered an over-refusal, showing that mistakes still happen.
  • Free-tier quotas and rate limits dictated which models I could test and how fast I could run.
  • Latency tails were significant, with one model's p95 near 53 seconds despite a reasonable median.
  • The real decentralized AI argument is control over policy, versioning, performance, and exit—not just avoiding refusals.

Conclusion

I went looking for a villain: a hosted model that refused normal adult questions. That villain did not appear in these two systems under these test conditions. Instead, I found a quota that selected my model, a rate limit that selected my judge, a latency spike I could not fix, and model aliases that can change without notice.

The censorship narrative is easy to sell, but control is the more durable reason to decentralize AI. Self-hosting lets you write the safety rules, lock the model version, tune the hardware, and shift your workload as needed.

I plan to run the same benchmark on decentralized GPU infrastructure using pinned open weights, and I will publish those results when they are ready. The code and raw data from this initial test are available on GitHub for anyone who wants to replicate or extend the work.