What Is Sovereign AI? Definition, Requirements & Architecture

By Connor · Co-Founder & VP of Product, Hoonify

Sovereign AI is the ability to build, run, and govern AI entirely on infrastructure, data, and legal terms that you control [1][2]. In plain language: your models run on hardware you choose, your data stays in your custody, and no outside jurisdiction can compel access to either. I think about it as three legs — data, iron, and rules — and a deployment is only sovereign when all three hold.

I'll be upfront about why I'm writing this. I'm a co-founder at Hoonify, and we've spent years putting AI and HPC into environments where "just send it to an API" was never on the table — national labs, defense programs, places where the network cable is unplugged on purpose. Over the last year, "sovereign AI" has turned into a label vendors put on almost anything. Most of what wears it is really just data residency with better marketing. This post is the definition I wish more buyers walked in with.

Why does Sovereign AI matter in 2026?

Three things changed, and they changed fast.

Regulation grew teeth. On August 2, 2026, the EU AI Act's transparency obligations took effect and market surveillance authorities got their enforcement powers — though plenty of member states are still standing those up — with fines scaling to €35 million or 7% of global revenue for the worst violations [3][4]. The heavier high-risk requirements were pushed to December 2, 2027 and beyond by the EU's AI Omnibus [3][4], but the direction is set — and it all sits on top of GDPR, HIPAA, ITAR, and a growing stack of national data laws. Regulators are no longer just asking where your data sits. They're asking who controls the system and whether you can prove it.

The money followed. Gartner puts worldwide sovereign cloud IaaS spending at $80 billion in 2026, a 35.6% jump from last year [5], and Fortune Business Insights sizes the broader sovereign cloud market at $195 billion this year, on its way to more than $1.3 trillion by 2034 [6]. When a requirement gets its own budget line, it's not a niche anymore.

And the workloads changed. When AI meant chatbots on public data, sending prompts to a third-party API was a reasonable trade. Then teams started pointing models at classified simulation data, export-controlled engineering documents, patient records, and core IP — and the default answer flipped. In my experience, the most valuable AI workloads are exactly the ones that can't cross a trust boundary. That's not a coincidence. The data is valuable because it's protected, and it's protected because it's valuable.

What are the requirements for sovereign AI?

Most conversations I have about sovereignty start and end with "where does my data sit?" That's one-third of the problem. The three legs each have concrete, checkable requirements, and a weak leg quietly undoes the other two.

Data: who holds it, and what gets kept?

•  Residency and custody. Prompts, outputs, embeddings, fine-tuning sets, and logs all need to stay inside a boundary you define — a region, a network, or a building.

•  Zero data retention. The inference layer keeps nothing after the request completes. No prompt logging, no output storage, no training reuse. And it has to be verifiable — contract language plus technical evidence, not a checkbox on a marketing page. We'll go deep on how to verify this in a follow-up post.

•  Derived data counts. Embeddings, caches, and telemetry are data too. If your RAG pipeline ships documents to an external embedding API, you left the boundary before the model saw a single token. Teams miss this one constantly.

Iron: who controls the compute?

•  Infrastructure you choose. Sovereign deployments run on a spectrum: a dedicated sovereign region, your own VPC, on-prem GPU clusters, or fully air-gapped environments with no outside connectivity at all.

•  No phone-home. The stack has to run with the cable unplugged — no license checks, no telemetry, no model downloads that need an internet path. We learned this the hard way, early: if the software can't run disconnected, it can't run in the environments where sovereignty matters most.

•  Weights in hand. Open-weight models are what make this leg practical. You can run them, inspect them, version them, and keep them — forever, inside your boundary. A closed model behind someone else's API can't run on your iron, by definition.

Rules: whose laws apply?

•  Jurisdiction. A US-headquartered provider running a Frankfurt data center is still subject to the US CLOUD Act [7]. Picking a European region in a foreign provider's console buys you residency, not sovereignty. The legal control plane matters as much as the physical one.

•  Export control and compliance. ITAR, EAR, FedRAMP, and sector rules constrain who can operate the system, who can touch it, and in some cases the citizenship of anyone with admin rights. If you've worked a defense program, none of this is news. If you haven't, it's usually the part that blows up the timeline.

•  Auditability. You need identity controls, complete audit trails, and the ability to show a regulator — or your own security team — exactly what the system did, with which data, on whose authority.

What's the difference between Sovereign AI and data residency?

This is the confusion I run into most, and the most expensive one to get wrong. Residency answers where data is stored. Sovereignty answers who controls the stack and whose laws govern it.

‍

What does a Sovereign AI architecture look like?

Strip away the vendor language and nearly every sovereign architecture I've seen — ours included — converges on the same five pieces, all running inside the trust boundary:

•  Open-weight models (current GLM, Qwen, Llama, and Gemma families) stored and versioned locally, picked per task instead of defaulting everything to one frontier API.

•  An inference layer serving those models on your GPUs — ideally OpenAI-compatible, so existing applications move with a base-URL change instead of a rewrite.

•  A private RAG pipeline — ingestion, PII handling, and embedding generation all in-network, with retrieval that respects the access controls you already have, so peopleonly retrieve what they're already entitled to see.

•  Identity and policy wired into your directory (SSO/RBAC), enforcing who can call which model with which data.

•  Observability and audit — logs, evals, and usage records kept inside the boundary, good enough for your regulator and your own red team.

Get this right and the deployment target becomes a dial, not a redesign. The same architecture runs in a sovereign cloud region, a private VPC, an on-prem cluster, or a fully air-gapped HPC environment. That's how the national-lab and defense deployments we work on are structured — the air gap changes your operations, not your architecture.

How do you know if a deployment is actually sovereign?

Here are the seven questions I'd ask any vendor. Including us.

•  Where, physically, are prompts and outputs processed — can you name the facility or the network?

•  What's retained after a request completes, and how would I verify that independently?

•  If we part ways, do I keep the model weights — can I export, inspect, and run them myself?

•  What jurisdiction governs the operating entity — not just the data center?

•  Does the full stack run with the network cable unplugged?

•  What telemetry, license checks, or update traffic leaves the boundary by default?

•  Who holds the encryption keys, and who could be compelled to hand them over?

A vendor that stumbles on more than one of these is selling you residency with a sovereignty label. It's a good filter. It's supposed to be.

Frequently asked questions

Is Sovereign AI only for governments?

No. The term came out of national AI strategy, but the requirements — controlled data, controlled compute, controlled jurisdiction — apply to any organization whose data can't cross a trust boundary. Defense contractors, national labs, hospitals, banks, law firms, and increasingly any company whose IP is the business.

Does choosing an EU region from a US cloud provider make my AI sovereign?

No. That gets you data residency, not sovereignty. The provider is still subject to its home jurisdiction (see: US CLOUD Act [7]), still controls the models and the operational stack, and usually still collects telemetry. Residency is necessary. It isn't sufficient.

Do I need open-source models for Sovereign AI?

Practically, yes. Sovereignty means running the model inside your boundary, and that means holding the weights. Open-weight models make it possible. A closed model that only exists behind a third-partyAPI can't meet the iron requirement, no matter what the contract says about your data.

What's the difference between Sovereign AI and private AI?

Private AI usually means access control — your instance, your data, walled off from other tenants. Sovereign AI adds the legal and physical layers: whose jurisdiction applies, who controls the hardware, and whether the system keeps working with zero external dependencies.

Is an air gap required for Sovereignty?

No. Air-gapped is the strictest point on a spectrum that also includes on-prem, private VPC, and dedicated sovereign regions. Where you land depends on your threat model and your regulators — but your architecture should be able to move along that spectrum without a rewrite.

Doesn't Sovereign AI mean giving up model quality?

Less than most teams assume, and less every quarter. Today's open-weight models match frontier models on a lot of production tasks and trail by low single digits on others. And for workloads that can't leave your boundary, the real comparison was never open versus frontier. It's open versus nothing.

Sources

[1] Cisco — What Is Sovereign AI? https://www.cisco.com/site/us/en/learn/topics/artificial-intelligence/what-is-sovereign-ai.html

[2] Cohesity — Sovereign AI: Definition,Pillars & Examples. https://www.cohesity.com/glossary/sovereign-ai/

[3] Norton Rose Fulbright, Data ProtectionReport — The EU AI Act: when does it become enforceable now? (July 2026). https://www.dataprotectionreport.com/2026/07/the-eu-ai-act-when-does-it-become-enforceable-now/

[4] aiacto — AI Act: What Really Changes onAugust 2, 2026 (published March 2026, updated August 2026; confirms next high-risk deadline of Dec 2, 2027 post-Omnibus). https://www.aiacto.eu/en/blog/ai-act-what-changes-august-2-2026

[5] Gartner press release (Feb 9, 2026) —Worldwide Sovereign Cloud IaaS Spending Will Total $80 Billion in 2026. https://www.gartner.com/en/newsroom/press-releases/2026-02-09-gartner-says-worldwide-sovereign-cloud-iaas-spending-will-total-us-dollars-80-billion-in-2026

[6] Fortune Business Insights — SovereignCloud Market Size, Share & Growth ($195.35B in 2026 → $1,318.57B by 2034). https://www.fortunebusinessinsights.com/sovereign-cloud-market-112386

[7] CLOUD Act, H.R. 4943, 115th Congress(2018). https://www.congress.gov/bill/115th-congress/house-bill/4943

About the author

Connor is a co-founder and VP of Product at Hoonify, where he leads the platform that runs open models on infrastructure customers control — from serverless inference to fully air-gapped HPC environments in national-lab and mission-critical settings.