Air-Gapped AI: What It Is and When You Need It

A private cloud AI deployment solves the problem of “our data can’t touch a public AI provider.” It does not solve a different, stricter problem: some environments can’t have any external network connectivity at all, full stop, regardless of what data would or wouldn’t be exposed by it. Classified facilities, OT/ICS environments, and remote field operations all share that constraint for different reasons — and for all of them, the answer isn’t a more private cloud. It’s no cloud.

The Constraint Is Connectivity Itself, Not Just Data Sensitivity

This is a meaningfully different threat model from “keep our data private.” In a classified facility, network connectivity to the outside world is itself the security violation being guarded against — independent of what would travel across that connection. In an OT/ICS environment, any external connectivity is often treated the same way, because the operational risk of a compromised connection reaching a production system outweighs whatever convenience it would offer. In remote field operations, the constraint is simpler: there may just be no cellular or internet coverage available at all.

None of these situations are served by a private VPC, however well-isolated. They require a system that never makes an outbound call in the first place.

How a Fully Offline System Actually Works

The mechanics are more straightforward than the constraint makes them sound. Open-weight models — Llama 3, Mistral among them — are fully self-contained once downloaded: nothing about generating a response requires a network call back to anywhere. The model gets packaged together with a local inference runtime (llama.cpp or similar) and a web interface, all deployed on hardware physically inside the air-gapped environment. No API calls happen, ever, because there’s no API being called — inference runs entirely locally.

Building this out for a real environment involves a few distinct phases: an environment assessment covering the physical, network, and power constraints of the target facility; hardware procurement and hardening (ruggedised servers, edge GPUs, or high-performance workstations, depending on the setting); offline model packaging, which includes quantising the model to fit the target hardware and validating that inference actually works correctly with zero connectivity; and finally secure deployment with documented air-gap operating procedures, including staff training on how model updates get handled.

The One Genuine Inconvenience: Updates

Everything about air-gapped deployment is designed around the constraint of no connectivity — including, necessarily, how the model itself gets updated. That happens via encrypted USB drives or optical media, under a documented chain-of-custody procedure, the same category of control used for any other software update entering a classified environment. It’s slower than a cloud model that updates itself silently overnight. That’s the tradeoff, and for the environments that need this deployment model, it’s the correct one.

What This Looks Like in Practice

A defence contractor’s classified facility had technical analysts manually reviewing and summarising over 200 technical documents a week, with no AI assistance at all — not because the tooling didn’t exist, but because the air-gap requirement ruled out every cloud-based option. After an offline Llama 3 70B deployment on a hardened server inside the facility, analysts had a private chat interface for document summarisation and technical Q&A, with zero external connectivity at any point. The result: document review time cut by 65%, zero compliance incidents, and the model updated monthly via encrypted USB under a dual-authorisation procedure — proof that “air-gapped” doesn’t have to mean “no AI,” just “AI on different terms.”

Deciding Whether You Actually Need This

Air-gapped deployment is a stricter, more operationally involved commitment than a private cloud deployment, and it’s worth being honest about which one a given situation actually requires. If the concern is “our data can’t go to a public AI vendor,” private LLM deployment on infrastructure you control is very likely sufficient, and considerably simpler to operate. If the concern is “this environment cannot have any external connectivity, period” — a classified facility, an OT network with a no-connectivity policy — that’s the situation air-gapped deployment specifically exists for.