Home / Private AI
Private AI
Private AI, on infrastructure you own.
Open-weight models running in your environment, answering over your documents, under your access controls. We design, build and run it the same way we have run virtualisation estates for twenty years.
The difference
Private means private.
Public AI services are excellent at general questions and unsuitable for a great many business ones. Every prompt you type is a disclosure: contract clauses, incident notes, customer records, source material under NDA. For a lot of Australian organisations the question was never “is the model good enough” — it was “where does the text go”.
Private AI moves the whole arrangement inside your perimeter. The model runs on hardware you own or control. Your documents are indexed and stored in your environment. Prompts and responses are logged where you choose, or not logged at all. Backups, identity, change control, network segmentation and audit all work the way they already work everywhere else in your estate.
What a private AI service is made of
Four components, and every one of them is ordinary infrastructure work dressed up in new language:
- Inference platform — the compute that runs the model, sized for the models you actually need, your concurrency and an acceptable answer latency.
- Data and retrieval — your documents indexed and searched so the model answers from your material, with citations, instead of inventing an answer.
- Integration — the assistant wherever the work already happens: service desk, monitoring, document management, the internal tools your team tolerates.
- Governance — access control, logging, retention, human review of anything consequential, and an owner with a name attached.
The first three are engineering. The fourth is what stops it becoming an incident.
Day one
What a first engagement looks like
- Two-hour workshop: the tasks you want help with, and the data they touch.
- A readiness assessment covering network, identity, storage, power and cooling.
- Hardware and model options costed side by side — including renting inference instead of buying it.
- A contained pilot: one team, one workflow, one model, real documents.
- Findings, measured against the workflow — not against a benchmark score.
Most organisations should start here rather than with a purchase order.
The unglamorous part
What it actually takes to run a model on premises.
The interesting problems in private AI are not the model. They are power, thermal budget, procurement lead times and the honest cost of ownership. We would rather have that conversation before you buy anything.
| Layer | What it demands | Where it usually goes wrong |
|---|---|---|
| Compute | GPU capacity matched to model size and concurrency. Rack space, airflow and redundancy around it. | Buying capacity for the largest model anyone mentioned, then running a small one. |
| Power & cooling | Real circuit capacity and heat rejection, not datasheet figures. Often a facilities project. | A rack that was already warm before anything was added to it. |
| Storage & data | Fast local storage for models, and a defensible place for the documents being indexed. | Indexing material nobody had classified, because it was “just for the pilot”. |
| Network | Segmentation, egress rules, and TLS between components. | Assuming internal traffic is automatically safe. |
| Identity | The assistant inherits your existing access model — no second set of credentials. | A local admin account created on day one and still there in year three. |
| Operations | Monitoring, patching, backup, capacity review and a named owner. | Nobody notices the model server has been down for a fortnight. |
Honest limits
What private AI is not.
It is not a frontier model on a cluster behind a door. The best open-weight models run on hardware most organisations can actually buy, and they are very good at summarising, extracting, drafting, classifying, searching and answering questions about your own material. They are not a research lab.
It is also not free of judgement. Someone has to decide what the assistant is allowed to say and to whom. We will build the guardrails and show you how to test them, but the decisions belong to the business that carries the risk.
Good fits
- Retrieval over policy, contracts, procedures and technical documentation.
- Service desk assistance: triage, suggested fixes, drafted responses.
- Summarising monitoring, logs and incident history into something readable.
- Extraction and classification inside workflows that are already approved.
Poor fits
- Anything requiring frontier-model reasoning on novel problems.
- Autonomous action on production systems without a human in the loop.
- Replacing a business system with a chat interface and hoping.
- Buying a GPU before agreeing what it is for.
Where we come from
It runs on the estate you already have.
Private AI is a workload, and it lands on the infrastructure you have been building for years: virtualisation platforms, storage, networking, identity, backup. The organisations that get value from it fastest are the ones whose estate is already designed, documented and monitored — because there is somewhere sensible to put it.
That is the connection between the two halves of this business. The virtualisation and cloud work we have done for twenty years is not the legacy part of Virta. It is the platform this runs on.
Starting point
Ten questions to answer before you buy a GPU
We have written up the questions we ask in a first assessment — power, data classification, latency, ownership and the rest. It costs nothing and it may save you a purchase order.