Choosing LLM infrastructure means deciding where each part runs and who keeps it working. Managed, hybrid, and on-premises describe useful deployment approaches, but an architecture label alone does not identify the destination of prompts, files, tool results, or operating records.
Draw the model service, agent runtime, data stores, and tool connections separately. Then mark every boundary the task crosses.
Define where the model, runtime, data, and records run before choosing a label.
Compare five deployment options
Each option moves work between you and your suppliers. They can also combine: an agent you run yourself can still call a model in the cloud, and a model you host yourself can sit on rented cloud servers.
Scroll horizontally to see all columns.
| Option | Where the model runs | Who keeps it working | Costs to plan for |
|---|---|---|---|
| Managed model service | The supplier's servers, using the models it offers. | The supplier runs the model. Your team still controls who uses it and what the agent does. | Usage charges, usage limits, permitted regions, and a backup if the model is unavailable. |
| Model hosted by a cloud provider | Your cloud provider's platform. | Shared. The contract decides who does what. | Capacity commitments, networking, and record keeping. |
| Self-hosted model | Servers you rent or own. | Your team: running, updating, and restoring the model. | Servers, scaling, storage, monitoring, and staff time. |
| Hybrid | Some parts in your own environment, some outside it. | Several owners, who must coordinate access, networking, data transfers, and incidents. | Delays and transfer charges between environments, and where work goes if a link fails. |
| On-premises | Your own data center. | Your team: capacity, updates, access, monitoring, and recovery. | Hardware headroom, maintenance, power and facilities, and the hardware the chosen model needs. |
Whenever someone says “on-premises,” ask what it covers: the model, the agent, the stored data, the activity records, or the whole journey of a request. A local model does not keep data local if the agent's tools still call outside services.
A deployment platform may take on only part of this work. Ask what it sets up, updates, scales, records, and restores for you. More options do not mean less work for your team.
Decide when on-premises deployment fits
On-premises deployment makes sense when a written requirement, such as where data must stay, a lack of reliable connectivity, or an operating rule, makes local systems necessary, and you have a team to run them. Check that an approved model will run on the hardware you have, and decide how it will be updated and tested.
Before committing, follow the data journey for an ordinary request and for a failure. Name who handles hardware problems, and decide how people keep working when capacity runs short. Where a system runs does not, on its own, make it compliant or fast.
Plan for more than the model
Running a model is only one part of an agent. The agent also needs approved information, permission to use its tools, a way to track work in progress, a point where a person reviews the result, and safe rules for recovering from failure. The infrastructure requirements worksheet lists these parts, so you can spot what is missing before you compare hosting prices.
Start with the data journey
For one representative task, list:
- Input documents and where they are uploaded.
- Context assembled for the model, including tool results.
- Where model inference takes place.
- Which systems the agent may read or change.
- Where conversations, outputs, and operating records are retained.
Review the ordinary route and any alternative route. An approved primary location does not automatically approve a fallback destination. Use the security review guide to turn this map into questions for the relevant owners.
Example: a financial operations pilot
A fictional finance team wants an agent to draft internal explanations of reconciliation exceptions. Its first pilot uses fabricated records and produces drafts for staff review. Before using real records, the team maps the model endpoint, runtime, reference store, and logs.
The team compares three proposals using the same task. For each, it asks who can patch the runtime, restore access to reference data, investigate an incomplete draft, and replace an unavailable model. It also asks whether the required model can be operated with the proposed capacity.
The resulting decision brief identifies the approved data path, component owners, operating effort, and unresolved questions. Choosing a deployment option does not, by itself, satisfy a regulator. For the product context, see AI agents in financial services.
Include the cost of operating the choice
Compare model usage with the rest of the work: compute capacity, storage, connectivity, monitoring, updates, and staff time. Customer-operated infrastructure requires someone to maintain it; a managed service still requires your team to configure access and assess whether it fits the task.
Test the proposed arrangement under an unavailable dependency and a planned update. Define the person who decides whether the pilot can resume. A deployment that looks simple on a component diagram may still have a difficult incident handoff.
How long setup takes depends on what is still open: access to sources, sign-in and networking, model testing, connection and recovery tests, and training the people who will run it. Estimate those steps for your task instead of borrowing a generic timeline.
Match the architecture to a documented offer
Ask a supplier to identify the supported components, customer duties, data destinations, and recovery behavior in writing. Keep optional architecture ideas separate from currently offered deployments.
In AgentShelf
The documented offer includes AgentShelf-managed services and supported customer-managed or external runtimes. These are the appropriate starting points for an evaluation. If you need a hybrid or fully on-premises arrangement, confirm with AgentShelf what is available for your requirements before planning around it.
Continue with LLM infrastructure fundamentals and workspaces and external agents to distinguish the components and their bindings.