Skip to content
Running AI in production· Guide 3 of 6Intermediate6 min readUpdated

Self-hosting an AI agent platform ​

Self-hosting an AI agent platform means running its server, its database and its workers on infrastructure you control, so prompts, tool results, run history and credentials stay inside your network. The models can be local or remote: a self-hosted platform can still call a hosted model API, but you decide which one, with which key, and what is recorded. In exchange, you take on installing, upgrading, backing up and watching the platform yourself.

Why teams self-host ​

Teams that self-host usually cite data and control:

  • Where data sits. An agent run records the user's message, every tool call with its arguments, every tool result and the answer. On a self-hostedSelf-hosted: Software that you run on your own servers or cloud account, so its data and its keys stay under your control. platform that history sits in your database, under your retention and backup rules.
  • Network reach. Agents are useful because they reach internal systems: a ticket database, an internal API, a code host on a private network. A platform inside the network can call them without opening inbound access from someone else's cloud.
  • Credentials. The platform holds keys for those systems. Self-hosting keeps them in your secret store, encrypted with a key you hold.
  • Audit. Security reviews ask who ran what, with which tool and which data. Logs and audit records on your own systems can be kept, exported and checked on your terms.
  • Model choice. Some data may only go to a local model. A self-hosted platform can send those runs to a model server on your hardware and others to a hosted API.

Hosted workflow tools trade all of this for less operations work. If your agents only read public data and your team has no one to run a service, a hosted tool may be the better choice.

What runs ​

A self-hosted agent platform is a handful of ordinary services:

The parts of a self-hosted agent platformBrowsers and API callers reach a reverse proxy. Behind it, a web UI and API replicas. API replicas and workers share one database. Workers and API call model endpoints and MCP servers.Users, webhooksReverse proxy (TLS)Web UIAPI replicasQueue workersDatabaseModels andMCP servers
One database holds the state; API replicas and workers can be added beside it.
  • The API server. It holds the workflow definitions, starts runs, runs the agent loopAgent loop: The cycle an AI agent repeats: call the model, run the tools it asks for, add the results to the conversation, then call the model again until it answers., calls models and tools, and serves webhooks.
  • The web UI. The editor and admin screens, usually static files behind the same proxy.
  • The database. Users, workflows, run history, encrypted secrets, audit records. Everything stateful lives here.
  • Workers. Processes that take queued runs, so long agent runs do not tie up the processes answering HTTP.
  • Model endpoints. A hosted model API, a local model server, or both.
  • MCP serversMCP server: A program that offers tools, resources or prompts to AI applications over the Model Context Protocol, locally over stdio or remotely over HTTP.. The tools: remote servers over HTTP, or local programs the platform starts over stdiostdio transport: The MCP transport in which the client starts the server as a child process and exchanges newline-delimited JSON-RPC messages over the process's standard input and output..
  • A reverse proxy for TLS, and optionally a load balancer when there is more than one API process.

Start small: one process, one file ​

The first install can be a single process with an embedded database file. That is enough for a team trying agents out, and it has fewer parts to break. Move to a database server when you need any of these:

  • more than one API process (an embedded database file has one writer);
  • managed backups, replication or point-in-time recovery;
  • a database your operations team already monitors.

Plan the move early. Copying data between databases is easier while there is little of it.

Scaling out ​

When one process is not enough, scale the two kinds of load separately:

  • Request traffic (the UI, webhooks, gateway calls): add API replicas behind a load balancer, with a readiness check so a draining replica stops getting traffic.
  • Run volume (agents mostly wait on models and tools): add workers that take runs from a queue in the database.

Watch for state that lives in one process's memory. Anything a second replica cannot see, such as a long-lived protocol session, needs sticky routing or a shared store.

Backups and the master key ​

A self-hosted platform encrypts stored secrets with a key you supply. That key is now the most important thing you hold:

  • A database backup without its key restores without the secrets. Keep the key for as long as you keep backups made with it.
  • Someone with both the backup and the key can read every stored secret. Store the master keyMaster key: The one key that encrypts the other stored secrets. Without it the stored secrets cannot be decrypted, so it is backed up apart from the database. apart from the backups, with its own access control.
  • Test a restore into a fresh instance before you need one. A backup that was never restored is a hope, not a plan.
  • Retention and deletion do not reach backups already taken. Rotate backups within the retention you promise.

Verify what you install ​

A platform that holds your credentials is worth checking before you run it:

  • Pin images by digest, not by a moving tag, so the image you tested is the image you deploy.
  • Check the signature and build provenance: who built the image, from which commit, in which build system.
  • Read the SBOMSBOM: Software bill of materials: a list of every component and version inside a piece of software, used to check it for known vulnerabilities and licences. (software bill of materials) for the packages inside, and scan it with the tools you already use.
  • Check what it calls out to. Know which outbound connections the platform makes on its own, and block the rest at the network.

What you take on ​

Self-hosting moves work to your team:

  • Upgrades. Read release notes, test upgrades with a copy of production data, and keep a rollback path.
  • On-call. Someone has to notice when runs fail, the queue backs up or the database fills.
  • Security patches for the platform, its base images and the database.
  • Capacity. Disk for run history, memory for agents in flight, database connections for every replica.
  • Your own controls. Sign-in, roles, network rules and retention are yours to set. The NIST Generative AI Profile is a useful checklist of risks to cover.

A first production checklist ​

  1. Generate the master key, store it in your secret manager, and keep a copy apart from backups.
  2. Bind the API to localhost or a private network, and expose it only through a TLS proxy.
  3. Turn off public sign-up; create the first admin from configuration.
  4. Choose the database: the embedded file for one process, a database server for more.
  5. Schedule backups and run one restore drill.
  6. Pin image digests and verify signatures.
  7. Send logs and metrics to the systems you already watch.
  8. Set retention for run history and agent memoryAgent memory: What an agent stores from earlier turns or runs and feeds back into later model calls: conversation history, summaries, saved tool results or retrieved notes. to what your policy allows.
  9. Set a deployment budget and an iteration limit on every agent.
  10. Write down who upgrades it and who answers alerts.

In L2M ​

L2M is open source (MIT) and runs only on your infrastructure: there is no hosted edition, and the product sends no analytics. Two things do load from the internet by default: the studio's web fonts (Google Fonts) and the API reference page at /api/docs (Swagger UI from a public CDN). Know both before you lock down outbound traffic. Beyond the defaults, everything below is optional.

  • Install. Run from source with Node.js and pnpm (Quickstart), with the Docker Compose files in the repository, or with the Helm chart in ops/helm. The API and web images are published to GHCR for linux/amd64 and linux/arm64.
  • Database. SQLite by default with nothing to configure; Postgres with DB_TYPE=postgres, and db:copy to move data across.
  • Scaling. In cluster mode (DB_TYPE=postgres and HA_ENABLED=true), several API instances and separate queue workers share one Postgres, with one leader for schedules and purges. On SQLite, one process is its own worker. Scaling covers roles, sizing and a load test harness. MCP gateway sessions are the one piece that needs sticky routing; the cluster page shows how to route them.
  • Backups. Backup and recovery has procedures for SQLite, Postgres and a portable export, a restore drill, and what to do after losing the master key (SECRET_MASTER_KEY_BASE64).
  • Verification. Each release image is signed with keyless cosign and carries SLSA build provenance and CycloneDX SBOMs: Verify a release.
  • Observability. Structured logs, Prometheus metrics at /metrics, and optional OTLP export of traces and metrics: Observability and OpenTelemetry.

Try it in L2M ​

Follow the Quickstart to run L2M on your laptop with SQLite, then import the zero-key MCP agent recipe: it runs with no API key. Before your first shared install, work through Backup and recovery and Verify a release.

Further reading ​

Last updated:

This website (not the L2M product) uses Google Analytics, only after you accept cookies. Cookie settings