Use Azure OpenAI in a self-hosted workflow
What you get: A workflow that sends a prompt to your Azure OpenAI deployment, with the key held by L2M's secret store, never in the workflow.
Needs: An Azure OpenAI resource, deployment and key · Time: 10 minutes · Nodes: 4
A Text Input and a Prompt Template build the prompt, the Azure OpenAI Chat Model node sends it to your deployment, and the Output returns the answer. The node has fields for the resource endpoint, the deployment name, the API version (2024-10-21 as shipped) and the key, which is a reference to a stored secret. The workflow file holds only the secret's id. You need an Azure OpenAI resource with a chat deployment; there is no keyless way to run this one.
Steps
- Download azure-openai-flow.json, or take it from the samples folder on GitHub.
- In the studio's workflow editor, import the file (see import and export).
- Store the key as a secret. In the node's config, open Azure OpenAI Secret and choose + Add New Credential... (the provider is already set to Azure OpenAI), name it and paste the key. Or use the API (below).
- On the Azure OpenAI Chat Model node, set Endpoint to
https://<your-resource>.openai.azure.com, Deployment to your deployment's name (not the model's name, unless you named it after the model) and pick the secret under Azure OpenAI Secret. Save. - Run it. The answer is in the run's output as
result.
How it works
| Node | Type | Role |
|---|---|---|
| Text Input | text_input | Step in the run |
| Prompt Template | prompt_template | Step in the run |
| Azure OpenAI Chat Model | azure_openai_chat_model | Step in the run |
| Output | output | Step in the run |
4 node types. Each type links to its config fields in the node reference.
Store the key as a secret
# An API key with builder rights, from Settings → Account → API Keys.
curl -X POST http://localhost:4000/api/secrets \
-H "Authorization: Bearer $L2M_API_KEY" -H "Content-Type: application/json" \
-d '{"name": "azure-openai-prod", "provider": "azure_openai", "value": "<your-azure-openai-key>", "projectId": "<your-project>"}'The answer holds the secret's id (sec_...) and never the value; no API returns the value again. It is encrypted at rest with AES-256-GCM under SECRET_MASTER_KEY_BASE64. Without projectId the secret lands in the default project and the answer carries a warning saying so. See Secrets.
The value can be the raw key, or JSON such as {"apiKey": "..."} (or accessToken). L2M sends it in the api-key header. A value with three dot-separated parts, or starting with Bearer , is sent as Authorization: Bearer ... instead, for Microsoft Entra ID tokens. L2M does not fetch or refresh such tokens itself.
What a run does
We had no Azure resource, so we ran the sample two ways on a fresh install.
As shipped, the run fails at the Azure node: Provider azure_openai requires an API key via secretRef or AZURE_OPENAI_API_KEY. The placeholder secret id does not exist, and a missing secret does not fail on its own: the node falls back to the server's AZURE_OPENAI_API_KEY. If that is set, a workflow with a wrong secret id quietly uses the server's key.
Against a local stand-in for the Azure endpoint, with a stored secret, the node sent one request to POST /openai/deployments/gpt-4o-mini/chat/completions?api-version=2024-10-21 with the key in the api-key header. The body held the prompt as the only message (the sample has no System Prompt node, so no system message is sent), temperature: 0.2, max_tokens: 1024 and stream: true with stream_options.include_usage. The answer came back as result, and the node recorded the token usage the endpoint reported. With a key the endpoint refused, the run failed with Provider azure_openai stream request failed (401): ... and the endpoint's own message.
A real Azure endpoint was not called, so this page does not show Azure's own errors, content filtering or quotas.
Costs
Workflow runs record tokens for Azure calls but no cost: the built-in price list has no Azure prices for runs, so Azure spend does not count toward a USD budget. To price it, add your deployment to LLM_PRICING_OVERRIDES_JSON under the provider id azure_openai, keyed by the deployment name, for example {"azure_openai":{"gpt-4o-mini":{"inputUsdPer1M":0.15,"outputUsdPer1M":0.6}}} (use your contract's prices).
Serve the deployment to other apps (optional)
The node calls Azure directly from the workflow. To let applications outside L2M use the same deployment with L2M's client tokens, budgets and audit, add a route on the /v1 AI gateway: provider azure_openai, the deployment name as the upstream model, your endpoint as the base URL and the same stored secret as the key. Apps then call POST /v1/chat/completions with a public model name. Unlike workflow runs, /v1 traffic to a deployment named gpt-4o or gpt-4o-mini is priced by default. The gateway is off until you add a route set.
Make it yours
- Add a System Prompt node before the Azure node; it reads
system_prompt. - Use the same node as the chat model of an agent, on its
chat_modelport. - For embeddings, use the Embeddings Azure OpenAI node or the
azure-openai-embedderin RAG Retrieve.
Make it safer
These controls are optional; the recipe runs without them.
- Keep one secret per environment and per project, and share a default-project secret with other projects only on purpose: see project scope.
- Restrict which deployments workflows may call with a model allowlist.