Azure OpenAI development in Microsoft Foundry

Azure OpenAI suits teams that already run on Azure or must keep AI inside it (for the platform itself, see Azure development):

• You want the model inside your Azure tenant. Microsoft hosts it in Azure, and it does not interact with services run by OpenAI.
• You need to control where prompts are processed. Data Zone deployments keep processing in the US, EU or Asia Pacific zone, and Standard deployments keep it in your Azure geography.
• Your security team wants no public endpoint and no stored keys. You can disable public access, use private endpoints and sign in with a managed identity.
• You want answers grounded in your content with Azure AI Search, using hybrid keyword and vector queries and document-level security trimming.

Plain OpenAI is often simpler. Plan around these before you commit:

• You do not use Azure and have no rule that requires it. The OpenAI API needs no subscription, resource, deployment or quota setup.
• You need the newest models in a specific region on day one. New models reach Global deployments first, and geography-based deployments arrive last with no guaranteed date.
• You expect to turn the content filters off. Default filters apply to every deployment, and turning them off for completions needs Microsoft's approval.
• You cannot plan for quota. Quota is assigned to your subscription, and an increase is a request that Microsoft can decline.

You create a resource in Microsoft Foundry and deploy a model to it. Your code calls the deployment by its name, which can differ from the model name. The deployment type sets where prompts are processed: Global types may use any Azure region, Data Zone types stay within the US, EU or Asia Pacific zone, and Standard types stay within your Azure geography. Data stored at rest stays in your chosen geography in every case. Microsoft says prompts and completions are not available to OpenAI and are not used to train the models. Content filters check prompts and completions for hate, sexual content, violence and self-harm at levels you can set, and optional prompt shields flag jailbreak attempts and instructions hidden in documents. The v1 API works with the OpenAI client libraries with minimal code changes and no monthly api-version.

• Identity: Microsoft Entra ID instead of keys. The app's managed identity gets the Cognitive Services OpenAI User role, and the resource needs a custom subdomain for token sign-in.
• Networking: public access disabled, a private endpoint in your virtual network and private DNS, so only your network can reach the model.
• Filtered results: a blocked prompt returns HTTP 400, and a filtered answer ends with finish_reason set to content_filter. We handle both and show the user a clear message. If the filter service is unavailable, the request completes unfiltered with an error noted in the response, so we check for that too.
• Retrieval: Azure AI Search indexes your content in chunks with vectors. Hybrid queries run keyword and vector search together, and semantic ranking can reorder the results. Security trimming keeps users to the documents they may see.
• Our own guards: filters catch harmful content, not business mistakes. We add checks for what your bot must never say, such as health claims, quoted prices or links to other sites.
• Quota and retries: quota is set per subscription and model. We retry 429 errors, increase traffic gradually and consider provisioned throughput when latency must stay steady.

1. Agree the data rules first: region, data zone, network and identity needs decide the deployment type and setup
2. Build retrieval before the chat: index your content and test that the right passages come back for real questions
3. Set filters and guards: content filter levels, prompt shields and your own output checks before any customer sees an answer
4. Hand over properly: deployments, roles, network setup, index design and guard rules documented, with notes on reading a filtered or failed request