5 Critical Deployment Decisions for Azure OpenAI
The infrastructure choices that determine compliance, cost, and performance.
Infrastructure & Deployment
Make the right choices for your deployment.
Before writing a single line of agent code, you need to answer five infrastructure questions. These decisions directly impact GDPR compliance, scaling capability, and daily costs. Get them right, and your agent scales effortlessly. Get them wrong, and you'll regret it in production.
🎯 The 5 Decisions You Need to Make
Here are the critical infrastructure decisions you need to make:
| Decision | Impact | Recommendation |
|---|---|---|
| Which deployment type? | Compliance + Routing | Standard (single-region) ✅ |
| How to version the model? | Behavior consistency | Lock to specific version ✅ |
| What throughput model? | Cost & Scaling | Flex Consumption for variable traffic ✅ |
| How to authenticate? | Security & Audit | Managed Identity (no API keys) ✅ |
| How to handle variable traffic? | Cost optimization | Auto-scaling on Flex ✅ |
🔄 The Big Picture: What's In Scope?
✅ What You ARE Configuring
❌ What You're NOT Doing
- ❌ Rewriting agent prompts
- ❌ Changing agent logic or tools
- ❌ Modifying conversation memory schema
- ❌ Implementing new features
- ❌ Changing the agent's behavior
Key insight: Agent1 is already built. Your job is to decide how to run it safely and cost-effectively.
🔄 The Big Picture: What's In Scope?
✅ What You ARE Configuring
❌ What You're NOT Doing
- ❌ Rewriting agent prompts
- ❌ Changing agent logic or tools
- ❌ Modifying conversation memory schema
- ❌ Implementing new features
- ❌ Changing the agent's behavior
Key insight: Agent1 is already built. Your job is to decide how to run it safely and cost-effectively.
🎯 Choose Your Scenario
Each organization has different requirements. Pick yours:
Startup/SMB
Predictable traffic, single region, cost-sensitive, no compliance requirements
Mid-Market
Variable traffic, single region, GDPR/compliance required, need flexibility
Enterprise Multi-Region
High volume, multiple regions, strict compliance, reserved budget
Real-World Example: Contoso (Mid-Market)
To illustrate, we'll use Contoso, Ltd. — a multinational retailer. Their context:
Traffic Pattern
Variable: 10 req/sec off-peak → 100 req/sec peak
(9 AM - 5 PM busy, 7 PM - 9 AM quiet)
Compliance Requirement
GDPR Article 5: All data must stay in EU (France Central only)
Budget Constraint
No upfront reserved capacity budget. Pay-as-you-go preferred.
Business Goal
Support 10,000+ daily customer inquiries with personalized, accurate responses.
📊 The Five Key Decisions
1️⃣ Deployment Type: Standard vs Global
This is the MOST CRITICAL decision for Contoso.
Why Standard is CORRECT:
- 🟢 GDPR Compliant: Data stays in France Central. Never routed elsewhere.
- 🟢 Predictable Latency: EU customers always hit local infrastructure.
- 🟢 Cost Aligned: Scales with usage, no upfront commitments.
Why Global is WRONG:
- 🔴 GDPR Violation: "Global" means traffic routes to BEST AVAILABLE capacity. Could be US, Singapore, anywhere.
- 🔴 Compliance Risk: Data leaving EU = Article 5 violation = up to €20M fine.
- 🔴 Unknown Routing: You lose control over data placement.
2️⃣ Version Update Policy: Lock vs Auto-Upgrade
You have three approaches. Here's which one works best:
Real Timeline Example:
2026-08-15: Deploy with gpt-4o-2024-08-06
✅ Agent behavior locked
2027-02-01: Microsoft announces gpt-4o-2024-08-06 will retire
⏰ 6-month notice period
2027-08-01: Model retires, forced upgrade
✅ Your team had 6 months to test gpt-4o-2025-01-xx
✅ Customers experience smooth transition
vs. Aggressive Policy:
2026-08-15: Deploy with gpt-4o-2024-08-06
2026-09-20: New gpt-4o-2024-09-xx released
⚠️ Auto-upgrades immediately
❌ Agent behavior suddenly changes
❌ Customers get different responses
❌ Support team overwhelmed
3️⃣ Throughput Model: Flex Consumption vs Provisioned
What payment model fits Contoso's traffic pattern?
When to use each:
| Scenario | Best Model | Reason |
|---|---|---|
| Variable traffic (10-100 req/sec) | Flex | Pay only for actual usage |
| Consistent high volume (>100 req/sec) | Provisioned | Guaranteed throughput, better unit cost |
| Bursty workloads (quiet then busy) | Flex | Avoid paying for idle capacity |
| SLA requirements (strict latency) | Provisioned | Guaranteed resources |
Contoso's choice: Flex Consumption. Their traffic varies 10:1 between off-peak and peak. Provisioned would waste money during nights.
4️⃣ Authentication: Managed Identity vs API Keys
Zero-trust is non-negotiable.
How it works:
# ❌ NEVER DO THIS (API key in code)
from azure.ai.openai import AzureOpenAI
client = AzureOpenAI(
api_key="sk-proj-abc123xyz...", # VISIBLE IN CODE
endpoint="https://my-resource.openai.azure.com/"
)
# ✅ ALWAYS DO THIS (Managed Identity)
from azure.identity import DefaultAzureCredential
from azure.ai.openai import AzureOpenAI
credential = DefaultAzureCredential()
client = AzureOpenAI(
credential=credential,
endpoint="https://my-resource.openai.azure.com/"
)
What happens behind the scenes:
- Agent runs with Managed Identity (assigned at deployment time)
- Azure automatically creates access token
- Token rotated every ~8 hours (you don't manage it)
- Every call logged: who called, when, what resource
- Compliance team can audit everything
5️⃣ Auto-Scaling Strategy
Handle the 10:1 traffic ratio gracefully.
Configuration settings:
# Azure CLI: Enable auto-scaling on Azure OpenAI Deployment
az cognitiveservices account deployment-sku-scale \
--resource-group "contoso-ai" \
--resource-name "contoso-openai" \
--deployment-id "agent1-deployment" \
--scale-settings "{
\"scale_type\": \"automatic\",
\"capacity\": {\"min\": 1, \"max\": 10}
}"
📌 Deep Dive: Each Decision's Business Impact
Deployment Type Impact on Compliance
Real-world cost of non-compliance:
- GDPR fine: Up to €20 million OR 4% of global revenue (whichever is higher)
- Brand damage: Loss of customer trust
- Legal costs: Defending against regulators
- Operational impact: Resource diversion to response
Contoso's decision: Standard deployment. Cost of compliance << cost of violation.
🎯 Summary: The Right Configuration
| Configuration | Decision | Applied To | Cost Impact | Compliance |
|---|---|---|---|---|
| Deployment Type | Standard | Azure OpenAI | ✅ Compliant | GDPR Article 5 |
| Version Update Policy | Once expires | Model lifecycle | ✅ Stable behavior | Controlled upgrades |
| Throughput Model | Flex Consumption | Billing model | $900/month avg | Scales with traffic |
| Authentication | Managed Identity | All APIs | ✅ Auditable | Zero-trust |
| Auto-scaling | Enabled (1-10) | Deployment SKU | ✅ Always available | High SLA |
📚 What's Next?
Next Steps: As you continue building Contoso's deployment, you'll need to explore:
- Standard vs Global routing decisions
- Flex Consumption vs Provisioned pricing models
- Model version strategies and auto-upgrade policies
- GDPR compliance and data residency
- Managed Identity authentication setup
These topics will be added to the learning path as we convert interview questions into detailed guides.
Last updated: 2026-09-28