What is Microsoft Foundry: Foundations & Architecture
From "Hello, LLM" to "Hello, Governed Enterprise AI."
Platform foundation
A governed platform for building enterprise AI systems.
Microsoft Foundry provides AI-specific capabilities such as models, agents, evaluation, and observability, while Azure continues to provide the enterprise foundation around them.
📖 The Story: From Prototype to Production
The magical journey from "simple idea" to "wait, we need what?"
"Can we build an AI assistant to help employees submit expense reports?"
"Great demo! The AI reads receipts and suggests categories. Can we go live?"
The realization:
That gap between "nice demo" and "production system" is where Microsoft Foundry comes in.
🎯 What Foundry Actually Is
Beyond a model endpoint
Azure still matters
Keep critical rules deterministic
This is the first article in the Foundations track.
Read in this order:
- Foundations & Architecture: what the platform is and why it exists
- Core Capabilities: Models, Agents, Data & Tools: what you actually build with
- Operational Excellence: Evaluation, Safety & Governance: how you run it safely at scale
This article was refreshed on September 28, 2026. Microsoft now uses the Microsoft Foundry brand. Older content might still say Azure AI Foundry, Azure AI Studio, or Foundry (classic).
📝 Executive Summary
The shortest accurate definition is:
Microsoft Foundry is Microsoft's enterprise AI platform on Azure for building, deploying, observing, evaluating, securing, and governing AI applications and AI agents.
It is not just "a place to call a model."
What a Production AI Solution Actually Needs
A proof-of-concept needs inference. Production needs everything else:
Orchestration & grounding
• Model access and switching
• Agent orchestration
• Enterprise data grounding
• Tool calling and integrations
Measurement & safety
• Evaluations and quality metrics
• Tracing and monitoring
• Safety controls and guardrails
• Cost tracking per request
Identity & governance
• Identity and RBAC
• Networking and secrets
• Governance and versioning
• Compliance and audit trails
The Split: What Foundry Handles vs What Azure Handles
Microsoft Foundry owns:
- ✅ Models, agents, tools, evaluations
- ✅ AI-specific observability and tracing
- ✅ Safety guardrails and jailbreak prevention
- ✅ Multi-team agent governance
Azure platform provides:
- ✅ Identity (Entra ID)
- ✅ Networking and firewalls
- ✅ Data encryption and secrets (Key Vault)
- ✅ Compliance and audit (Purview, Azure Defender)
- ✅ Operations and monitoring infrastructure
Together:
Foundry handles the AI-specific layers. Azure handles the enterprise foundation. Your team focuses on business logic and use cases.
Microsoft Foundry vs Microsoft 365 Copilot
The question: "We want to add AI to our organization. Which should we use?"
This is one of the most common points of confusion because both sit inside Microsoft's broader AI ecosystem. But they solve fundamentally different problems.
The Distinction
Think of them as two different answers to two different questions:
Microsoft 365 Copilot: "How do we add AI assistance to the tools our users already use every day?"
- Goal: Productivity in Microsoft 365 apps (Word, Excel, Teams, Outlook, etc.)
- Audience: End users
- You control: Prompts and data access
- Microsoft controls: Models, app integration, user experience
- Outcome: Users get smarter in their daily work
Microsoft Foundry: "How do we build a custom AI system for our specific business problem?"
- Goal: Build production AI agents and applications
- Audience: Your developers and architects
- You control: Everything (models, logic, deployment, governance)
- Microsoft controls: Infrastructure and platform
- Outcome: Your custom AI application, running at scale
The Practical Comparison
| Dimension | Microsoft 365 Copilot | Microsoft Foundry |
|---|---|---|
| What you're building | Embedded productivity aids | Custom AI applications |
| Users | Everyone in Microsoft 365 | Your specific users |
| Your control | High-level configuration | Full architectural control |
| Grounding | Microsoft 365 content + Microsoft Graph | Your data + your APIs |
| Workflow | "Add AI to this app" | "Build an AI app" |
| Example | Copilot in Word, Excel, Teams | Expense approval agent |
They're Actually Complementary
You can use them together:
- Build in Foundry - Create a custom AI agent for your specific workflow
- Publish to 365 - Make that agent available inside Teams or other Microsoft 365 apps
- Result - Users access your custom AI where they already work
How to Decide
Use Microsoft 365 Copilot if:
- ✅ You want AI to enhance existing Microsoft 365 workflows
- ✅ Your users spend time in Word, Excel, Outlook, Teams, PowerPoint
- ✅ You prefer a managed, pre-built experience
- ✅ You want Microsoft to handle model updates and scaling
Use Microsoft Foundry if:
- ✅ You need a custom AI application for your business problem
- ✅ You need to integrate with your internal systems and APIs
- ✅ You need full control over agent logic and governance
- ✅ You need multi-agent orchestration or complex workflows
- ✅ You need to own model selection and evaluation
Wrong question: "Should we choose Foundry or Microsoft 365 Copilot?"
Right question: "Which parts of our AI strategy need a managed product experience, and which parts need a custom platform?"
The Big Idea
Here's what actually matters:
Building an enterprise AI system is not just "call a model." It's an entire architecture—identity, data, safety, governance, operations—all working together.
Microsoft Foundry sits in the middle of that architecture:
The Dependency Flow
Your application calls Foundry. Foundry uses Azure services. Azure provides the enterprise foundation (identity, security, compliance).
The practical rule:
Foundry complements enterprise architecture. It does not replace it.
Specifically:
- Foundry handles: AI orchestration, agents, models, evaluation, guardrails
- Azure provides: Identity, networking, secrets, data, monitoring, compliance, operations
- You provide: Business logic, integrations, product experience
You still rely on:
- Microsoft Entra ID (identity)
- Azure API Management (API governance)
- Azure AI Search (data grounding)
- Azure Key Vault (secrets)
- Application Insights (monitoring)
- Microsoft Defender (security)
- Your own APIs and databases (system of record)
Why Microsoft Foundry Exists
The problem starts simple:
"Build an internal AI assistant that helps employees submit expenses, checks policy, and triggers reimbursement workflows."
Prototype view (what the pitch looks like):
But the moment you ship to employees, the questions start:
That's the real list. Not "build an LLM call." But:
- How do I orchestrate multiple AI components?
- How do I keep the system safe and compliant?
- How do I measure if it's actually working?
- How do I control costs?
- How do I audit what it did?
That's what Foundry is designed to address.
DIY AI Stack vs Platform Approach
Without a platform: You build everything
If you assemble an AI system without Foundry, you typically own all these pieces:
The cost: Every box is code you maintain. Every box is infrastructure you operate. Every update to policy means code changes. Every security incident requires you to trace through your custom system.
With Microsoft Foundry: Platform handles the AI layers
Much of that AI-specific infrastructure is managed:
The advantage: You focus on your business logic (policy rules, approval workflows, integration with your HR system). Foundry handles the AI platform layer (models, safety, evaluation, observability, tracing).
You still need Foundry when:
- You have multiple teams building AI systems (governance, RBAC)
- You need to swap models without redeploying code (model catalog)
- You need to evaluate quality repeatedly (evaluation framework)
- You need enterprise audit trails (built-in logging)
- You need safety guardrails (multi-layer protection)
- Your AI system talks to multiple data sources (grounding)
Terminology Note: What's Changed in 2026
As of September 28, 2026, the platform has unified under a single brand and direction:
- Current brand: Microsoft Foundry
- Current model: Single management plane with unified agents, models, and tools
- Current APIs: Responses API is the main entry point
- Older terms you'll still see: "Azure AI Foundry," "hub-based projects," "Foundry classic"
This matters because documentation and codebases from 2024-2025 still use the old terms. If something doesn't make sense, check the date of the documentation.
A Clean Mental Model
Think in layers:
This is the architectural separation to remember:
- Your application layer still owns product behavior and deterministic business rules
- Foundry supplies AI-specific platform capabilities
- Your enterprise platform still owns identity, network isolation, secrets, compliance, and operations
How Foundry Runs Agents
First: When Do You Actually Build an Agent?
Not every AI use case needs an agent. Let's be clear on the distinction:
You use a model directly if:
- You need a single LLM call (classify text, extract data, summarize)
- No reasoning, no decision loops, no multi-step workflows
- Examples: Content moderation, document classification, text generation
You build an agent if:
- The AI needs to make decisions based on context (not just generate text)
- It needs to loop—call a tool, check the result, decide next steps
- It needs to interact with your systems (call APIs, check databases, access files)
- Examples: Expense approval (decide approve/reject), customer support (route requests), data analysis (query, analyze, summarize)
Agents are more complex, so only build one if you need the decision-making loop.
The Agent Choice: Two Fundamentally Different Approaches
If you've decided you need an agent, Foundry gives you two ways to build it:
"Do I define the agent declaratively in Foundry's platform (no code), or write custom agent code?"
This choice profoundly affects your development speed, flexibility, testing, and operations. Foundry supports both because they solve different problems.
The Problem: One Size Doesn't Fit All
Scenario 1: Structured workflows
- Expense approval follows a known process
- Decision rules are predictable
- The workflow rarely changes mid-flight
- You want fast deployment and easy policy updates
Scenario 2: Complex reasoning
- Customer support needs real-time context switching
- The agent must handle exceptions and edge cases
- Complex logic that's hard to express as "platform configuration"
- You need to test agent behavior through code
Neither approach is inherently better. You pick based on your use case.
Option 1: Managed Agents (Configuration-Based, No Code)
Your agent is entirely defined within Foundry's platform using configuration and instructions—no code required.
What you do:
- ✍️ Write the agent's instructions and behavior in Foundry
- ✍️ Define what tools the agent can call
- ✍️ Specify decision logic or guardrails
- ✍️ Foundry handles the agent execution loop
What you get:
- ⚡ Fast to deploy (no code compilation)
- 🔧 Easy to update (change instructions, redeploy instantly)
- 📊 Built-in observability (Foundry tracks everything)
- 🛡️ Built-in safety (Foundry's guardrails apply by default)
Real example:
Expense Approval Agent:
- "Read the receipt image"
- "Extract amount and category"
- "Check against policy rules using the PolicyLookup tool"
- "Reject if > $10k, approve if policy allows"
- "Log the decision"
Foundry orchestrates these steps. No custom code.
Best when:
- ✅ Workflows are known and predictable
- ✅ You want fast iteration on rules
- ✅ Non-developers need to modify behavior (policy updates, instruction changes)
- ✅ Decision logic is straightforward
- ✅ You want the fastest possible deployment
Option 2: Custom Code Agents (Written in Your Language)
Your agent logic is written in code and runs as a managed workload in Foundry:
What you do:
- 💻 Write agent logic in your language (Python, C#, TypeScript)
- 💻 Define tools, decision-making, loops, error handling
- 💻 Push code to Foundry
- 💻 Foundry handles infrastructure, scaling, monitoring
What you get:
- 🎯 Full control over agent behavior
- 🧠 Complex reasoning and real-time decisions
- 🔌 Custom integrations and middleware
- 🧪 Test locally, deploy to Foundry
- 📈 Managed scaling (you don't run infrastructure)
Real example:
class CustomerServiceAgent:
def route_request(self, customer_intent):
# Complex logic: context from conversation history
if self.is_billing_issue(customer_intent):
return self.billing_tool()
elif self.requires_escalation(customer_intent):
return self.escalation_tool()
# Handle edge cases...
You own the logic. Foundry runs it at scale.
Best when:
- ✅ Logic is complex or context-dependent
- ✅ You need custom error handling and retry logic
- ✅ You want to test locally before deploying
- ✅ Developers are building the agent
- ✅ You need full control over agent behavior
Choosing Between Them
Use this framework to decide:
| Question | Managed Agent (Config) | Custom Code Agent |
|---|---|---|
| Do I write code? | ❌ No | ✅ Yes |
| Can non-devs change behavior? | ✅ Yes (edit instructions) | ❌ No (code review needed) |
| Is my logic simple/predictable? | ✅ Yes | ❌ Complex logic |
| How fast to deploy? | ✅ Minutes | ⏱️ Hours (build pipeline) |
| Can I test locally? | ❌ Platform only | ✅ Yes (local environment) |
| Do I need full control? | Limited (configuration) | ✅ Full control |
| When to use it | Structured workflows, policy-driven logic | Complex reasoning, edge cases, custom orchestration |
Connection: The Responses API
Regardless of which you choose, your application calls agents the same way:
Your App → Responses API → Foundry Agent → Result
(same interface for both types)
Key benefit: You can start with a managed agent for speed, then graduate to custom code agents as your needs grow—without changing your application code.
What Foundry Does Not Do
Good architecture is defined by boundaries. Let's be clear about what Foundry doesn't do—because that clarity prevents dangerous misuse.
What Foundry Is NOT
A replacement for your application architecture
A replacement for identity and security
A replacement for deterministic controls
The Concrete Reality: The $50k Expense
Imagine your expense approval agent approves a $50,000 reimbursement. The AI analyzed the receipt, checked historical approvals, and made a confident decision.
But your policy says: Maximum approval is $10,000.
The breakdown:
| Layer | Responsibility | Who Owns It |
|---|---|---|
| Agent runs & suggests | ✅ Foundry executes the agent logic | Foundry |
| Agent traces reasoning | ✅ Foundry logs what the agent said | Foundry |
| Validates against policy | ✅ Enforce $10k limit | Your code (deterministic) |
| Approves or rejects | ✅ Your system decides, possibly human | Your code or human |
| Records the decision | ✅ Audit trail with who approved it | Your system |
| Executes the approval | ✅ Integrate with finance system | Your code |
The core pattern:
Agent suggests → Your system validates → Human approves (if needed) → Deterministic system executes
What You Still Need to Build
Microsoft Foundry does not remove the need for:
Product Layer
- 🎯 Business logic and workflow design
- 🎯 User experience and product decisions
- 🎯 API design and contracts
- 🎯 Application orchestration
Enterprise Platform Layer
- 🔐 Identity and access control (Entra ID)
- 🔐 Network isolation and security (NSGs, firewalls)
- 🔐 Secrets management (Key Vault)
- 🔐 Compliance and audit (Purview, Policy)
Data Layer
- 📊 Data quality and validation
- 📊 Database design and optimization
- 📊 Search indexing and retrieval
- 📊 Data governance and lineage
The Bottom Line
Foundry owns: AI orchestration, model execution, agent loops, evaluation, guardrails, observability
Azure owns: Identity, networking, secrets, compliance, operations
Your team owns: Business logic, deterministic controls, critical decisions, integrations
They work together. None replaces the others.
The Seven Core Areas of Microsoft Foundry
Foundry is organized around seven key capabilities. Understanding these helps you know what the platform provides and what you need to build yourself.
The Areas, Briefly
Foundation (areas 1-4): How you build
- Models: Access to GPT-4o, Llama, and model selection
- Agents: Managed agents or custom code agents
- Enterprise data: Connect to your internal systems (APIs, databases, search)
- Tools: Define what actions your agent can take
Observability (areas 5-6): How you know if it works
- Evaluation: Measure agent quality and performance
- Safety: Guardrails, jailbreak prevention, content filtering
Operations (area 7): How you control it
- Governance: RBAC, versioning, audit trails, compliance
The next article in this series goes deeper into the first four building blocks: Models, Agents, Enterprise Data, and Tools. That's where you actually make design decisions.
Key Takeaways
If you take nothing else from this article, remember these five insights:
-
The gap between "demo works" and "production ready" is huge. That gap is what Foundry closes.
-
Foundry is a platform, not just model access. It's the entire operating system for building enterprise AI systems—models, agents, grounding, evaluation, safety, and governance.
-
Foundry and Azure work together, not separately. Foundry handles AI-specific layers (models, agents, safety). Azure handles enterprise layers (identity, networking, data). You own business logic.
-
You have choices in how to build agents. Managed agents for speed and simplicity. Custom code agents for control and complexity. Same API, different approaches.
-
Production AI systems need both intelligence AND deterministic controls. Your agent can suggest. Your system must validate. LLMs are helpers, not decision-makers for irreversible actions.
What's Next?
You now understand what Foundry is and why it exists.
The next articles in this learning path will dive deeper into:
- Core Capabilities: Models, Agents, Data & Tools — How to select models, structure agents, ground them in enterprise data, and define tools
- Operational Excellence: Evaluation, Safety & Governance — How to evaluate quality, enforce safety, and govern at scale
📢 More articles coming soon. Check back here for the full learning path as it rolls out.