Skip to main content

Core Capabilities: Models, Agents, Data & Tools

Foundations trackPart 2 of 3

Core building blocks

Models reason; agents coordinate; data grounds; tools act.

These four capabilities work as a system. The design challenge is deciding which responsibility belongs to the model and which belongs to controlled application logic.

Where this fits in the learning path

This is the second article in the Foundations track.

If you have not read the platform overview yet, start with What is Microsoft Foundry: Foundations & Architecture.

Freshness note

This article was refreshed on September 28, 2026. Model availability, agent features, deployment options, and portal experiences can change by region and release wave, so always confirm production choices in current Microsoft documentation before rollout.

This article covers the first four core capability areas:

  1. Models: choosing the right intelligence for the workload
  2. Agents: turning one-shot answers into managed task execution
  3. Enterprise data: grounding outputs in trusted business information
  4. Tools: connecting AI to real actions and external systems
When to use which

If your main need is user productivity inside Word, Excel, Outlook, or Teams, Microsoft 365 Copilot is usually the starting point.

If you need custom model strategy, custom grounding, external business tools, APIs, agent hosting, or enterprise-specific control over behavior, Microsoft Foundry is usually the better fit.

1. Models: The Intelligence Layer​

The first architectural mistake many teams make is treating the model as the whole solution.

It is not.

The model is one component inside the system:

Microsoft Foundry's advantage is that it can expose models from multiple families through one platform experience. In practice, that means an enterprise can compare, deploy, and govern different model options without redesigning its entire AI platform every time model strategy changes.

Model choice is a workload decision​

Avoid both extremes:

  • "Always use the smartest model"
  • "Always use the cheapest model"

The right objective is:

Choose the lowest-cost model that reliably meets the quality, latency, safety, and operational requirements of the workload.

Typical examples:

Email classification        -> small or cost-optimized model
Entity extraction           -> small or mid-tier model
Document summarization      -> mid-tier model
Grounded enterprise Q&A     -> capable general model
Complex policy analysis     -> stronger reasoning model
Agent planning with tools   -> model proven on tool use and multi-step tasks

Evaluate before standardizing​

Do not standardize on a model because a few interactive prompts looked good.

Use representative tasks and measure:

  • Accuracy
  • Relevance
  • Groundedness
  • Instruction following
  • Structured-output reliability
  • Tool-call success
  • Latency
  • Token consumption
  • Cost
  • Safety outcomes
  • Consistency over repeated runs

Model evaluation loop​

Model routing matters​

Many enterprise systems should use more than one model, but only when there is a real reason.

Two common patterns:

  • Static routing: the application already knows the task type
  • Dynamic routing: a routing layer decides based on complexity, context size, risk, modality, SLA, or tool needs

Use multi-model architecture when it improves outcomes. Do not add it just because the catalog is broad.

Cost per successful business outcome​

A cheap model with low task success can be more expensive overall than a pricier model that succeeds consistently.

Track:

  • Token cost
  • Retrieval cost
  • Tool cost
  • Retry rate
  • Failure rate
  • Human review cost
  • Operational overhead

The better question is:

What is the cost per successful business outcome?

2. Agents: From Answers to Actions​

A chatbot answers.

An agent pursues a goal using reasoning, tools, and iteration.

Why Foundry Agent Service matters​

Current Microsoft Foundry guidance positions Foundry Agent Service as the managed platform for building, deploying, and scaling AI agents.

At a high level, teams can choose among different operating models:

  • Prompt agents for platform-authored agents
  • Hosted agents for your own agent code running with managed endpoints, identity, scaling, and observability
  • Different framework choices, while still using Foundry as the operating layer

That makes the platform useful even if your team does not want a single agent framework.

Deterministic vs probabilistic responsibilities​

This is a core enterprise rule.

Use the model for:

  • Language understanding
  • Summarization
  • Extraction
  • Reasoning
  • Planning
  • Natural-language generation

Use deterministic systems for:

  • Authorization
  • Financial calculations
  • Threshold checks
  • Workflow state transitions
  • Database writes
  • Transaction processing
  • Enforcement of mandatory policy rules

Safe action pattern​

The model can recommend an action.

It should not be the final source of truth for whether the action is allowed.

Human approval still matters​

As autonomy increases, approval design matters more.

Examples:

  • Low-risk refund recommendation under a small threshold may be auto-executed after policy checks
  • High-value payment, regulated record change, or destructive system action should often require explicit human approval

Good approval criteria include:

  • Financial impact
  • Regulatory impact
  • Irreversibility
  • Data sensitivity
  • Customer impact
  • Confidence level of the agent workflow

3. Enterprise Data: Grounding and RAG​

Models do not know your private, current, organization-specific facts unless you provide them.

That is why grounding matters.

The most common grounding pattern is RAG:

What RAG improves and what it does not​

RAG often improves factual reliability because the model works from retrieved evidence instead of memory alone.

But RAG is not magic.

It can still fail because of:

  • Bad source documents
  • Poor chunking
  • Weak retrieval
  • Missing permissions trimming
  • Out-of-date documents
  • Weak prompts
  • Poor reranking
  • Inadequate evaluation

The correct mindset is:

RAG reduces the chance of unsupported answers. It does not eliminate hallucinations by itself.

Grounding options are broader than classic RAG​

A lot of teams speak as if all grounding equals vector search plus chunks.

That is too narrow.

Grounding might involve:

  • Azure AI Search
  • Structured database queries
  • Business APIs
  • Files and document repositories
  • Short-lived workflow state
  • Tool-generated facts

As of September 28, 2026, Azure AI Search also has agentic retrieval capabilities, but some of those experiences still have preview-only aspects in the portal even though parts of the API surface are already generally available. That means architects should separate:

  • What is safe for production today
  • What is still preview in the portal or API

A practical grounding principle​

Use grounding when the answer depends on:

  • Private enterprise data
  • Current facts
  • Authoritative policy
  • Auditable evidence

Do not force a RAG stack into workloads that do not need it.

4. Tools: Turning AI Into Action​

Grounding helps a system know things.

Tools let it do things.

Examples:

  • Search a customer
  • Check an invoice
  • Query a database
  • Create a support case
  • Trigger an Azure Function
  • Call an internal REST API
  • Start a business workflow

Generic tool architecture​

MCP: a useful standard, not a mandatory one​

One of the most important tool-integration standards today is Model Context Protocol (MCP).

MCP is now an open standard for connecting AI applications to external systems through a common tool and context interface. The common analogy is still accurate:

MCP is like USB-C for AI-tool integration.

That matters because it reduces duplicated point-to-point connector work across agents and applications.

Before and after standardization​

When MCP is a good fit​

MCP is especially useful when you want:

  • Reusable tool contracts across multiple agents
  • Standardized discovery and invocation
  • Cleaner separation between agent logic and tool implementation
  • Faster integration into an agent ecosystem

But it is not the only valid pattern.

Depending on the requirement, a better option might be:

  • REST APIs
  • Native SDKs
  • Azure Functions
  • Service Bus
  • Event Grid
  • Platform-native connectors

Choose the pattern that best fits:

  • Security model
  • Transaction boundaries
  • Throughput and latency
  • Ownership model
  • Operational maturity

Tool boundaries matter​

Do not automatically create one giant tool surface for an entire business domain.

Split tool boundaries based on:

  • Risk level
  • Read versus write access
  • Data sensitivity
  • Authorization model
  • Team ownership
  • Release cadence

For example, a read-only finance tool surface and a payment-approval tool surface should often be separated.

Key Takeaways​

The four practical lessons are:

  1. Model strategy is an architecture decision, not a popularity contest.
  2. Agents should reason probabilistically but act through deterministic controls.
  3. Grounding is about trusted evidence, not just about adding a vector database.
  4. Tools are where AI systems become useful, so integration boundaries and permissions deserve serious design.

Continue the Series​

Next, read Operational Excellence: Evaluation, Safety & Governance to learn how these systems are measured, observed, and governed in production.