Download Book

From The Book: Agentic AI - Theories and Practices (Ken Huang, 2025, Springer)

Part 1 of 3

AI Agent Tools and Frameworks

Introduction: From “Big Ideas” to “How Do We Actually Build This?”

Chapter 1 of the book did something important: it convinced us that AI agents are real, powerful, and here to stay. It talked about their history, their potential, and why they matter.

But Chapter 2 asks the next, much harder question:

“Okay, now how do we actually build these things?”

This chapter is not about philosophy or hype. It’s about engineering reality.

It explains:

what tools exist today,
how AI agent systems are structured,
which frameworks do what,
and what problems organizations will definitely run into when trying to deploy agents in the real world.

Think of this chapter as a map of the AI agent construction site — showing you the layers, the machinery, the scaffolding, and the safety rails.

The Big Organizing Idea: The Seven-Layer AI Agent Architecture

Before talking about tools and frameworks, the authors introduce a mental model — something they call the Seven-Layer AI Agent Architecture.

This is extremely important, because without it, the AI agent world feels chaotic.

Instead of thinking:

“There are 50 tools and frameworks and I don’t know where anything fits”

This model lets you think:

“Ah, this tool belongs here, and that problem belongs there.”

Why a Layered Architecture Matters (In Plain Language)

Imagine building a modern app like Uber or Amazon.

You wouldn’t mix:

database logic,
UI design,
security rules,
and server infrastructure
all into one giant mess of code.

You separate concerns.

The same idea applies to AI agents — except the systems are even more complex.

The seven-layer model breaks an AI agent system into:

clear responsibilities,
modular components,
replaceable parts.

Each layer:

builds on the one below it,
hides complexity from the one above it,
and lets teams work independently.

A Bird’s-Eye View of the Seven Layers

Here’s the full stack, top to bottom:

Foundation Models – the “brains”
Data Operations – memory, retrieval, pipelines
Agent Frameworks – how agents think and act
Deployment & Infrastructure – how agents run at scale
Evaluation & Observability – how we measure and monitor agents
Security & Compliance – how we keep everything safe (a vertical layer)
Agent Ecosystem – where users and businesses actually interact with agents

The chapter explains these top-down, starting from where value is created and moving down to raw intelligence.

Let’s walk through them the same way — slowly and conversationally.

Layer 7: The Agent Ecosystem (Where AI Actually Touches Reality)

This is the most human-facing layer.

If everything else is plumbing, wiring, and machinery, this is the storefront.

What Is the Agent Ecosystem?

The agent ecosystem is where:

businesses deploy AI agents,
users interact with them,
and real value is created.

Examples include:

customer support chatbots,
document analysis tools,
AI-powered research assistants,
workflow automation systems,
decision-support dashboards.

This is the layer people see.

Vertical vs Horizontal AI Agents

The chapter makes an important distinction:

Vertical agents
Built for a specific industry
- legal document review
- medical diagnosis assistance
- financial analysis
Horizontal agents
Built for a function across industries
- scheduling
- summarization
- search
- workflow automation

Most successful products combine both.

Marketplaces, SDKs, and Plug-and-Play Agents

Another key idea: the agent ecosystem is becoming a marketplace.

Instead of building everything from scratch, organizations can:

discover prebuilt agents,
reuse components,
integrate via SDKs,
evaluate reputation and performance.

This mirrors how:

app stores,
cloud marketplaces,
and open-source ecosystems evolved.

Why UX Still Matters (Even with Smart AI)

A subtle but important point in the chapter:

Even the smartest AI agent fails if the user experience is bad.

At this layer, success depends on:

smooth integration with existing systems (CRM, ERP),
scalability,
customization without chaos,
and intuitive user interaction.

This is where theoretical AI becomes practical AI.

Layer 6: Security and Compliance (The Layer That Touches Everything)

This layer is special.

Unlike the others, it’s not “one box” in the stack. It’s a vertical layer that affects every other layer.

Why Security Can’t Be an Afterthought

AI agents:

access sensitive data,
make decisions,
store memory,
sometimes act autonomously.

That’s a huge risk surface.

The chapter stresses:

If you bolt security on later, you’ve already failed.

Why Security Is Still Shown as a Separate Layer

The authors intentionally place Security & Compliance as Layer 6 to:

force organizations to treat it seriously,
centralize policy and oversight,
build specialized expertise,
manage regulatory obligations.

This includes compliance with:

GDPR
HIPAA
EU AI Act
UK AISI guidelines

“Defense in Depth” — What That Really Means

Instead of one big security gate, the chapter recommends multiple overlapping protections:

Secure model training (Layer 1)
Data privacy and access controls (Layer 2)
Input validation and API security (Layer 3)
Infrastructure hardening (Layer 4)
Monitoring and anomaly detection (Layer 5)
Safe deployment and access control (Layer 7)

Security is everyone’s job, not just the security team’s.

Layer 5: Evaluation and Observability (How Do We Know the Agent Is Doing the Right Thing?)

This is one of the most important — and most underestimated — layers.

Why Evaluating AI Agents Is Harder Than Evaluating Models

Traditional ML evaluation is simple:

fixed input,
fixed output,
accuracy score.

AI agents are different:

they act over time,
make multiple decisions,
adapt to changing environments,
sometimes surprise you.

So you’re not evaluating answers — you’re evaluating behavior.

Government-Level Attention: UK AI Safety Institute (AISI)

The chapter highlights work by the UK AI Safety Institute, which focuses on:

evaluating long-horizon agents,
testing autonomy,
measuring safety in complex environments.

They even launched bounty programs encouraging new evaluation techniques — a signal of how important this problem has become.

What Modern Agent Evaluation Looks Like

Evaluation now includes:

Safety metrics

containment
alignment
robustness
interpretability

Performance metrics

task completion
efficiency
adaptability
scalability

Cost metrics

token usage
compute spend
performance vs cost trade-offs

This reflects reality: an agent that works but bankrupts you is not a good agent.

Observability: Seeing Inside the Black Box

Observability tools help teams:

trace agent decisions,
monitor tool calls,
detect anomalies,
debug failures.

The chapter mentions real tools, including:

LangSmith
Langfuse
Arize AI
Weave
AgentOps.ai
Braintrust

Each focuses on slightly different aspects:

tracing,
bias detection,
user interaction analysis,
lifecycle management.

Where We’ll Continue in Part 2

In Part 2, I’ll cover:

Layer 4: Deployment & Infrastructure
Layer 3: Agent Frameworks
Layer 2: Data Operations
Layer 1: Foundation Models
RAG vs Agentic RAG (with a simple mental model)

Then in Part 3, I’ll cover:

deep framework comparisons (AutoGen, LangGraph, LlamaIndex, AutoGPT),
selection guidance,
real-world challenges (scalability, cost, compliance, talent),
and the chapter’s final takeaways.

Part 2 of 3

AI Agent Tools and Frameworks — From Infrastructure to Intelligence

Recap in One Paragraph (So We’re Oriented)

In Part 1, we introduced the seven-layer AI agent architecture, and we covered:

Layer 7: Agent Ecosystem (where users interact)
Layer 6: Security & Compliance (the vertical safety layer)
Layer 5: Evaluation & Observability (measuring agent behavior)

At this point, we’re standing halfway down the stack.

Now we move deeper — into:

how agents are deployed and scaled,
how their “thinking loops” are implemented,
how data flows through them,
and finally, what sits at the very bottom: foundation models.

This is where AI agents stop being product ideas and start becoming serious engineering systems.

Layer 4: Deployment & Infrastructure

“Where Do AI Agents Actually Live?”

If Layer 7 is the storefront and Layer 5 is the monitoring room, Layer 4 is the factory floor.

This layer answers questions like:

Where does the agent run?
How does it scale to thousands or millions of users?
How do we keep it reliable?
How do we manage cost?

Why Deployment Is Harder for Agents Than for Normal Apps

Traditional apps:

respond quickly,
are mostly deterministic,
have predictable resource usage.

AI agents:

run long workflows,
make multiple model calls,
use tools unpredictably,
may loop or branch,
may fail in unexpected ways.

This makes deployment much trickier.

Cloud vs On-Prem vs Hybrid

The chapter explains three major deployment patterns:

1. Fully Cloud-Based

Fast to start
Easy to scale
Higher long-term cost
Less control over data

This is common for startups and early-stage products.

2. On-Premise / Private Cloud

Strong data control
Regulatory compliance
Higher upfront cost
Slower iteration

This is common in:

finance
healthcare
government

3. Hybrid Deployment

Sensitive data stays local
Heavy compute runs in the cloud
More complex to manage

This is increasingly the default choice for enterprises.

Inference Infrastructure: Why GPUs Are Only Half the Story

People often think:

“If I have a GPU, I’m good.”

But inference infrastructure also includes:

load balancing,
request routing,
batching,
caching,
rate limiting,
fallback mechanisms.

The chapter emphasizes that model inference is not just a model problem — it’s a systems problem.

Cost Control Is an Engineering Requirement

AI agents can burn money very fast.

So Layer 4 often includes:

token budgets,
timeouts,
max-step limits,
circuit breakers.

An agent that thinks forever is not “smart” — it’s broken.

Layer 3: Agent Frameworks

“How Do Agents Actually Think and Act?”

Now we reach one of the core layers of the chapter.

Agent frameworks are the software libraries that define:

how agents loop,
how they reason,
how they call tools,
how they store state,
how they recover from failure.

If foundation models are the “brain,” agent frameworks are the nervous system.

Why You Need a Framework at All

Could you build an agent from scratch?

Yes — but it would be:

fragile,
hard to debug,
hard to extend,
hard to scale.

Frameworks give you:

structure,
safety rails,
reusable patterns.

They encode hard-earned lessons from many failed experiments.

The Core Agent Loop (Explained Simply)

Almost all frameworks implement some version of this loop:

Observe the current state
Decide what to do next
Take an action (model call or tool call)
Observe the result
Repeat until done

This loop may look simple, but implementing it safely is not.

ReAct: The Concept That Changed Everything

The chapter highlights ReAct (Reason + Act) as a turning point.

Instead of:

reasoning first,
acting later,

ReAct allows:

reasoning,
acting,
observing,
revising — in a loop.

This mirrors how humans work:

“Let me think… I’ll try this… okay that didn’t work… let me adjust.”

Many modern agent frameworks are essentially structured ReAct systems.

Popular Agent Frameworks (High-Level View)

The chapter introduces several major frameworks, each with a different philosophy.

LangChain

Early leader
Very flexible
Large ecosystem
Can become complex if misused

LangGraph

Graph-based agent workflows
Clear state transitions
Easier to reason about
Better for production-grade agents

AutoGen

Multi-agent focus
Conversation-based agents
Strong for coordination and delegation

LlamaIndex

Data-first agent design
Strong RAG integration
Great for knowledge-heavy agents

AutoGPT (and similar)

Fully autonomous agents
Minimal human intervention
Powerful but risky
Hard to control in production

The chapter does not say “one framework is best.”
Instead, it stresses fit-for-purpose selection.

Single-Agent vs Multi-Agent Systems

Another critical distinction.

Single-Agent Systems

Easier to build
Easier to debug
Limited scalability of reasoning

Multi-Agent Systems

Specialized roles (planner, executor, critic)
Better reasoning depth
More complexity
Coordination challenges

The chapter strongly suggests:

Multi-agent systems are powerful, but only when the problem truly needs them.

Layer 2: Data Operations

“Where Memory, Retrieval, and Context Come From”

This layer is where AI agents stop being forgetful.

Why Data Is Not “Just Context”

Many people think:

“We’ll just dump documents into the prompt.”

That works… until it doesn’t.

Agents need:

structured memory,
selective retrieval,
relevance ranking,
summarization,
expiration policies.

This is data engineering, not prompt engineering.

Short-Term vs Long-Term Memory

The chapter distinguishes between:

Short-Term Memory

Current conversation
Recent actions
Scratchpad reasoning

Often stored in:

prompt context
temporary state objects

Long-Term Memory

User preferences
Past interactions
Learned knowledge

Often stored in:

vector databases
relational databases
knowledge graphs

Vector Databases: The Backbone of Modern Agents

Vector databases enable:

semantic search,
similarity matching,
memory recall.

Popular options include:

FAISS
Pinecone
Weaviate
Milvus

But the chapter warns:

Vector databases are powerful, but misuse leads to noise, hallucinations, and wasted cost.

Retrieval-Augmented Generation (RAG)

RAG is explained as:

“Give the model the right information before asking it to answer.”

For agents, RAG is not optional — it’s foundational.

Agentic RAG vs Classic RAG

Classic RAG:

Retrieve once
Answer once

Agentic RAG:

Retrieve
Reason
Retrieve again
Refine answer
Validate output

This iterative retrieval is what allows agents to:

explore topics,
cross-check facts,
reduce hallucinations.

Data Pipelines and Freshness

Agents are only as good as their data.

This layer includes:

ingestion pipelines,
document chunking,
embedding generation,
refresh schedules.

Stale data leads to confidently wrong agents.

Layer 1: Foundation Models

“The Brains at the Bottom of the Stack”

Finally, we reach the base.

Everything above depends on foundation models.

What Foundation Models Actually Provide

Foundation models offer:

language understanding,
reasoning,
general knowledge,
pattern recognition.

They do not:

know your business rules,
understand your users,
manage workflows,
ensure safety.

That’s why the other six layers exist.

Closed vs Open Models

The chapter compares:

Closed Models

GPT-4, Claude, Gemini
High performance
Less control
Usage-based pricing

Open Models

LLaMA, Mistral, Falcon
Full control
Requires infrastructure
Customizable

Most real systems use a mix of both.

Model Selection Is a Strategic Decision

Choosing a model affects:

cost
latency
compliance
flexibility
vendor lock-in

The chapter emphasizes:

Model choice is not just a technical decision — it’s a business decision.

Where We’ll Finish in Part 3

In Part 3, we’ll wrap up with:

framework selection guidance,
common failure modes,
scalability challenges,
organizational readiness,
and the chapter’s final synthesis.

This is where everything ties together.

Part 3 of 3

Choosing Frameworks, Avoiding Pitfalls, and What This Chapter Is Really Teaching Us

Stepping Back for a Moment: What Chapter 2 Is Really About

By now, we’ve gone through a lot:

seven architectural layers,
dozens of tools,
multiple frameworks,
deployment strategies,
and evaluation challenges.

But before diving into more details, it’s worth pausing and asking:

“What is this chapter actually trying to teach?”

The answer is surprisingly simple:

AI agents are not a single technology. They are systems.

And systems fail or succeed not because of one brilliant component, but because of how everything fits together.

This chapter is less a catalog of tools and more a warning against naïve thinking.

Framework Selection: “Which One Should I Use?”

This is probably the most common question people ask after reading this chapter.

And the chapter’s answer is refreshingly honest:

“It depends.”

But that’s not a cop-out — it’s a reminder that different agent problems require different architectural choices.

Let’s unpack how the chapter suggests thinking about framework selection.

Don’t Start with the Framework — Start with the Problem

The chapter strongly discourages this approach:

“LangChain is popular, so we’ll use LangChain.”

Instead, it suggests starting with questions like:

Is this a short task or a long-running workflow?
Does the agent need memory across sessions?
Does it need to call many tools?
Is safety critical?
Will this run at scale?
Does this need multi-agent coordination?

Your answers determine the framework — not hype.

A Practical Way to Think About Major Frameworks

Rather than ranking frameworks, the chapter positions them by strengths and trade-offs.

LangChain: The Swiss Army Knife

LangChain is described as:

flexible,
powerful,
and easy to prototype with.

Why people love it:

huge ecosystem,
lots of integrations,
fast iteration.

Why people struggle with it:

abstractions can pile up,
debugging becomes hard,
production systems can get messy.

LangChain shines when:

you’re experimenting,
you’re learning agent patterns,
you’re building MVPs.

LangGraph: Structure and Control

LangGraph is presented as a response to LangChain’s flexibility.

Key idea:

make agent workflows explicit graphs.

Why this matters:

clearer state transitions,
fewer hidden loops,
easier debugging,
better production readiness.

LangGraph is ideal when:

workflows are complex,
failures must be handled gracefully,
you need determinism.

AutoGen: Conversations Between Agents

AutoGen takes a different approach.

Instead of:

one agent with tools,

It focuses on:

multiple agents talking to each other.

Examples:

planner agent delegates to executor agent,
critic agent reviews output,
manager agent coordinates.

This is powerful for:

research tasks,
coding,
collaborative reasoning.

But it also adds:

coordination overhead,
harder debugging,
higher compute cost.

LlamaIndex: Data-Centric Agents

LlamaIndex is built around one core belief:

“Most agents fail because they don’t understand their data.”

It shines when:

documents matter more than actions,
RAG is central,
knowledge retrieval dominates.

Think:

enterprise search,
research assistants,
compliance tools.

AutoGPT-Style Agents: Maximum Autonomy, Maximum Risk

Fully autonomous agents get a lot of attention — and the chapter treats them cautiously.

Pros:

minimal human involvement,
impressive demos,
long-horizon reasoning.

Cons:

unpredictable behavior,
runaway costs,
difficult safety guarantees,
hard to deploy responsibly.

The chapter’s tone here is clear:

Autonomy without constraints is not a virtue in production systems.

Single-Agent vs Multi-Agent: A Reality Check

Multi-agent systems sound exciting — and they are — but the chapter urges restraint.

When Single-Agent Systems Are Enough

Single agents work well when:

tasks are linear,
reasoning depth is moderate,
safety constraints are tight,
debugging simplicity matters.

They are easier to:

test,
monitor,
secure,
explain.

When Multi-Agent Systems Shine

Multi-agent systems are justified when:

problems are complex,
reasoning needs to be decomposed,
specialization improves outcomes,
collaboration mimics human teams.

But the chapter warns:

Multi-agent systems multiply complexity faster than they multiply intelligence.

Common Failure Modes (The “We Learned This the Hard Way” Section)

This is one of the most valuable parts of the chapter.

It reads like a list of mistakes that everyone makes at least once.

Failure #1: Treating Agents Like Chatbots

Many teams start with:

a chat interface,
add tools,
and call it an agent.

But agents require:

state,
memory,
planning,
constraints.

Without these, you get:

shallow reasoning,
repeated mistakes,
hallucinations.

Failure #2: Overloading the Prompt

Trying to do everything with:

longer prompts,
more instructions,
bigger system messages.

This leads to:

higher cost,
worse performance,
fragile behavior.

The chapter’s lesson:

Prompts are not architecture.

Failure #3: No Budget or Step Limits

Agents that:

loop endlessly,
retry forever,
explore too much,

will burn money and time.

Production agents need:

token budgets,
max iterations,
timeouts,
circuit breakers.

Failure #4: Ignoring Evaluation Until It’s Too Late

Many teams build agents first and ask:

“Is it good?” later.

By then:

behavior is inconsistent,
bugs are hard to trace,
users are unhappy.

The chapter emphasizes:

Evaluation must be designed before deployment, not after.

Failure #5: Assuming the Model Will “Figure It Out”

This is a big one.

People assume:

the model will reason correctly,
tools will be used properly,
safety will emerge naturally.

In reality:

models hallucinate,
tools get misused,
errors compound.

Robust agents assume failure is normal.

Organizational Reality: Tools Are Easy, Teams Are Hard

This chapter quietly makes another point:

Most AI agent problems are not technical — they are organizational.

Cross-Functional Collaboration Is Mandatory

Agent systems touch:

ML teams,
backend engineers,
security teams,
legal and compliance,
product and UX,
business stakeholders.

Siloed teams struggle.

Successful organizations:

align incentives,
share ownership,
communicate constantly.

Talent Shortage Is Real

Building agents requires people who understand:

LLM behavior,
systems engineering,
data pipelines,
safety and evaluation.

These people are rare — and expensive.

The chapter suggests:

Start simple. Grow capability gradually.

Scaling Reality: What Works in Demos Breaks in Production

The chapter repeatedly reminds us:

“Most agent demos are toys. Production is different.”

At scale, problems appear:

latency spikes,
cost explosions,
memory inconsistencies,
tool outages,
user edge cases.

This is why layers like:

infrastructure,
observability,
security,
exist at all.

How the Layers Fit Together (The Full Picture)

Let’s reassemble the stack one last time, in plain terms:

Layer 1 (Models) give raw intelligence
Layer 2 (Data) gives memory and grounding
Layer 3 (Frameworks) give structure and reasoning
Layer 4 (Infrastructure) makes it scalable
Layer 5 (Evaluation) keeps it sane
Layer 6 (Security) keeps it safe
Layer 7 (Ecosystem) delivers value

Remove any one layer, and the system degrades.

The Chapter’s Final Message (In Simple Language)

If Chapter 1 said:

“AI agents are coming”

Chapter 2 says:

“Building them well is hard — but doable.”

And its core lesson is this:

AI agents are not magic.
They are engineered systems that reward discipline and punish shortcuts.

Success comes from:

thoughtful architecture,
careful tool choice,
strong evaluation,
and realistic expectations.

Final Takeaway: Think Like a Systems Engineer, Not a Prompt Engineer

This chapter gently but firmly shifts the reader’s mindset.

The future of AI agents will not belong to:

people who write the cleverest prompts,
or chase the newest framework.

It will belong to those who:

understand system design,
respect complexity,
build incrementally,
and measure everything.

In short:

AI agents are software systems first — intelligence second.

And that’s what makes them powerful.

Pages

Friday, January 2, 2026

AI Agent Tools and Frameworks (Chapter 2)