Picking an agent framework is less about which demo looks best and more about what your team must operate for years. Language fit, how state survives a crash, where humans approve actions and who owns tracing all matter more after launch than before it.
This guide compares nine open-source frameworks on the same eight criteria. Every cell comes from each project's own repository or documentation, read on October 29, 2026. Where official material did not confirm a feature, we say so rather than guess.
How We Evaluated
We did not run benchmarks or build the same agent nine times. Instead, we read each project's official repository and documentation pages and recorded what they state. The same eight criteria were applied to every framework:
• Languages: which official SDKs exist
• Orchestration model: how agents and steps are coordinated
• State and durability: whether runs can persist, checkpoint or resume
• Human-in-the-loop: whether pausing for approval or input is documented
• Tool interoperability: whether the Model Context Protocol (MCP) is documented
• Observability: what tracing or telemetry is built in or recommended
• License: the license on the main repository
• Vendor coupling: any paid platform or cloud the project points to for hosting
"Not confirmed" means neither the README nor the docs pages we read stated it. The feature may still exist deeper in the docs, so check before ruling a framework out.
We picked frameworks that are code-first, actively maintained (each repository had commits in October 2026) and aimed at production agents. We left out hosted no-code platforms, which our agentic AI platforms guide covers.
The Comparison at a Glance
| Framework | Languages | Orchestration | State and durability | Human-in-the-loop | MCP | Observability | License |
|---|---|---|---|---|---|---|---|
| LangGraph | Python, JavaScript | Graph of nodes and edges | Durable execution, memory | Interrupts documented | Via LangChain MCP integration | LangSmith (separate product) | MIT |
| CrewAI | Python | Crews (roles) and Flows (events) | Checkpointing | Human input documented | Documented, plus A2A | Tracing and observability | MIT |
| Microsoft Agent Framework | Python, .NET (Go in a separate repo) | Graph workflows: sequential, concurrent, handoff, group | Checkpointing, durable extension | Documented | Documented, plus A2A | OpenTelemetry built in | MIT |
| OpenAI Agents SDK | Python (separate JS/TS SDK) | Agents with handoffs or agents-as-tools | Sessions for conversation history | Documented | Documented | Built-in tracing | MIT |
| Google ADK | Python, Java, Go, TypeScript, Kotlin | Agents and nested workflows | Sessions; cloud and database persistence options | Documented | Documented | OpenTelemetry documented | Apache-2.0 |
| Strands Agents | Python, TypeScript | Model-driven agent loop, multi-agent patterns | Session management | Interrupts documented | Documented | Observability and tracing docs | Apache-2.0 |
| Pydantic AI | Python | Typed agents | Durable execution integrations, step persistence | Approval-required deferred tools | Documented | OpenTelemetry (Logfire optional) | MIT |
| Mastra | TypeScript | Agents plus graph workflow engine | Suspend and resume, memory | Suspend for approval documented | MCP servers documented | Observability and evals docs | Apache-2.0, except `ee/` code |
| LlamaIndex | Python | Workflows and agents | Workflow context can be saved and resumed | Documented | Documented | Not confirmed | MIT |
Read the table as a shortlist filter, not a verdict. Two frameworks with the same cell can still differ a lot in how mature or ergonomic that feature is.
The Nine Frameworks, One by One
1. LangGraph
LangGraph models an agent as a graph of steps with shared state. Its README highlights durable execution, so agents "persist through failures" and resume where they stopped, plus human-in-the-loop and short- and long-term memory.
• Best fit: teams that want explicit control over every step and branch
• Watch for: MCP comes through LangChain's MCP integration rather than LangGraph itself, and observability and managed deployment point to LangSmith, a separate commercial product; budget for it or wire your own tracing
2. CrewAI
CrewAI offers two styles: Crews, where role-based agents collaborate, and Flows, which give event-driven, step-by-step control. Its README lists checkpointing, human input, structured outputs and MCP and A2A support.
• Best fit: teams that think in roles ("researcher," "reviewer") and want a fast start
• Watch for: the README positions CrewAI AMP, a paid platform, for managed deployment, governance and enterprise support
3. Microsoft Agent Framework
Microsoft Agent Framework is the successor to AutoGen, and Microsoft publishes migration guides from both AutoGen and Semantic Kernel. The AutoGen repository now says it is archived and advises new users to start here.
It supports Python and .NET with consistent APIs, and Microsoft publishes a separate Go repository. Workflows are graph-based, with checkpointing, streaming and human-in-the-loop. OpenTelemetry is built in.
• Best fit: .NET shops and teams already on Azure or migrating from AutoGen
• Watch for: if you run AutoGen today, plan the migration before adding features
4. OpenAI Agents SDK
The OpenAI Agents SDK keeps a small surface: agents with instructions, tools, guardrails and handoffs. Sessions manage conversation history across runs, with an optional Redis backend. Tracing is built in, and human-in-the-loop has its own docs page.
• Best fit: teams that want a light framework and fast iteration
• Watch for: durable, long-running workflows need more than session history; check whether you need an external workflow engine
5. Google Agent Development Kit (ADK)
Google ADK is a code-first toolkit with official SDKs in Python, Java, Go and TypeScript. Its docs also show Kotlin. Its README lists OpenAPI and MCP tools, human-in-the-loop and nested workflows. It is deployment-agnostic, with paths to Cloud Run or Vertex AI Agent Engine.
• Best fit: polyglot teams and Google Cloud users
• Watch for: in-memory sessions are lost on restart, so plan for the documented database or cloud session options. Observability is documented with OpenTelemetry
6. Strands Agents
Strands Agents is an open-source SDK for Python and TypeScript. The code now lives in a monorepo named harness-sdk, which also holds a batteries-included "harness" layer. It documents MCP, multi-agent patterns, session management, interrupts for human approval, observability and evals.
Model support is broad. The README names Amazon Bedrock, Anthropic, OpenAI and Gemini as first-class, plus Ollama and others.
• Best fit: AWS-leaning teams that still want provider choice
• Watch for: the repository rename means older tutorials may point to old paths
7. Pydantic AI
Pydantic AI brings typed, validated outputs to agents. Its README shows durable execution through integrations, including DBOS and Prefect, and a step persistence feature to save, resume and fork runs without a separate engine.
Instrumentation is plain OpenTelemetry. Pydantic Logfire is offered, but any OpenTelemetry backend works.
• Best fit: Python teams that value type safety and testability
• Watch for: its docs note that tool approval is not an authorization boundary against an untrusted client, so enforce permissions server-side
8. Mastra
Mastra is a TypeScript framework for agents and AI apps. It combines agents with a graph-based workflow engine. Workflows and agents can suspend to wait for user input or approval, then resume. MCP servers, memory, evals and observability are documented.
• Best fit: TypeScript and Next.js teams who want agents in the same stack as their app
• Watch for: code in any `ee/` directory is source-available under the Mastra Enterprise License and needs a valid license for production use; everything else is Apache-2.0
9. LlamaIndex
LlamaIndex built its name on data connectors and retrieval, and offers Workflows for agents. Its docs cover human-in-the-loop tools and MCP. Its README now states that the company's focus has shifted to LlamaParse, its document platform, while the open-source framework remains available.
• Best fit: retrieval-heavy agents over documents, especially if you already use LlamaIndex
• Watch for: weigh the stated focus shift when you plan a multi-year build
Choosing: Five Questions That Narrow the List
➤ What language will you maintain this in? If the answer is TypeScript, start with Mastra, Strands, the OpenAI JS SDK, LangGraph JS or ADK TypeScript. For .NET, Microsoft Agent Framework stands out.
➤ Can a run take minutes, hours or days? Long runs need checkpointing or durable execution. Shortlist LangGraph, Microsoft Agent Framework, Pydantic AI (with an integration), CrewAI and Mastra, then test a crash and resume.
➤ Which actions need a human signature? Refunds, emails to customers and record changes usually do. Prefer frameworks with documented pause-and-approve patterns.
➤ Where will traces go? If you already run an OpenTelemetry backend, frameworks that emit OpenTelemetry fit with less glue. Our guide to LLM observability metrics covers what to track.
➤ What happens if the vendor's priorities change? Check the license, the paid platform each project promotes and whether a migration path exists.
A Two-Week Bake-Off Template
Comparison tables only go so far. A short, structured trial on your own use case shows more. Here is a template you can adapt to your own workflow.
Pick two or three finalists. More than three dilutes the effort.
Build the same small agent in each. For example: read a support ticket, look up the customer in a test CRM, draft a reply and wait for human approval before sending.
Score each build on the same sheet:
| Test | What to record |
|---|---|
| Kill the process mid-run | Does it resume, restart or lose work? |
| Approval step | Lines of code to pause, notify and resume |
| Tool failure | What the agent does when the CRM call times out |
| Trace review | Can a reviewer see every model and tool call in one place? |
| Prompt-injection test | Does a malicious instruction in the ticket trigger a tool call? |
| Upgrade | How painful is moving to the next minor version? |
The prompt-injection row matters because frameworks do not stop it for you. See our guide to agent security, prompt injection and tool actions.
Keep notes on developer time as well as results. The framework your team debugs fastest at 2 a.m. is often the right one.
Common Mistakes When Choosing
• Choosing on stars or hype. Popularity says little about fit with your language, cloud or approval needs.
• Ignoring the paid layer. Several projects point to a commercial platform for deployment or tracing; price it in early.
• Treating multi-agent as a goal. One well-scoped agent with good tools often beats a crew of agents.
• Skipping the crash test. Durability claims mean little until you kill a run and watch what happens.
• Locking into one model. Check how each framework handles model providers before you commit.
Our Own Product Note
Disclosure: this is our own service. LuMay's AI agent development team builds production agents for clients and can help run a framework bake-off. We are not ranked in this list.
Next Step
Shortlist two or three frameworks with the five questions above, then run the two-week bake-off. If you want help, talk to our AI engineering services team.
References
• LangGraph README (GitHub): https://github.com/langchain-ai/langgraph
• LangGraph docs, Durable execution: https://docs.langchain.com/oss/python/langgraph/durable-execution
• CrewAI README (GitHub): https://github.com/crewAIInc/crewAI
• CrewAI docs, Human input on execution: https://docs.crewai.com/en/learn/human-input-on-execution
• Microsoft Agent Framework README (GitHub): https://github.com/microsoft/agent-framework
• Microsoft Learn, Agent Framework documentation: https://learn.microsoft.com/en-us/agent-framework/
• Microsoft Learn, Migration guide from AutoGen: https://learn.microsoft.com/en-us/agent-framework/migration-guide/from-autogen
• AutoGen README (archive notice): https://github.com/microsoft/autogen
• OpenAI Agents SDK README (GitHub): https://github.com/openai/openai-agents-python
• OpenAI Agents SDK docs, Human in the loop: https://openai.github.io/openai-agents-python/human_in_the_loop/
• OpenAI Agents SDK docs, Tracing: https://openai.github.io/openai-agents-python/tracing/
• Google ADK README (GitHub): https://github.com/google/adk-python
• Strands Agents monorepo README (GitHub): https://github.com/strands-agents/harness-sdk
• Strands docs, Session management: https://strandsagents.com/latest/documentation/docs/user-guide/concepts/agents/session-management/
• Pydantic AI README (GitHub): https://github.com/pydantic/pydantic-ai
• Pydantic AI docs, Durable execution: https://pydantic.dev/docs/ai/capabilities/durable_execution/overview/
• Mastra README (GitHub): https://github.com/mastra-ai/mastra
• Mastra docs, Suspend and resume: https://mastra.ai/docs/workflows/suspend-and-resume
• Mastra LICENSE: https://github.com/mastra-ai/mastra/blob/main/LICENSE.md
• LlamaIndex README (GitHub): https://github.com/run-llama/llama_index
• LlamaIndex docs, Human in the loop: https://developers.llamaindex.ai/python/framework/understanding/agent/human_in_the_loop/
• LlamaIndex docs, MCP: https://developers.llamaindex.ai/python/framework/module_guides/mcp/
• LangChain docs, MCP: https://docs.langchain.com/oss/python/langchain/mcp
• LangGraph docs, Interrupts: https://docs.langchain.com/oss/python/langgraph/interrupts
• Google ADK docs, Sessions: https://google.github.io/adk-docs/sessions/
• Google ADK docs, Observability: https://google.github.io/adk-docs/observability/
• Strands docs, Interrupts: https://strandsagents.com/latest/documentation/docs/user-guide/concepts/interrupts/
• Pydantic AI docs, Deferred tools: https://pydantic.dev/docs/ai/tools-toolsets/deferred-tools/
• Microsoft Agent Framework for Go (GitHub): https://github.com/microsoft/agent-framework-go
