Concept guide · LangChain, LangGraph, and CrewAI
How do you test a framework-based agent?
A slower explanation for when the short interview answer is not enough. Start with the intuition, then connect it to the technical idea and the way an interviewer expects you to reason about it.
ML area
Generative AI → Framework testing
Interview format
Advanced · Defense
Question
GENAI-099
Start with the idea
In simple terms
Test the building blocks independently, then test the complete chain, graph, or crew with fixed fixtures and failure scenarios. A framework does not make an agent deterministic or correct by itself.
Make it concrete
Mock the retriever and tool responses to test routing, then run a smaller number of live evaluations to measure provider behaviour, cost, and final task success.
Build the answer
How to defend your decision
- 1Separate deterministic application logic from probabilistic model behaviour.
- 2Inject tool, provider, retrieval, and human-review failures deliberately.
- 3Use trace and cost assertions alongside answer-quality evaluation.
Go one level deeper
The technical explanation
Unit-test state transformations, schemas, permissions, and tool adapters. Integration-test model and retriever boundaries. Scenario-test loops, retries, interrupts, malformed output, slow tools, and partial failures. Keep traces and assert state transitions so a framework upgrade cannot silently change the path while preserving the final text in one happy-path test.
Say this in the interview
Test the underlying tools and nodes independently, then run deterministic fixtures through the chain or graph, followed by scenario and failure tests. Capture framework traces, assert state transitions and permissions, inject tool failures, and evaluate the final task outcome and cost.
Keep preparing