langsmith-fetch | Apiara SkillsSkill
ComposioHQ/awesome-claude-skills62.1klangsmith-fetch
Debug LangChain and LangGraph agents by fetching execution traces from LangSmith Studio. Use when debugging agent behavior, investigating errors, analyzing tool calls, checking memory operations, or examining agent performance. Automatically fetches recent traces and analyzes execution patterns. Requires langsmith-fetch CLI installed.
Overview
# LangSmith Fetch - Agent Debugging Skill
Debug LangChain and LangGraph agents by fetching execution traces directly from LangSmith Studio in your terminal.
When to Use This Skill
Automatically activate when user mentions:
- đ "Debug my agent" or "What went wrong?"
- đ "Show me recent traces" or "What happened?"
- â "Check for errors" or "Why did it fail?"
- đž "Analyze memory operations" or "Check LTM"
- đ "Review agent performance" or "Check token usage"
- đ§ "What tools were called?" or "Show execution flow"
Prerequisites
1. Install langsmith-fetch
```bash
pip install langsmith-fetch
```
2. Set Environment Variables
```bash
export LANGSMITH_API_KEY="your_langsmith_api_key"
export LANGSMITH_PROJECT="your_project_name"
```
**Verify setup:**
```bash
echo $LANGSMITH_API_KEY
echo $LANGSMITH_PROJECT
```
Core Workflows
Workflow 1: Quick Debug Recent Activity
**When user asks:** "What just happened?" or "Debug my agent"
Validate with a real task
Run one small real task before keeping it in your long-term workflow.
langsmith-fetch traces --last-n-minutes 5 --limit 5 --format pretty
1. â
Number of traces found
2. â ī¸ Any errors or failures
3. đ ī¸ Tools that were called
4. âąī¸ Execution times
**Example response format:**
Found 3 traces in the last 5 minutes:
- Agent: memento
- Tools: recall_memories, create_entities
- Duration: 2.3s
- Tokens: 1,245
- Agent: cypher
- Error: "Neo4j connection timeout"
- Duration: 15.1s
- Failed at: search_nodes tool
- Agent: memento
- Tools: store_memory
- Duration: 1.8s
- Tokens: 892
đĄ Issue found: Trace 2 failed due to Neo4j timeout. Recommend checking database connection.
Workflow 2: Deep Dive Specific Trace
**When user provides:** Trace ID or says "investigate that error"
langsmith-fetch trace <trace-id> --format json
**Analyze JSON and report:**
1. đ¯ What the agent was trying to do
2. đ ī¸ Which tools were called (in order)
3. â
Tool results (success/failure)
4. â Error messages (if any)
5. đĄ Root cause analysis
**Example response format:**
Deep Dive Analysis - Trace abc123
Goal: User asked "Find all projects in Neo4j"
1. â
search_nodes(query: "projects")
2. â get_node_details(node_id: "proj_123")
â Error: "Node not found"
â This is the failure point
3. âšī¸ Execution stopped
The search_nodes tool returned node IDs that no longer exist in the database,
possibly due to recent deletions.
1. Add error handling in get_node_details tool
2. Filter deleted nodes in search results
3. Update cache invalidation strategy
Token Usage: 1,842 tokens ($0.0276)
Execution Time: 8.7 seconds
Workflow 3: Export Debug Session
**When user says:** "Save this session" or "Export traces"
# Create session folder with timestamp
SESSION_DIR="langsmith-debug/session-$(date +%Y%m%d-%H%M%S)"
langsmith-fetch traces "$SESSION_DIR/traces" --last-n-minutes 30 --limit 50 --include-metadata
# Export threads (conversations)
langsmith-fetch threads "$SESSION_DIR/threads" --limit 20
â
Session exported successfully!
Location: langsmith-debug/session-20251224-143022/
- Traces: 42 files
- Threads: 8 files
1. Review individual trace files
2. Share folder with team
3. Analyze with external tools
4. Archive for future reference
Workflow 4: Error Detection
**When user asks:** "Show me errors" or "What's failing?"
langsmith-fetch traces --last-n-minutes 30 --limit 50 --format json > recent-traces.json
grep -i "error\|failed\|exception" recent-traces.json
1. đ Total errors found
2. â Error types and frequency
3. đ When errors occurred
4. đ¯ Which agents/tools failed
**Example response format:**
Error Analysis - Last 30 Minutes
Failed Traces: 7 (14% failure rate)
1. Neo4j Connection Timeout (4 occurrences)
- Agent: cypher
- Tool: search_nodes
- First occurred: 14:32
- Last occurred: 14:45
- Pattern: Happens during peak load
2. Memory Store Failed (2 occurrences)
- Agent: memento
- Tool: store_memory
- Error: "Pinecone rate limit exceeded"
- Occurred: 14:38, 14:41
3. Tool Not Found (1 occurrence)
- Agent: sqlcrm
- Attempted tool: "export_report" (doesn't exist)
- Occurred: 14:35
1. Add retry logic for Neo4j timeouts
2. Implement rate limiting for Pinecone
3. Fix sqlcrm tool configuration
Common Use Cases
Use Case 1: "Agent Not Responding"
**User says:** "My agent isn't doing anything"
1. Check if traces exist:
langsmith-fetch traces --last-n-minutes 5 --limit 5
2. **If NO traces found:**
- Tracing might be disabled
- Check: `LANGCHAIN_TRACING_V2=true` in environment
- Check: `LANGCHAIN_API_KEY` is set
- Verify agent actually ran
- Review for errors
- Check execution time (hanging?)
- Verify tool calls completed
Use Case 2: "Wrong Tool Called"
**User says:** "Why did it use the wrong tool?"
1. Get the specific trace
2. Review available tools at execution time
3. Check agent's reasoning for tool selection
4. Examine tool descriptions/instructions
5. Suggest prompt or tool config improvements
Use Case 3: "Memory Not Working"
**User says:** "Agent doesn't remember things"
1. Search for memory operations:
langsmith-fetch traces --last-n-minutes 10 --limit 20 --format raw | grep -i "memory\|recall\|store"
- Were memory tools called?
- Did recall return results?
- Were memories actually stored?
- Are retrieved memories being used?
Use Case 4: "Performance Issues"
**User says:** "Agent is too slow"
langsmith-fetch traces ./perf-analysis --last-n-minutes 30 --limit 50 --include-metadata
- Execution time per trace
- Tool call latencies
- Token usage (context size)
- Number of iterations
- Slowest operations
3. Identify bottlenecks and suggest optimizations
Output Format Guide
Pretty Format (Default)
langsmith-fetch traces --limit 5 --format pretty
**Use for:** Quick visual inspection, showing to users
JSON Format
langsmith-fetch traces --limit 5 --format json
**Use for:** Detailed analysis, syntax-highlighted review
Raw Format
langsmith-fetch traces --limit 5 --format raw
**Use for:** Piping to other commands, automation
Advanced Features
Time-Based Filtering
# After specific timestamp
langsmith-fetch traces --after "2025-12-24T13:00:00Z" --limit 20
# Last N minutes (most common)
langsmith-fetch traces --last-n-minutes 60 --limit 100
Include Metadata
langsmith-fetch traces --limit 10 --include-metadata
# Metadata includes: agent type, model, tags, environment
Concurrent Fetching (Faster)
langsmith-fetch traces ./output --limit 100 --concurrent 10
Troubleshooting
"No traces found matching criteria"
1. No agent activity in the timeframe
# 1. Try longer timeframe
langsmith-fetch traces --last-n-minutes 1440 --limit 50
# 3. Try fetching threads instead
langsmith-fetch threads --limit 10
# 4. Verify tracing is enabled in your code
# Check for: LANGCHAIN_TRACING_V2=true
"Project not found"
langsmith-fetch config show
export LANGSMITH_PROJECT="correct-project-name"
# Or configure permanently
langsmith-fetch config set project "your-project-name"
Environment variables not persisting
# Add to shell config file (~/.bashrc or ~/.zshrc)
echo 'export LANGSMITH_API_KEY="your_key"' >> ~/.bashrc
echo 'export LANGSMITH_PROJECT="your_project"' >> ~/.bashrc
Best Practices
1. Regular Health Checks
# Quick check after making changes
langsmith-fetch traces --last-n-minutes 5 --limit 5
2. Organized Storage
â âââ 2025-12-24/
â âââ 2025-12-25/
âââ performance-tests/
3. Document Findings
1. Export the problematic trace
2. Save to `error-cases/` folder
3. Note what went wrong in a README
4. Share trace ID with team
4. Integration with Development
langsmith-fetch traces --last-n-minutes 10 --limit 5
langsmith-fetch trace <error-id> --format json > pre-commit-error.json
Quick Reference
langsmith-fetch traces --last-n-minutes 5 --limit 5 --format pretty
langsmith-fetch trace <trace-id> --format pretty
langsmith-fetch traces ./debug-session --last-n-minutes 30 --limit 50
langsmith-fetch traces --last-n-minutes 30 --limit 50 --format raw | grep -i error
langsmith-fetch traces --limit 10 --include-metadata
Resources
- **LangSmith Fetch CLI:** https://github.com/langchain-ai/langsmith-fetch
- **LangSmith Studio:** https://smith.langchain.com/
- **LangChain Docs:** https://docs.langchain.com/
- **This Skill Repo:** https://github.com/OthmanAdi/langsmith-fetch-skill
Notes for Claude
- Always check if `langsmith-fetch` is installed before running commands
- Verify environment variables are set
- Use `--format pretty` for human-readable output
- Use `--format json` when you need to parse and analyze data
- When exporting sessions, create organized folder structures
- Always provide clear analysis and actionable insights
- If commands fail, help troubleshoot configuration issues
**Author:** Ahmad Othman Ammar Adi
**Repository:** https://github.com/OthmanAdi/langsmith-fetch-skill