Agent harnesses and runtimes
The harness decides what the model sees, which tools it may use, where its state lives, and what happens when a run stops for an approval and resumes an hour later on another machine. I have worked on that layer for assistants that carry multi-step work through to the end, from proof of concept to a phased production rollout.
- Planning and delegation: a supervisor breaks work into steps and hands them to sub-agents, each with scoped context and tools.
- Durable execution. Every step is checkpointed, so a run survives interrupts, crashes and deploys.
- Human approval modelled as a state of the run, not an instruction in a prompt.
- Context engineering: what to load, what to compact and what to drop as a run grows.
- Streaming APIs kept apart from the workers that do the long tasks.
- Sandboxed code execution for analysis and generated files.
LangGraph · Deep Agents · FastAPI · Postgres
Multi-agent architecture
Some problems need several kinds of expert judgement on the same input. The architecture I have worked on gives each kind its own specialist agent, runs them concurrently under one orchestrator, and reconciles their typed results into a single structured output. The real design work is at the boundaries: what each agent may see, how its output is validated, which model each role warrants, and how cost and latency are budgeted across a run.
- Fan-out and fan-in under one orchestrator, so a run takes as long as its slowest specialist, not the sum of all of them.
- Typed, validated contracts between agents. Nothing downstream parses prose.
- Model routing per role, sized to the job instead of the largest model everywhere.
- Cache-aware prompt layout, with cost accounting that counts cache discounts.
- Tracing across every agent and tool call in a run.
async Python · Pydantic · structured outputs · Langfuse
MCP servers
Model Context Protocol servers that put real services within reach of AI assistants, published on the Anthropic and OpenAI app marketplaces. The interesting problems are the ones demos skip: identity, large files, and tools that take minutes rather than seconds.
- A small, deliberate tool surface, designed around how models actually call tools.
- Large files go straight to object storage through presigned URLs, never through the model.
- Results rendered as interactive UI resources that follow the host’s theme.
- OAuth 2.0, with every token verified against an identity provider.
- Asynchronous jobs for long-running tools, with a status the model can poll.
TypeScript · MCP SDK · OAuth 2.0 · React
Batch LLM pipelines
Not every model call has a person waiting on it. Some work is thousands of items scored, labelled or extracted in one run. The pipelines I have worked on are built to be cheap to run, safe to interrupt and easy to audit.
- Jobs submitted through an API, with inputs and results kept in object storage.
- A queue feeding containerised workers that scale with its depth, and to zero between runs.
- A larger model where judgement matters, a smaller one for retries and easy cases.
- Per-item checkpoints, so an interrupted run resumes instead of starting over.
- A written rationale with every output, so any result can be checked.
Python · AWS · queues · containers
Model platform
One interface to every model the products use: frontier models across Anthropic, OpenAI, Google and Bedrock, and in-house small and fine-tuned models served alongside them. Text and image, sync, async, streaming and batch. Configuration, fallbacks and staged rollouts live in one place, so a model change ships like any other release, measured. It serves about 1.2 million requests on an average day, which is also why it is where cost is won or lost.
- Caching audited per request, because a cache that looks enabled is not the same as one that hits.
- Prompts ordered so the stable, cacheable part always comes first.
- Batch APIs for work that can wait, real time only where someone is waiting.
- Throughput work: workload identity, rate limits, and concurrency that queues bursts instead of failing them.
Python · Anthropic · OpenAI · Gemini · Bedrock