Tag: observability

A practical 2026 map of AI agent frameworks by infrastructure primitive: orchestration, tools, state, multi-agent delegation, approvals, observability, evaluation, and deployment.

Outcome: Reframed agent-framework selection away from tier lists and toward an operating contract for which primitives the framework owns, which ones the team must own, and where bespoke orchestration is still justified.

Why custom LLM logging leaves you flying blind in production, and how OpenTelemetry's GenAI semantic conventions turn every model call, tool invocation, and agent step into a traceable, cost-accountable span.

Outcome: Reader can instrument an LLM pipeline or agent workflow with OTEL GenAI conventions, export spans and cost metrics to any compatible backend, and build alerts on real token spend and latency instead of inferring from flat logs.

A practical evaluation loop for multi-agent workflows that catches demo-friendly failures in task handoff, tool use, permissions, latency, and completion criteria before release.

Outcome: Established a repeatable evaluation workflow that gates multi-agent releases on task completion, handoff quality, tool correctness, latency, and recoverability instead of demo impressions.