Grok 4.6 ships for long-running agents: $2/$6, live in Cursor and Grok Build, matches Sol on AA Intelligence Index
August 12, 2026 · via xAI
Topic Pillar
AI Observability.You cannot fix what you cannot see.
Observability is the unsexy half of production AI that decides whether you keep your job. Per-step traces, score-bearing evals, cache-hit telemetry, tool-selection scores — these are the primitives that let you debug agents instead of guessing at them.
● 86 cluster pages· 32 posts· 4 notes· 49 updates· 1 events
The four telemetry layers
Input/output traces per step (Langfuse-shaped). Eval scoring on representative inputs, not just happy paths. Cache-hit rate per route — the new throughput metric. Tool-selection score deltas from the model boundary. Without all four, multi-agent debugging is folklore.
Eval datasets that surface real bugs
A 100-case eval set that scores 95% means nothing if all 100 cases are the happy path. The cases that matter are the ambiguous ones, the conflicting ones, the malformed ones. Build the eval set adversarially — ask "what would a real user actually paste in?" — and most agents drop 15–25 points.
Per-agent cost attribution
Tools like Langfuse now expose per-agent cost attribution and step-level cache-hit telemetry. You will discover 30–50% of your agent traffic is repeat queries you should be caching. This is the report nobody wants to read but everyone needs to.
32 blog posts
Deep dives on AI Observability
Production
Claude cyber-eval incidents: the sandbox escape checklist I run before any offensive agent test
On July 30, 2026 Anthropic disclosed three incidents where Claude models reached live production systems during third-party cybersecurity evaluations. The root cause was a harness misconfiguration that left internet access open while prompts claimed there was none. Here is the defense-in-depth checklist I now require before any capture-the-flag or offensive agent eval.
Jul 31, 202616 min
Read the post Multi-AgentCursor's agent swarm economics: why frontier planners and cheap workers beat solo GPT bills
Cursor's July 2026 swarm write-up rebuilt SQLite in Rust from the docs and measured planner vs worker spend. Frontier-only GPT-5.5 hit about $10.5k; Opus 4.8 planning with Composer 2.5 workers landed near $1.3k at similar quality. Here is how I translate that into routing rules for production graphs.
Jul 29, 202615 min
Read the post MCPMCP GA is four days out: the final cutover checklist I run before July 28 goes live
The MCP 2026-07-28 final specification publishes July 28. You already did inventory, header security, and handle migration. These four days are cutover discipline: freeze pins, delete sticky sessions only after round-robin passes, confirm cacheScope with two identities, and staff an on-call window for clients still sending Mcp-Session-Id.
Jul 24, 202613 min
Read the post ProductionGemini 3.6 Flash shipped, 3.5 Pro did not: the routing checklist I run after July 21
On July 21, 2026 Google released Gemini 3.6 Flash ($1.50/$7.50 per million), 3.5 Flash-Lite ($0.30/$2.50, ~350 tok/s), and a limited-access 3.5 Flash Cyber via CodeMender. Gemini 3.5 Pro stays in partner testing. Here is how I re-benchmark cost per completed task against Sonnet 5 and Fable without betting the roadmap on Pro.
Jul 21, 202614 min
Read the post ProductionClaude Code Week 29: live MCP artifacts change the blast radius of a shared dashboard
Claude Code Week 29 (July 13–17, v2.1.207–v2.1.212) lets published artifacts call MCP connectors on each view through the viewer's own connections, with first-call approval. Public sharing links, editor roles, screen reader mode, and auto-background for long MCP calls shipped in the same window. Treat live artifacts like production integrations, not pretty exports.
Jul 17, 202612 min
Read the post ProductionClaude Code Week 28: the desktop browser and /doctor checklist I run before enabling it org-wide
Claude Code Week 28 (July 6–10, releases v2.1.202 through v2.1.206) shipped a sandboxed in-app browser on Desktop and upgraded /doctor from a read-only report into a repair tool. Auto mode also blocks transcript tampering. Here is the governance checklist I run before teams turn browsing and auto-fix on for every engineer.
Jul 14, 202612 min
Read the post ArchitectureThe maker is not the verifier: how I build self-improving agent loops without pretending models self-learn
Most teams prompt harder, get a better answer, and start over tomorrow. That is not compounding. Self-learning updates model weights from experience; no public model including Fable 5 does that today. Self-improving means the system gets better: run, log, distill, repeat. The golden rule is maker ≠ verifier. Here is the four-layer architecture and loop patterns I ship.
Jul 9, 202614 min
Read the post ProductionClaude Cowork is not Claude Code for civilians: the knowledge-worker playbook after the mobile launch
Anthropic shipped Claude Cowork on mobile and web July 8, starting with Max subscribers. Usage data from 1.2 million sessions shows more than 90% of Cowork work is non-technical: memos, RFP reviews, inbox triage, decks. Tasks run in the cloud, sync across devices, and continue when you close the app. Here is how I govern Cowork without treating it like a coding agent.
Jul 8, 202614 min
Read the post ArchitectureHow to actually use Fable 5: the four-layer architecture behind Mythos-tier results
Fable 5 is back globally, but most teams use it like a bigger Sonnet: prompt harder, better answer, start over tomorrow. Mythos-tier models need Mythos-tier systems: primitives, orchestration, memory, and self-improvement. The golden rule is maker ≠ verifier. Here is the architecture I draw on every engagement after the July 7 billing cliff.
Jul 7, 202614 min
Read the post ProductionClaude Sonnet 5 is the new default: the migration checklist I run before swapping claude-sonnet-4-6 in production
Anthropic shipped Sonnet 5 on June 30 as default on Free and Pro chat and as claude-sonnet-5 on the Platform API at $2/$10 per million through August 31. Three breaking changes matter for agent builders: adaptive thinking on by default, manual extended thinking removed, and non-default sampling parameters return 400. The tokenizer adds roughly 30% tokens for the same text. Here is the swap checklist.
Jul 5, 202614 min
Read the post ProductionFable 5 is back globally: the redeployment routing checklist I run before July 7
Anthropic restored Claude Fable 5 worldwide on July 1 after US export controls lifted June 30. Pro through Enterprise plans get up to 50% of weekly usage limits included through July 7, then usage credits. A new safety classifier blocks the Amazon-reported jailbreak in 99%+ of cases. Here is how I re-promote Fable without repeating June's config chaos.
Jul 4, 202614 min
Read the post ProductionOne agent spend dashboard for Cursor, Claude Code, and Copilot: what Copilot's ai_credits_used field unlocks
GitHub added ai_credits_used to the Copilot usage metrics API on June 19. It is a per-user total, not yet split by feature or model, but it closes a gap I have been papering over with spreadsheets. Here is how I unify Copilot credits with Claude API keys and Cursor team usage into one attributable spend view.
Jul 1, 202613 min
Read the post ProductionFable 5 got suspended worldwide in three days: the frontier model adoption checklist I now run on every engagement
Anthropic launched Claude Fable 5 on June 9, promised included subscription access through June 22, then suspended Fable and Mythos 5 globally on June 12 after a US export-control directive. Most teams never finished piloting. Here is the governance checklist I use when capability, retention, pricing, and access can flip faster than your rollout calendar.
Jun 25, 202614 min
Read the post ProductionClaude Code Artifacts turn terminal output into live review pages: what Team and Enterprise buyers should pilot first
Artifacts in Claude Code beta publish self-contained HTML to claude.ai that republishes to the same URL as the session progresses, with version history and org-only sharing. Strict CSP, no external fetch, no backend. Requires Team or Enterprise and claude.ai login. Here is the workflow I use for PR walkthroughs and incident timelines without screenshot threads in Slack.
Jun 22, 202613 min
Read the post ProductionAgentjacking is real: poisoned Sentry errors can hijack Cursor, Claude Code, and Codex without touching your repo
Tenet Threat Labs injected a fake stack trace through a public Sentry DSN and watched 100+ coding agents execute attacker commands during normal triage. No git write access required. The agent treats the error as ground truth. Here is how I harden observability MCP feeds, scope triage prompts, and block auto-exec on untrusted telemetry.
Jun 17, 202613 min
Read the post ProductionThe June 15 Claude billing change: Agent SDK credits, model retirement, and the checklist I run before anything breaks
Two Anthropic changes land on the same day: programmatic Claude usage moves to a separate monthly credit pool, and claude-opus-4-20250514 plus claude-sonnet-4-20250514 stop answering on the API. Interactive Claude Code is fine. Cron jobs and CI agents are not. Here is how I audit auth paths, claim credits, and grep for retiring model IDs before the first failed run.
Jun 15, 202614 min
Read the post ProductionGoverning agent autonomy in 2026: Auto-review, pre-push review, and why approval prompts are not a security model
Cursor made Auto-review the default run mode and shipped /review so Bugbot runs before you push. Together they treat agent autonomy as a dial: low-stakes actions flow, high-stakes actions slow down. Here is how I wire that pattern into local agents, SDK headless runs, and CI without mistaking convenience for a hard security boundary.
Jun 11, 202614 min
Read the post ArchitectureAgentic RAG vs vanilla RAG: why a Sufficient Context Agent beats retrieve-then-pray
Google Research shipped Agentic RAG on Gemini Enterprise with a Sufficient Context Agent that refuses to answer when retrieval is incomplete. On factuality benchmarks they report up to 34% higher accuracy versus standard RAG. Here is when one-shot RAG is still enough, when you need iterative retrieval, and how I wire the pattern without blowing latency budgets.
Jun 6, 202614 min
Read the post ProductionAgentic transformation is an operating-model problem, not a model problem
Microsoft published a 6-step playbook for rolling agents out across an enterprise, and the line that matters is "you do not need a bigger model, you need a better operating model." That matches what I see in consulting: the pilots that die do not die on model quality, they die on ownership, evals, and governance. Here is how I read the playbook for IT services teams, and the operating-model gaps that actually stall agent rollouts.
Jun 4, 202611 min
Read the post ArchitectureYour coding agent has amnesia. Persistent memory is the fix.
Claude Code forgets your architecture, your decisions, and why you ruled things out the moment a session ends. The reliability tax is not tokens, it is re-establishing context every morning. Here is what persistent agent memory actually is, how an open-source engine like Cortex implements it, and how to evaluate a memory layer for your own agents.
May 29, 202611 min
Read the post ProductionYour agent's supply chain is the attack surface now
A poisoned VS Code extension spent eighteen minutes on the marketplace and walked off with Claude Code credentials and MCP configs. The model was never the target. Your agent's supply chain is: the extensions, skills, MCP servers, tool definitions, and keys it is allowed to touch. Here is how I harden all four layers, and the checklist I run on every deployment.
May 27, 202612 min
Read the post ProductionHow an agentic studio screens, scores and shortlists candidates for your hiring team
Open Recruiting Atelier and you do not see a generic AI dashboard. You see five named specialists doing the work a screening team would do: catching duplicates, checking the brief, scoring on four dimensions, ranking, drafting the dispatch. Drop one CV or fifty. Click any candidate to see exactly why they landed where they did. This is what AI for recruitment looks like when it respects your judgment instead of replacing it.
May 24, 202610 min
Read the post Tool DesignTool registry design for agentic AI: how the wrong registry kills accuracy before the prompt is read
I reviewed a system last month with 47 tools in its registry and a 22 percent wrong-tool-selection rate. The team was about to migrate from Sonnet to Opus to fix it. The prompt was fine. The registry was the bug. This is the audit pattern I run on every client codebase before we change anything else, the seven failure modes I see in production, and the numbers from the cleanup.
May 22, 202612 min
Read the post ArchitectureAI agent vs agentic AI: what the distinction actually means when you ship one
Vendors blur the line because "agentic" sells. The two terms describe different architectures, with different cost shapes, different observability needs, and different scoping conversations. Here is the framing I use with clients and the three-question test for which one your project actually needs.
May 22, 202612 min
Read the post ProductionMCP governance just became a product: what Databricks Unity AI Gateway changes for enterprise agents
Every enterprise MCP deployment I have audited in the last six months has been hand-rolling tool-access policy, payload logging, and per-team cost limits on top of a gateway someone wrote in two days. Databricks just shipped that as a product. Here is what it actually changes, where the gaps still are, and the migration I would run for a Databricks shop.
May 20, 202612 min
Read the post Tool DesignTool descriptions are prompts. Fix the registry, not the agent.
When an agent picks the wrong tool, the registry is broken, not the agent. Three rules I now apply before debugging anything in a multi-tool system: precise names, "when to use" triggers, and a curated load list. Anthropic's new tool-selection telemetry finally puts numbers on what changes accuracy.
May 13, 20269 min
Read the post ProductionThe cheapest LLM call is the one you do not make. GitHub's 19-62% token cut, decoded
GitHub published an instrumented analysis of their agentic CI workflows and reported 19-62% token-cost reductions. The savings are the headline. The technique (pre-agentic data fetching and tool-registry hygiene) is the story most teams will miss.
May 11, 20269 min
Read the post ArchitectureClaude Opus 4.7's 1M context: when to RAG and when to just stuff it
A million tokens reliably is real now, but it does not retire RAG. It changes the calculus. Cost, latency, recency, and the prompt-cache angle nobody is talking about.
May 6, 20268 min
Read the post ProductionPrompt caching is not optional anymore. Measuring a 47% cost drop
A walkthrough from a client engagement: identifying stable prefixes, restructuring the system prompt for cacheability, and the telemetry that proved caching was actually working.
Apr 19, 20267 min
Read the post ProductionThe agent observability stack we ship to every client
Traces, spans, evals, cost-per-completed-task, and the one dashboard panel that catches 80% of regressions. Vendor-agnostic; covers Langfuse, Honeycomb, and rolling your own.
Mar 28, 20268 min
Read the post ProductionEval datasets: stop testing your agents on the happy path
If your eval set is the demos you showed the client, you are testing the wrong thing. How we build evals from production failures and the minimum viable suite to ship.
Jan 19, 20268 min
Read the post ArchitectureRAG vs CAG: when to retrieve, when to cache, when to combine
A decision framework from real implementations. RAG retrieves. CAG stores in cache. Knowing which to use, and when to combine both, determines whether your agent finds the right answer at the right cost.
Sep 21, 20257 min
Read the post 4 carousel notes
Visual breakdowns on AI Observability
49 ship-news updates
Latest in AI Observability
Copilot for JetBrains: persistent memory, Ollama BYOK, and enterprise managed settings for plugins and MCP
August 11, 2026 · via GitHub
OpenAI expands Daybreak: Blue for defensive Sol work, Red for GPT-5.6-Cyber on authorized research
August 10, 2026 · via OpenAI
Copilot weekly: /worktree and Sessions sidebar in CLI, /btw side chat and element feedback in VS Code 1.132
August 7, 2026 · via GitHub
Claude Code self-hosted environments enter public beta on Team and Enterprise
August 6, 2026 · via Anthropic
Codex CLI 0.146.1 tightens auto-review defaults for cyber-capable models
August 5, 2026 · via GitHub
Field note: Claude cyber-eval incidents are a harness lesson. Prove egress before you raise autonomy
July 31, 2026 · via Jigar Joshi
Anthropic discloses three real-world incidents in cybersecurity evaluations: open internet path, CTF prompts, PyPI package published
July 30, 2026 · via Anthropic
MCP 2026-07-28 is GA: stateless core, MRTR, header routing, Tasks extension, Tier 1 SDKs updated
July 28, 2026 · via Model Context Protocol
1 talk
Speaking on AI Observability
Enterprise solutions
How AI Observability ships in our engagements
The pages below are the buyer-focused, conversion-grade versions of this topic — deliverables, methodology, ROI, security considerations, and CTAs to scope a real engagement.
Solution
Agentic AI Consulting
Designed, built, and handed off — production agentic systems for enterprise teams.
Explore the Agentic AI Consulting solutionSolution
AI Guardrails
Multi-layer safety, policy, and audit controls for agents in regulated environments.
Explore the AI Guardrails solutionSolution
AI Systems Engineering Training
Eight-day corporate training programs that take dev teams from AI-assisted coding to production agentic systems.
Explore the AI Systems Engineering Training solutionSolution
Enterprise AI Architecture
Reference architectures for organisations standing up an AI platform — not one agent, but the foundation for many.
Explore the Enterprise AI Architecture solutionSolution
AI Observability
Tracing, eval, cache-hit telemetry, and cost attribution for production agents.
Explore the AI Observability solutionSolution
Multi-Agent Workflows
Supervisor + handoff orchestration for portfolios of agents that need to cooperate without arguing.
Explore the Multi-Agent Workflows solutionFrequently asked
AI Observability — the questions teams actually ask
For your team
Train your team on AI Observability
Two tracks — one for developers who build agents, one for business teams who use them. Customised to your stack, hands-on from session 1.
See AI Observability training tracksBuild with Jigar
Ship your first AI Observability system
Architecture design, production implementation on Claude API and MCP, full observability, and a real handoff. Working agents, not slides.
Explore AI Observability consultingRelated topic pillars
Adjacent topics to read next
Go deeper on this topic
New breakdowns on this and related agentic AI topics, plus what I am shipping for clients — one email on Thursdays.