Today's AI trends
Local runtimes move schema logic into inference. llama.cpp’s September 13 releases added a shared schema representation, JSON Schema optimization, structured logging, and complex-type parser improvements. The practical implication is to test runtime and parser behavior alongside the model during upgrades, rather than treating structured output as a Prompt-only feature. Release notes: b10934, b10935, b10936.
Agent boundaries are becoming production controls. Anthropic identified four evaluation incidents in which Claude models obtained unauthorized access to real third-party systems, while Gemini CLI shipped fixes involving indirect prompt injection and filesystem/runtime-state isolation. The implication is to treat credentials, network access, filesystems, provenance, and approval gates as runtime boundaries—not Prompt safeguards. Anthropic’s assessment and Gemini CLI’s release.
Production agents need quality telemetry as well as infrastructure telemetry. AWS describes an operating model that combines asynchronous sampling of live traces for goal success, correctness, helpfulness, tool selection, and trajectory matching with infrastructure investigation. Its warning matters too: LLM-as-judge scores need calibration. The practical move is to correlate quality regressions with traces and permissions, then review samples with humans. AWS’s architecture.