Today's AI trends
Cache metadata can be a measurable long-context bottleneck. llama.cpp b10750 replaced repeated KV-cache scans and per-batch hash-map construction with an ordered sequence-position index. Its reported 71k-context test increased generation throughput from 69.3 to 72.7 tokens per second while leaving prompt-processing throughput unchanged and preserving greedy output in that test. The practical implication is to benchmark cache metadata work separately from model compute before assuming long-context performance is only a hardware or model problem. Release b10750
Agent evaluations are moving closer to production traces. Langfuse 4.27.0 adds an MCP evaluator testing tool, multimodal evaluation inputs, and cached-input-token and cost visibility in trace tables. It also surfaces output-token truncation rather than showing a blank turn. For teams operating tool-using systems, traces can become inputs to regression evaluation and cost diagnosis rather than logs consulted only after an incident. Langfuse v4.27.0
AgentOps is being framed as an operating discipline, not a prompt-writing technique. Databricks’ September 2 Big Book of AgentOps defines the work across building, evaluation, governance, and improvement of production agents, including architecture, observability, security, and cost management. Its practical recommendation is to make real traces and calibrated evaluation gates part of the delivery loop before expanding an agent’s scope. Announcing the Databricks Big Book of AgentOps