Today's AI trends
AWS has published an open benchmarking approach that combines cost per correct answer, agent trajectory cost, and rubric-graded deliverables. Its practical implication is to evaluate models on representative workloads with a fixed acceptance rule, rather than selecting them from token-price tables. AWS’s September 11 post also highlights why multi-turn context growth belongs in the cost model.
Coding-agent workflows are increasingly framed around evidence, not only generated diffs. Cognition describes Devin returning simulator recordings, passed checks, and explicitly untested areas alongside fixes; it also describes screenshot-based bug inputs with a resulting screenshot of the fix. The practical implication is to require test results, recordings or equivalent artifacts, and stated coverage boundaries in the agent’s handoff to reviewers. Cognition’s case study is a vendor account, so teams should validate the pattern on their own workflows.