Offer expires midnight PT October 18. Only at manning.com.
New MEAP! Evaluating AI Systems
AI systems fail in ways you’ve never seen before. A model scores high on benchmarks and still misleads users. An agent completes a task by taking harmful actions and then covering its tracks. A RAG pipeline retrieves the right documents and still returns a fabricated response. Evaluating AI Systems teaches you how to know—and prove—whether your LLM-based applications are ready to ship. In this practical and important book, author Panos Alexopoulos provides a data and decision-driven approach that starts with the questions you need to answer about your system and
and works backwards to the evidence you need to gather.
Along the way, you’ll test the book’s systematic approach against some of the most challenging qualities of modern AI systems, such as evaluating robustness when systems encounter unexpected conditions, assessing whether their behavior is fair across relevant groups, and identifying security and privacy risks. Plus, explore the challenges of evaluating increasingly general and adaptable AI systems whose capabilities and behaviors may be difficult to anticipate in advance. [Read more]
3 chapters of this new MEAP are available now with more to follow soon!
Changing an LLM to alter an agent’s behavior is costly and slow. In Self-Improving Agents: How to engineer adaptive agent harnesses, Micheal Lanham, author of AI Agents in Action, shows you how to design AI agents that measurably get better in production without fine-tuning, weight updates, or waiting for the next model release.
You’ll do this by building HelixAgent—a RAG general-knowledge agent that gains drift detection, rollout controls, and a reward-hacking runbook—along with a portfolio including a data-analyst agent, a helpdesk agent with four-tier memory, a hill-climbing research agent, and self-
searching coding agents that all take advantage of HelixAgent’s growing functionality. By the time you’re done, you’ll be able to craft multiple agents all running under one single improvement loop, and be able to hold them up to the production standards of auditability, human gates, and honest costs! [Read more]
4 chapters of this new MEAP are available now with more to follow soon!
