AI models are powerful, but they do not always behave as expected. They can give unhelpful or incorrect answers. To improve them, we need to guide them toward responses that are useful and safe. This book shows how to do this using Reinforcement Learning from Human Feedback (RLHF). It explains the main method used to train today’s advanced AI models.  Learn the complete process for training AI with feedback from people.  Understand how to collect human opinions and use them to guide an AI.  Build a model that teaches the AI what a good answer looks like.  Discover new, simpler ways to train AI, like Direct Preference Optimisation (DPO).  Find out how to test your AI to make sure it is becoming more helpful and safe.  The RLHF Book is the first complete guide to training AI with human feedback. Written by a leading expert who helped create these methods, this book gives you a clear plan to follow. It covers everything from getting data to training and testing your AI.  After reading this book, you will have the skills to build AI models that are more helpful, safe and act as expected. This book is for engineers, AI scientists and students who want to learn how to train modern AI.