Math for Deep Learning Cover

Math for Deep Learning

What You Need to Know to Understand Neural Networks
by Ronald T. Kneusel
October 2021, 344 pp.

Download Chapter 11: GRADIENT DESCENT

Look Inside!

Math for Deep Learning back coverMath for Deep Learning interior spreadMath for Deep Learning interior spreadMath for Deep Learning interior spreadMath for Deep Learning interior spreadMath for Deep Learning interior spread

Deep learning is everywhere, making this powerful driver of AI something more STEM professionals need to know. Learning which library commands to use is one thing, but to truly understand the discipline, you need to grasp the mathematical concepts that make it tick. This book will give you a working knowledge of topics in probability, statistics, linear algebra, and differential calculus – the essential math needed to make deep learning comprehensible, which is key to practicing it successfully.

Each of the four subfields are contextualized with Python code and hands-on, real-world examples that bridge the gap between pure mathematics and its applications in deep learning. Chapters build upon one another, with foundational topics such as Bayes’ theorem followed by more advanced concepts, like training neural networks using vectors, matrices, and derivatives of functions. You’ll ultimately put all this math to use as you explore and implement deep learning algorithms, including backpropagation and gradient descent – the foundational algorithms that have enabled the AI revolution.

You’ll learn:

  • The rules of probability, probability distributions, and Bayesian probability
  • The use of statistics for understanding datasets and evaluating models
  • How to manipulate vectors and matrices, and use both to move data through a neural network
  • How to use linear algebra to implement principal component analysis and singular value decomposition
  • How to apply improved versions of gradient descent, like RMSprop, Adagrad and Adadelta

Once you understand the core math concepts presented throughout this book through the lens of AI programming, you’ll have foundational know-how to easily follow and work with deep learning.

Author Bio 

Ronald T. Kneusel earned a PhD in machine learning from the University of Colorado, Boulder, has two decades of machine learning experience in industry, and is presently pursuing deep-learning projects with L3Harris Technologies, Inc. Kneusel is also the author of Practical Deep Learning: A Python-Based Introduction (No Starch Press 2021), Numbers and Computers (2nd ed., Springer 2017), and Random Numbers and Computers (Springer 2018).

Table of contents 

Chapter 1: Setting the Stage
Chapter 2: Probability
Chapter 3: More Probability
Chapter 4: Statistics
Chapter 5: Linear Algebra
Chapter 6: More Linear Algebra
Chapter 7: Differential Calculus
Chapter 8: Matrix Calculus
Chapter 9: Data Flow in Neural Networks
Chapter 10: Backpropagation
Chapter 11: Gradient Descent
Appendix: Going Further

View the Copyright page
View the detailed Table of Contents
View the Index


"An excellent resource for anyone looking to gain a solid foundation in the mathematics underlying deep learning algorithms. The book is accessible, well-organized, and provides clear explanations and practical examples of key mathematical concepts. I highly recommend it to anyone interested in this field."
—Daniel Gutierrez, insideBIGDATA

"Ronald T. Kneusel has written a handy and compact guide to the mathematics of deep learning. It will be a well-worn reference for equations and algorithms for the student, scientist, and practitioner of neural networks and machine learning. Complete with equations, figures and even sample code in Python, this book is a wonderful mathematical introduction for the reader."
—David S. Mazel, Senior Engineer, Regulus-Group

"What makes Math for Deep Learning a stand-out, is that it focuses on providing a sufficient mathematical foundation for deep learning, rather than attempting to cover all of deep learning, and introduce the needed math along the way. Those eager to master deep learning are sure to benefit from this foundation-before-house approach."
—Ed Scott, Ph.D., Solutions Architect & IT Enthusiast

Extra Stuff 

Find the book's source code on GitHub


View the latest errata.