“From the available books on deep learning, this textbook is outstanding. Drori has provided an extensive overview of the field including reinforcement learning – in its technical meaning and in his successful, common-sense approach to teaching and understanding.” Gilbert Strang, Massachusetts Institute of Technology

This book covers an impressive breadth of foundational concepts and algorithms behind modern deep learning. By reading this book, readers will quickly but thoroughly learn and appreciate foundations and advances of modern deep learning. Kyunghyun Cho, New York University

This book offers a fascinating tour of the field of deep learning, which in only ten years has come to revolutionize almost every area of computing. Drori provides concise descriptions of many of the most important developments, combining unified mathematical notation and ample figures to form an essential resource for students and practitioners alike. Jonathan Ventura, Cal Poly

Drori’s textbook goes under the hood of deep learning, covering a broad swath of modern techniques in optimization that are useful for efficiently training neural networks. The book also covers regularization methods to avoid overfitting, a common issue when working with deep learning models. Overall, this is an excellent textbook for students and practitioners who want to gain a deeper understanding of deep learning. Madeleine Udell, Stanford University

"This new book by Prof. Drori brings fresh insights from his experience teaching thousands of students at Columbia, MIT, and NYU during the past several years. The book is a unique resource and opportunity for educators and researchers worldwide to build on his highly successful deep learning course.", Claudio Silva, New York University

This textbook provides an excellent introduction to contemporary methods and models in deep learning. I expect this book to become a key resource in data science education for students and researchers. Nakul Verma, Columbia University

The Science of Deep Learning emerged from courses taught by the author that have provided thousands of students with training and experience for their academic studies, and prepared them for jobs in deep learning, machine learning, and artificial intelligence in leading companies in industry and academia. The book begins by covering the foundations of deep learning, followed by key deep learning architectures. Subsequent parts on generative models and reinforcement learning may be used either as part of a deep learning course or as part of a course on each topic. The book includes state-of-the-art topics such as Transformers, graph neural networks, variational autoencoders, and deep reinforcement learning, with a broad range of applications. Appendices provide equations for computing gradients in backpropagation and optimization, and best practices in scientific writing and reviewing. The text presents an up-to-date guide to the field built upon clear visualizations using a unified notation and equations, lowering the barrier to entry for the reader. The accompanying web site provides complementary code and hundreds of exercises with solutions.

Contents

Part I: Foundations

  1. Introduction

  2. Forward an Backpropagation

  3. Optimization

  4. Regularization

Part II: Architectures

  1. Convolutional Neural Networks

  2. Sequence Models

  3. Graph Neural Networks

  4. Transformers

Part III: Generative Models

  1. Generative Adversarial Networks

  2. Variational Autoencoders

Part IV: Reinforcement Learning

  1. Reinforcement Learning

  2. Deep Reinforcement Learning

Part V: Applications

  1. Applications

Appendices

A. Matrix Calculus

B. Scientific Writing and Reviewing Best Practices

References

Index

To cite this book please use the bibtex entry:

@book{drori2022dlbook,

title = {The Science of Deep Learning},

author = {Drori, Iddo},

publisher = {Cambridge University Press},

note = {\url{http://www.dlbook.org}},

year = {2022}

}

@ Iddo Drori, 2022