Gradient Descent for Machine Learning
A tutorial diving into the gradient descent algorithm for machine learning (ML) with Python This tutorial’s code is available on Github and its full implementation as well on Google Colab . Table of Contents What is Gradient Descent? Cost function Gradients Python Implementation Learning Rate Convergence Convex Function Batch Gradient Descent Stochastic Gradient Descent Mini-Batch Gradient Descent Conclusion Resources References 📚 Check out our convolutional neural networks tutorial. 📚 What is Gradient Descent? Figure 1: A gradient descent illustration | Source: Creative Commons by Wikimedia [ 3 ] Gradient descent is one of the most common machine learning algorithms used in neural networks [ 7 ], data science, optimization, and machine learning tasks. The gradient descent algorithm and its variants can be found in almost every machine learning model. Gradient descent is a popular optimization method of tuning the parameters in a machine learning model. Its goal is to apply o