Machine Learning vs. Deep Learning

Artificial Intelligence encompasses an extensive scope. You could even consider something like Dijkstra's shortest path algorithm as Artificial Intelligence. However, two categories of AI are frequently mixed up: Machine Learning and Deep Learning. Both of these refer to the statistical modelling of data to extract useful information or make predictions. In this article, we will list the reasons why these two analytical modelling techniques are not the same and help you further frame your understanding of these data modelling paradigms.

Overview Machine Learning is a method of statistical learning where each instance in a dataset is described by a set of features or attributes. In contrast, the term “Deep Learning” is a method of statistical learning that extracts features or attributes from raw data. Deep Learning does this by utilising neural networks with many hidden layers, big data, and powerful computational resources. The terms seem somewhat interchangeable; however, with Deep Learning methods, the algorithm constructs representations of the data automatically. In contrast, data representations are hard-coded as a set of features in machine learning algorithms, requiring further processes such as feature selection and extraction, (such as PCA).

Both of these terms are in dramatic contrast with another class of classical artificial intelligence algorithms known as Rule-Based Systems where each decision is manually programmed in such a way that it resembles a statistical model.

In Machine Learning and Deep Learning, many different models fall into two distinct categories, supervised and unsupervised. In unsupervised learning, algorithms such as k-Means, hierarchical clustering, and Gaussian mixture models attempt to learn meaningful structures in the data. Supervised learning involves an output label associated with each instance in the dataset. This output can be discrete/categorical or real-valued. Regression models estimate real-valued outputs, whereas classification models estimate discrete-valued outputs. Simple binary classification models have just two output labels, 1 (positive) and 0 (negative). Some popular supervised learning algorithms that are considered Machine Learning: are linear regression, logistic regression, decision trees, support vector machines, and neural networks, as well as non-parametric models such as k-Nearest Neighbors.

Data Size Both Machine Learning and Deep Learning can handle massive dataset sizes; however, machine learning methods make much more sense with small datasets. For example, if you only have 100 data points, decision trees, k-nearest neighbours, and other machine learning models will be much more valuable to you than fitting a deep neural network on the data. This is due to the next topic of difference, Interpretability.

Interpretability

A lot of the criticism of deep learning methods and machine learning algorithms such as Support Vector Machine or (maybe, because you can at least visualise the constituent probabilities making up the output), Naive Bayes, are due to their difficulty to interpret. For example, when a Convolutional Neural Network outputs ‘cat’ in a dog vs cat problem, nobody seems to know why it did that. In contrast, when you are modelling data such as an electronic health record or bank loan dataset with a machine learning technique, it is much easier to understand the reasoning for the model’s prediction.

One of the best examples of interpretability is decision trees where you follow logical tests down nodes of the tree until you reach a decision. Another machine learning algorithm with high interpretability is k-Nearest Neighbors. This is not a parametric learning algorithm but still falls under the category of machine learning algorithms. It is very interpretability because you easily reason about similar instances for yourself.

Conclusion In Conclusion, the image above is the best summary of the difference between deep learning and machine learning. A concrete anecdote would be to consider raw data forms such as pixels in images or sin waves in audio. It is challenging to construct semantic features from this data for machine learning methods. Therefore, deep learning methods dominate in these models. Deep learning also comes with many more nuances and unexplained phenomenon than traditional machine learning methods. Please let me know if this article helped frame your understanding of machine learning compared deep learning. Thank you for reading!