Understanding the Types of Machine Learning Models

Choose the wrong machine learning model for your data and you’ll spend months tuning a model that, no matter how you tune it, is not going to perform well. 

Different types will affect nearly all of the downstream: the amount of labelled data you will require, what algorithms will even be considered for your model, and the kind of understanding someone will have if they don’t have to do the modelling. 

It covers the four basic categories with the more recent fifth, namely self-supervised learning; its hybrid combinations for production problems; and matching a type to a problem you’re really solving.

Key Takeaways

• There are four main categories for most machine learning tasks supervised learning, unsupervised learning, semi-supervised learning and reinforcement learning, all with varying data scenarios and objectives.
• The fifth approach is self-supervised learning, and, aside from semi-supervised learning, it’s the approach most commonly used in training large language models.
• Now some of the most critical production use cases—such as fraud detection, credit scoring, and autonomous vehicles—are being addressed with hybrid and ensemble models, which use several different methods.
• According to Statista, the world machine learning market will expand at a 36.08% CAGR from 2024 to 2030 and reach a whopping $503.40 billion by 2030.
• The first question is whether you have labelled data of one kind or another, and what the amount of data you have is?

So what Determines What Type of Model you require?

The first question before comparing algorithms is: Do you have labelled data? And how much?

For labelled data, each example comes with the answer: For an email, a label of “spam” or “not spam”; for a transaction, a label of “fraudulent” or “legitimate”. 

There are many data points that are marked with their labels which seem to indicate supervised learning. 

• There is no indication whatsoever that unsupervised learning is indicated. 
• The crux of the matter is that there is a little labelled data and a lot of unlabeled data, which is exactly where semi-supervised learning is meant to fit in. 
• If there’s no static dataset and nothing but an environment it can interact with, get feedback from, it’s in the realm of reinforcement learning.

Supervised Learning: Forecasting a known result

In supervised learning models, the models are trained using labelled data, and the models map the inputs to the known output until the model can predict the correct output for new data. A typical instance is an email spam filter, where the training set consists of emails labeled as spam or not spam, and the model learns which characteristics of the emails, including patterns in the sender, specific wordings, formatting, etc., make them spam or not.

Typical algorithms include decision trees, k-nearest neighbours (KNN), support vector machines (SVM) and neural networks. There are two types of problems that are generally distinguished when dealing with supervised learning:

Sorting inputs by classes, such as the spam filter for a binary classification, or LinkedIn’s job-category sorting by user activity for a multiclass classification.

Regression: Predicting a continuous value like predicting stock prices from historical data or estimating a customer’s lifetime value from their purchase history.

Unsupervised Learning: Finding Structure Nobody Labelled

Unsupervised machine learning model is used when there are no output labels associated with the data, and the resulting structure, grouping, and patterns are not known in advance. If one was given the same set of e-mails, without the spam/not-spam labels, an unsupervised model would still classify the e-mails into groups, but without the knowledge beforehand that the groups represent spam/not-spam.

Typical algorithms: Principal component analysis (PCA), k-means clustering, and hierarchical clustering. This is the class of segmentation which drives most customer segmentation initiative and the more often the model’s groupings show a pattern that no one had specifically looked for.

Semi-Supervised Learning: A Little Labelled Data goes a Long Way

Semi-supervised learning is training a model on a small set of labeled data that is augmented by a large set of unlabeled data, particularly beneficial when the data labelling process is the expensive, time-intensive step of the project, not the data collection. If there are thousands of web pages and you have to tag each one by hand, for example, this isn’t feasible.

The standard method: learn a first model from the small labelled set with an algorithm like Support Vector Machines and apply a clustering technique (e.g., k-means) to find patterns in the unlabeled set data to increase the coverage of the model. Common methods are self-training (a model is provided with a set of unlabeled data, where the model labels its own data with which it then trains again), and co-training (two models are presented with two different views of the same data, which then check each other’s labels).

Reinforcement learning: Learning from Consequences (not examples)

Reinforcement machine learning model has no labeled data whatsoever; it is done by an agent acting upon an environment and learning from the rewards or penalties that the environment returns as a result of the agent’s actions, and then improving the actions it takes as it conducts more trials and errors. It is the usual way to play games and much of the current robotics, where there is no such thing as a fixed set of “correct answers.”

Modern Language Models are Primarily Based on Self-Supervised Learning

Although the terms are confusing, self-supervised learning has evolved into a new class of its own which is probably not categorized under semi-supervised learning. Produces its own labels from unlabeled data structures. For instance, it hides some of the words in a sentence, and learns a model to guess the missing word instead of learning from any labeled examples whatsoever. Most modern LLMs are pretrained by using the technique, where the “labels” are extracted from the text itself, rather than an example being labeled by a person. In contrast, semi-supervised learning requires a real human labelled subset to be present to guide the process.

Models that Combine Methods for More Challenging Problems: Hybrid and Ensemble Models

Not every problem has a neat fit in one of these categories; many times it is better to use a mixture of models rather than make everything fit into one. In particular, ensemble learning combines the prediction of several models to boost accuracy, such as bagging (Random Forest), boosting (Gradient Boosting Machines) or stacking. The neural networks with hidden markov models (NN-HMM) is a type of neural networks that is used with probabilistic sequence modelling, typically in speech to text systems.

Hybrid models appear on the production:

When used in the field of finance, credit scoring is done with decision trees to select features and make initial classification, and a neural network to learn more complex patterns and refine the credit score.

• Fraud detection (finance): An unsupervised anomaly detection identifies the odd transactions and a supervised classification model checks if the anomaly is a fraud.
• Customer segmentation (marketing): Segments are identified with the clustering, and a supervised predictive analytics predicts the response of each segment to various campaigns.
• Recommendation systems (marketing): Collaborative filtering (based on other users that liked something similar) is used in combination with content-based filtering (based on the features of the item) to make more accurate recommendations.
• Autonomous vehicles (transportation): With autonomous vehicles, computer vision is used for object recognition and reinforcement learning for decision making using a real-time cycle of rewards and penalties.
• Predictive maintenance (manufacturing): anomaly detection alerts on unusual equipment activity and predictive modelling provides the remaining useful life, to enable maintenance planning before equipment failure.

Correlating a Model Type With the Real Problem

When you have a well-defined goal and have a lot of labeled historical data, begin supervised learning. This is the most straightforward way to get from data to a functioning model. 

Unsupervised machine learning model can be useful when you don’t know what structure you’re trying to find in your data. 

If the limit for the problem isn’t the amount of data, but the process of labelling, then semi-supervised learning is more effective at utilizing the labelled data. 

If it is indeed a series of decisions with consequences instead of just a single prediction, then reinforcement learning may be a better solution (but it also is the most resource-heavy to implement properly).

After selecting a type, building and deployment of a type is a project in and of itself. ClicData’s machine learning model project lifecycle takes you through each step. But if the type-vs-technique gap is still murky, the ClicData categorization of AI, machine learning, deep learning and data science elucidates where each of these terms fits in, and where they don’t fit.

FAQs