Machine Learning Part 2: Supervised Learning Explained

Supervised learning is a fundamental concept in machine learning where we have labelled data available. The machine “learns” from this data, much like a student learning from a teacher.

In this article, we will explore the different types of supervised learning, including classification and regression, and look at some common algorithms.

Types of Machine Learning

Supervised Learning (Our focus today)
Unsupervised Learning
Reinforcement Learning

What is Supervised Learning?

In supervised learning, algorithms are trained using labeled examples, such as an input where the desired output is known. The learning algorithm receives a set of inputs along with the corresponding correct outputs, and the algorithm learns by comparing its actual output with correct outputs to find errors. It then modifies the model accordingly.

There are two main types of supervised learning problems:

1. Classification 🔖

Classification involves predicting a discrete class label. The output variable is a category, such as “red” or “blue” or “disease” and “no disease”.

Common algorithms include: * Logistic Regression * Support Vector Machines * Naive Bayes * Decision Trees

2. Regression 📈

Regression involves predicting a continuous quantity. The output variable is a real value, such as “dollars” or “weight”.

Common algorithms include: * Linear Regression (single value) * Multivariate Linear Regression

Mixed Methods ⚗

Some methods can be adapted for both classification and regression: * Tree-based methods * Random Forest * Neural Networks * Support Vector Machines

Deep Dive: Classification

Let’s look closer at Logistic Regression. despite its name, it is a classification algorithm.

Continuous vs. Discrete Values 🌀

Continuous values: Infinite possibilities within a range (e.g., 1.1, 1.11, 1.112). Think of weight or height.
Discrete values: Distinct, separate values (e.g., 1, 2, 3 or A, B, C). Think of counting apples or categories.

Logistic regression is used to classify data into categories. Instead of predicting continuous values like linear regression, we predict the probability that a given input point belongs to a certain class.

Example: Fruit Classification

Imagine we have a dataset of fruit weights and we want to classify them as either Watermelon (0) or Apple (1).

Weight (kg)	Type (0=Watermelon, 1=Apple)
9.5	0
10.0	0
0.1	1
9.4	0
0.2	1

We build a model based on this data. The model might learn that light weights (around 0.1 - 0.2 kg) are likely Apples (1), and heavy weights (around 9 - 10 kg) are likely Watermelons (0).

If we then feed the model a new weight, say 10.4 kg, it effectively “passes” it through the model, calculates the probability, and predicts 0 (Watermelon).

Supervised Clustering (k-NN)

k-Nearest Neighbors (k-NN) is another simple supervised algorithm. It assumes that similar things exist in close proximity. In other words, similar things are near to each other.

Example: Flower Classification 🌸

Let’s say we have data on flower petals:

Length	Width	Type
2	0.5	A
4	1	B
1.5	0.3	A
6	1.7	B

If we plot these on a graph, we might see two distinct clusters: Type A at the bottom left (small length/width) and Type B at the top right (larger length/width).

Now, suppose we find a new flower with Length 5 and Width 1.5. Is it Type A or Type B?

Visually, (5, 1.5) is much closer to the Type B cluster. Therefore, k-NN would classify it as Type B.

The “nearness” is defined by distance (like Euclidean distance). We calculate the distance between the new point and its neighbors. The label is determined by the majority vote of its nearest neighbors.

Conclusion

Supervised learning is the most common form of machine learning today. By understanding the difference between classification (predicting categories) and regression (predicting values), you have the foundation to explore more complex algorithms like Neural Networks and Random Forests.

Written by

Abdur-Rahmaan Janhangeer

Chef

Python author of 9+ years having worked for Python companies around the world