🤖 Machine Learning Types & Categories

Explore the world of Machine Learning through interactive examples

What is Machine Learning?

Machine Learning is a branch of Artificial Intelligence that enables computers to learn from data without being explicitly programmed.

📊

Supervised Learning

Learning from labeled data with known outcomes

🔍

Unsupervised Learning

Finding patterns in unlabeled data

🎮

Reinforcement Learning

Learning through trial and error with rewards

⚡

Other Types

Semi-supervised, self-supervised, transfer learning & more

ML Learning Process

1 Collect Data
→
2 Train Model
→
3 Make Predictions

📊 Supervised Learning

The model learns from labeled training data to make predictions on new, unseen data.

Classification

Predicting discrete categories or classes

Example: Email Spam Detection

Classification Algorithms:

1. Logistic Regression
Complexity: Low

Uses the logistic function to model the probability of a binary outcome. Despite its name, it's used for classification.

Best for: Binary classification, linearly separable data
Pros: Fast, interpretable, works well with small datasets
Cons: Assumes linear relationship, can't handle complex patterns
2. Decision Trees
Complexity: Medium

Creates a tree-like model of decisions based on feature values. Splits data recursively to create pure groups.

Best for: Both categorical and numerical data, non-linear patterns
Pros: Easy to understand and visualize, handles non-linear data
Cons: Prone to overfitting, unstable with small data changes
3. Random Forest
Complexity: Medium-High

An ensemble of decision trees trained on random subsets of data. Combines predictions through voting.

Best for: High-dimensional data, reducing overfitting
Pros: High accuracy, handles missing values, reduces overfitting
Cons: Less interpretable, computationally expensive
4. Support Vector Machines (SVM)
Complexity: Medium-High

Finds the optimal hyperplane that maximizes the margin between classes. Can use kernel tricks for non-linear separation.

Best for: High-dimensional spaces, clear margin of separation
Pros: Effective in high dimensions, memory efficient
Cons: Slow on large datasets, sensitive to feature scaling
5. K-Nearest Neighbors (KNN)
Complexity: Low

Classifies data points based on the majority class of their K nearest neighbors in feature space.

Best for: Small datasets, simple patterns
Pros: Simple, no training required, works well locally
Cons: Slow prediction, sensitive to irrelevant features
6. Naive Bayes
Complexity: Low

Applies Bayes' theorem with strong independence assumptions between features. Probabilistic classifier.

Best for: Text classification, spam detection
Pros: Fast, works well with small datasets, handles high dimensions
Cons: Assumes feature independence, can be outperformed
7. Neural Networks
Complexity: High

Layers of interconnected nodes (neurons) that learn complex patterns through backpropagation.

Best for: Complex patterns, large datasets, image/text data
Pros: Handles complex non-linear relationships, highly flexible
Cons: Requires lots of data, computationally expensive, black box
8. Gradient Boosting (XGBoost, LightGBM)
Complexity: High

Builds trees sequentially, each correcting errors of previous trees. Highly powerful ensemble method.

Best for: Structured/tabular data, competitions
Pros: State-of-the-art performance, handles missing data
Cons: Can overfit, requires careful tuning

Regression

Predicting continuous numerical values

Example: House Price Prediction

Predicted Price: $300,000

Regression Algorithms:

1. Linear Regression
Complexity: Low

Models the relationship between variables using a linear equation. Minimizes the sum of squared errors.

Best for: Linear relationships, simple predictions
Pros: Fast, interpretable, works well with small datasets
Cons: Only models linear relationships, sensitive to outliers
2. Polynomial Regression
Complexity: Medium

Extends linear regression by adding polynomial features. Can model curved relationships.

Best for: Non-linear but smooth relationships
Pros: Captures curved patterns, still relatively interpretable
Cons: Can overfit easily, sensitive to outliers
3. Ridge Regression (L2 Regularization)
Complexity: Low-Medium

Linear regression with L2 penalty on coefficients. Prevents overfitting by shrinking coefficients.

Best for: High-dimensional data, multicollinearity
Pros: Reduces overfitting, handles correlated features
Cons: Doesn't perform feature selection, requires tuning
4. Lasso Regression (L1 Regularization)
Complexity: Low-Medium

Linear regression with L1 penalty. Can set coefficients to zero, performing automatic feature selection.

Best for: Feature selection, sparse models
Pros: Built-in feature selection, interpretable
Cons: Selects only one feature from correlated group
5. ElasticNet Regression
Complexity: Medium

Combines L1 and L2 regularization. Balances feature selection with coefficient shrinking.

Best for: When you want both feature selection and regularization
Pros: Handles correlated features better than Lasso
Cons: Two hyperparameters to tune
6. Decision Tree Regression
Complexity: Medium

Splits data into regions and predicts the mean value in each region. Non-parametric approach.

Best for: Non-linear patterns, mixed data types
Pros: Handles non-linearity, no feature scaling needed
Cons: Prone to overfitting, unstable predictions
7. Random Forest Regression
Complexity: Medium-High

Ensemble of decision tree regressors. Averages predictions from multiple trees.

Best for: Complex non-linear relationships
Pros: High accuracy, handles outliers well
Cons: Less interpretable, computationally expensive
8. Support Vector Regression (SVR)
Complexity: Medium-High

Finds a function that deviates from actual values by no more than a threshold. Uses kernel tricks.

Best for: Non-linear relationships, robust to outliers
Pros: Effective in high dimensions, flexible with kernels
Cons: Slow on large datasets, requires careful tuning

🔍 Unsupervised Learning

The model finds hidden patterns and structures in unlabeled data without predefined categories.

Clustering

Grouping similar data points together

Example: Customer Segmentation

Budget Buyers
Premium Customers
Frequent Shoppers

Clustering Algorithms:

1. K-Means Clustering
Complexity: Low-Medium

Partitions data into K clusters by minimizing within-cluster variance. Iteratively updates centroids.

Best for: Spherical clusters, known number of clusters
Pros: Fast, simple, scalable to large datasets
Cons: Requires K specification, sensitive to outliers
2. Hierarchical Clustering
Complexity: Medium-High

Builds a tree of clusters (dendrogram) using either agglomerative (bottom-up) or divisive (top-down) approach.

Best for: When you want a hierarchy, don't know K beforehand
Pros: No need to specify K, produces dendrogram
Cons: Computationally expensive O(n³), not scalable
3. DBSCAN (Density-Based)
Complexity: Medium

Groups points that are closely packed together. Can find arbitrarily shaped clusters and identify outliers.

Best for: Non-spherical clusters, noisy data with outliers
Pros: Finds any shape, detects outliers, no K needed
Cons: Sensitive to parameters, struggles with varying densities
4. Gaussian Mixture Models (GMM)
Complexity: Medium-High

Assumes data comes from a mixture of Gaussian distributions. Uses EM algorithm for soft clustering.

Best for: Overlapping clusters, probabilistic assignments
Pros: Soft clustering, flexible cluster shapes
Cons: Can converge to local optima, computationally intensive
5. Mean Shift
Complexity: Medium-High

Finds clusters by shifting points toward modes of density. Non-parametric density estimation.

Best for: Unknown number of clusters, any cluster shape
Pros: No K needed, finds any shape, one parameter
Cons: Computationally expensive, sensitive to bandwidth
6. Spectral Clustering
Complexity: High

Uses eigenvalues of similarity matrix to perform dimensionality reduction before clustering in fewer dimensions.

Best for: Complex cluster shapes, graph-based data
Pros: Handles complex shapes, works with graphs
Cons: Computationally expensive, requires K

Dimensionality Reduction

Reducing the number of features while preserving important information

Example: Data Visualization

High Dimensional Data
Feature 1 Feature 2 Feature 3 Feature 4 Feature 5 Feature 6
↓
Reduced Dimensions
Component 1 Component 2

Dimensionality Reduction Algorithms:

1. Principal Component Analysis (PCA)
Complexity: Medium

Linear transformation that projects data onto principal components (directions of maximum variance).

Best for: Linear relationships, visualization, noise reduction
Pros: Fast, interpretable, removes multicollinearity
Cons: Only linear, assumes variance equals importance
2. t-SNE (t-Distributed Stochastic Neighbor Embedding)
Complexity: High

Non-linear technique that preserves local structure. Converts similarities to probabilities and minimizes divergence.

Best for: Visualization (2D/3D), preserving local structure
Pros: Excellent visualization, reveals clusters
Cons: Computationally expensive, non-deterministic, only for visualization
3. UMAP (Uniform Manifold Approximation)
Complexity: Medium-High

Non-linear technique based on manifold learning. Faster than t-SNE while preserving both local and global structure.

Best for: Large datasets, preserving global + local structure
Pros: Faster than t-SNE, preserves more structure, scalable
Cons: Less interpretable than PCA, hyperparameter sensitive
4. Autoencoders
Complexity: High

Neural networks that learn compressed representations through encoding and decoding. Non-linear dimensionality reduction.

Best for: Complex non-linear patterns, image compression
Pros: Handles complex patterns, flexible architecture
Cons: Requires lots of data, computationally expensive
5. Linear Discriminant Analysis (LDA)
Complexity: Medium

Supervised technique that finds linear combinations maximizing class separation. Used for classification and reduction.

Best for: Classification tasks, when labels available
Pros: Maximizes class separation, supervised
Cons: Assumes Gaussian distribution, limited to C-1 dimensions
6. Independent Component Analysis (ICA)
Complexity: Medium

Separates multivariate signal into independent non-Gaussian components. Used for blind source separation.

Best for: Signal processing, separating mixed sources
Pros: Finds independent components, works on mixed signals
Cons: Assumes independence, sensitive to initialization

Association Rule Learning

Discovering interesting relationships between variables

Example: Market Basket Analysis

🍞 Bread
→
🧈 Butter

If customer buys bread, they often buy butter

Association Rule Algorithms:

1. Apriori Algorithm
Complexity: Medium

Uses breadth-first search and candidate generation to find frequent itemsets, then generates association rules.

Best for: Market basket analysis, small to medium datasets
Pros: Simple, easy to understand, guarantees complete results
Cons: Slow on large datasets, generates many candidates
2. FP-Growth (Frequent Pattern)
Complexity: Medium-High

Builds a compressed FP-tree structure without candidate generation. Much faster than Apriori.

Best for: Large datasets, faster alternative to Apriori
Pros: Faster than Apriori, no candidate generation, memory efficient
Cons: More complex implementation, FP-tree can be large
3. ECLAT (Equivalence Class Transformation)
Complexity: Medium

Uses depth-first search and vertical data format. Intersects transaction IDs to find frequent itemsets.

Best for: Dense datasets, when you need just frequent itemsets
Pros: Faster than Apriori, simple intersection operations
Cons: High memory usage, less popular than FP-Growth

🎮 Reinforcement Learning

An agent learns to make decisions by performing actions and receiving rewards or penalties.

Interactive Example: Robot Navigation

🤖
❌
❌
⭐

Rewards: 0

Steps: 0

Key Concepts:

Agent: The learner (robot 🤖)
Environment: The world the agent interacts with
Actions: Moves the agent can make
Rewards: Feedback for actions (+10 for goal, -1 for obstacle)

Reinforcement Learning Algorithms:

1. Q-Learning
Complexity: Medium

Model-free algorithm that learns action-value function (Q-function). Updates Q-values using Bellman equation.

Best for: Discrete action spaces, tabular problems
Pros: Simple, off-policy learning, guaranteed convergence
Cons: Doesn't scale to large state spaces, requires extensive exploration
2. SARSA (State-Action-Reward-State-Action)
Complexity: Medium

On-policy temporal difference learning. Updates Q-values based on the action actually taken by current policy.

Best for: When you need safer exploration, on-policy learning
Pros: More conservative than Q-learning, learns safer policies
Cons: Slower convergence, still requires tabular representation
3. Deep Q-Networks (DQN)
Complexity: High

Uses deep neural networks to approximate Q-function. Introduced experience replay and target networks.

Best for: High-dimensional state spaces (images), Atari games
Pros: Scales to complex environments, handles visual input
Cons: Sample inefficient, unstable training, only discrete actions
4. Policy Gradient Methods (REINFORCE)
Complexity: Medium-High

Directly optimizes the policy by following the gradient of expected rewards. No Q-function needed.

Best for: Continuous action spaces, stochastic policies
Pros: Works with continuous actions, can learn stochastic policies
Cons: High variance, sample inefficient, slow convergence
5. Actor-Critic Methods
Complexity: High

Combines policy gradient (actor) with value function (critic). Reduces variance while maintaining benefits of policy gradients.

Best for: Continuous control, reducing variance
Pros: Lower variance than pure policy gradient, handles continuous actions
Cons: More complex, requires tuning two networks
6. Proximal Policy Optimization (PPO)
Complexity: High

Clips policy updates to prevent large changes. More stable and easier to tune than other policy gradient methods.

Best for: Robotics, continuous control, general-purpose RL
Pros: Stable, sample efficient, easy to tune, state-of-the-art
Cons: Still computationally expensive, requires hyperparameter tuning
7. Deep Deterministic Policy Gradient (DDPG)
Complexity: High

Actor-critic method for continuous action spaces. Combines DQN with policy gradients using deterministic policies.

Best for: Continuous control tasks, robotics
Pros: Handles continuous actions well, off-policy learning
Cons: Sensitive to hyperparameters, can be unstable
8. Monte Carlo Tree Search (MCTS)
Complexity: Medium-High

Builds a search tree through simulation. Used in AlphaGo. Balances exploration and exploitation.

Best for: Games (Go, Chess), planning with simulators
Pros: No domain knowledge needed, anytime algorithm, proven in games
Cons: Requires simulator, computationally intensive
9. Soft Actor-Critic (SAC)
Complexity: High

Maximum entropy RL framework that encourages exploration. Combines off-policy learning with stochastic policies.

Best for: Continuous control, sample efficiency
Pros: Very sample efficient, stable, automatic exploration
Cons: Complex implementation, computationally demanding

Real-World Applications:

  • Game AI (AlphaGo, Chess engines)
  • Robotics and autonomous vehicles
  • Resource management
  • Recommendation systems

⚡ Other Machine Learning Types

Semi-Supervised Learning

Uses a small amount of labeled data combined with a large amount of unlabeled data

Labeled Data (small)
✓
✓
+
Unlabeled Data (large)
?
?
?
?

Use Case: Medical image analysis where labeled data is expensive to obtain

Self-Supervised Learning

The model creates its own labels from the input data structure

The cat sat on the ___
↓ Predict
mat

Use Case: Language models (BERT, GPT), image pretraining

Transfer Learning

Reusing a pre-trained model on a new, related problem

Pre-trained Model

Trained on millions of images

→
Fine-tuned Model

Adapted for your specific task

Use Case: Using ImageNet-trained models for medical imaging

Online Learning

The model learns incrementally as new data arrives

Data 1 →
Data 2 →
Data 3 →
📊 Model Updates Continuously

Use Case: Stock price prediction, fraud detection

Ensemble Learning

Combining multiple models to improve predictions

Model 1: 🐱
Model 2: 🐱
Model 3: 🐕
↓ Majority Vote
Final: 🐱 Cat

Ensemble Methods:

1. Bagging (Bootstrap Aggregating)
Complexity: Medium

Trains multiple models on different random subsets of data (with replacement). Averages predictions to reduce variance.

Best for: Reducing overfitting, high-variance models
Examples: Random Forest, Bagged Decision Trees
Pros: Reduces variance, simple parallelization
Cons: Doesn't reduce bias, less interpretable
2. Boosting
Complexity: High

Sequentially trains models where each new model focuses on correcting errors of previous models. Combines weak learners.

Best for: Reducing bias, achieving high accuracy
Examples: XGBoost, AdaBoost, Gradient Boosting, LightGBM, CatBoost
Pros: State-of-the-art performance, reduces bias and variance
Cons: Prone to overfitting, sequential (slower), sensitive to outliers
3. Stacking (Stacked Generalization)
Complexity: High

Trains a meta-model to combine predictions from multiple base models. Uses cross-validation to prevent overfitting.

Best for: Kaggle competitions, when you have diverse models
Pros: Can achieve best performance, leverages diverse models
Cons: Complex, computationally expensive, hard to interpret
4. Voting Ensembles
Complexity: Low-Medium

Combines predictions from multiple models through majority voting (classification) or averaging (regression).

Best for: Simple ensemble, combining independent models
Pros: Simple to implement, improves robustness
Cons: Limited performance gain, treats all models equally

⚖️ Algorithm Comparison Guide

Choose the right algorithm for your problem using this interactive comparison tool.

Algorithm Selector

Problem Type

Dataset Size

Interpretability Need

Quick Reference Tables

Supervised Learning Comparison

Algorithm Speed Accuracy Interpretability Best Use Case
Logistic Regression ⚡⚡⚡ ⭐⭐ 🔍🔍🔍 Binary classification, baseline
Decision Trees ⚡⚡ ⭐⭐ 🔍🔍🔍 Interpretability needed
Random Forest ⚡⚡ ⭐⭐⭐ 🔍 General purpose, high accuracy
XGBoost ⚡⚡ ⭐⭐⭐⭐ 🔍 Competitions, tabular data
SVM ⚡ ⭐⭐⭐ 🔍🔍 High-dimensional data
Neural Networks ⚡ ⭐⭐⭐⭐ 🔍 Images, text, complex patterns

Unsupervised Learning Comparison

Algorithm Speed Scalability Cluster Shape Best Use Case
K-Means ⚡⚡⚡ ⭐⭐⭐ Spherical Quick clustering, large data
DBSCAN ⚡⚡ ⭐⭐ Any shape Outlier detection, any shape
Hierarchical ⚡ ⭐ Any shape Small data, need hierarchy
PCA ⚡⚡⚡ ⭐⭐⭐ N/A Visualization, noise reduction
t-SNE ⚡ ⭐ N/A 2D/3D visualization