S - Compare Multiple Machine Learning Models

 

Compare Multiple Machine Learning Models

When comparing multiple machine learning models, you train, evaluate, and analyze the performance of various algorithms on the same dataset to identify which one is best suited for a given predictive task. This article is designed for you if you’re interested in learning how to train and compare different machine learning models. I will walk you through the steps involved in training and comparing several machine learning models to tackle a regression problem using Python.

Steps to Compare Multiple Machine Learning Models

The goal of comparing different models is to select the algorithm that strikes the best balance between accuracy, complexity, and overall performance for your specific problem. Here’s the step-by-step process:

1. Data Preparation: Clean the dataset by addressing missing values, removing duplicates, and correcting errors to ensure high-quality data for modeling.
2. Dataset Division: Split the dataset into training and testing sets, commonly using splits like 70-30% or 80-20%.
3. Diverse Model Selection: Pick a range of models for comparison, including simple linear models, tree-based approaches, ensemble techniques, and even more sophisticated algorithms based on the complexity of your problem and the characteristics of your data.
4. Model Fitting: Train each chosen model on the training set so that the model can learn from the input features and corresponding target values.
5. Evaluation: Assess the performance of each model on the test set using appropriate evaluation metrics.
6. Model Comparison: Compare the models based on the results of these metrics, also considering factors such as computational efficiency and model complexity.

I've discovered a perfect dataset for this problem, which you can download from here.

Training and Comparing Multiple Machine Learning Models

Let’s begin the process of training and comparing various machine learning models. Start by importing the required Python libraries and checking out this case study for further guidance.

Google Colab

!wget https://statso.io/wp-content/uploads/2023/12/Real_Estate.csv

import pandas as pd

data = pd.read_csv('Real_Estate.csv')

# display the first few rows

data_head = data.head()

print(data_head)

Transaction date House age Distance to the nearest MRT station \ 0 2012-09-02 16:42:30.519336 13.3 4082.0150 1 2012-09-04 22:52:29.919544 35.5 274.0144 2 2012-09-05 01:10:52.349449 1.1 1978.6710 3 2012-09-05 13:26:01.189083 22.2 1055.0670 4 2012-09-06 08:29:47.910523 8.5 967.4000 Number of convenience stores Latitude Longitude \ 0 8 25.007059 121.561694 1 2 25.012148 121.546990 2 10 25.003850 121.528336 3 5 24.962887 121.482178 4 6 25.011037 121.479946 House price of unit area 0 6.488673 1 24.970725 2 26.694267 3 38.091638 4 21.654710

print(data.info())

<class 'pandas.core.frame.DataFrame'> RangeIndex: 414 entries, 0 to 413 Data columns (total 7 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Transaction date 414 non-null object 1 House age 414 non-null float64 2 Distance to the nearest MRT station 414 non-null float64 3 Number of convenience stores 414 non-null int64 4 Latitude 414 non-null float64 5 Longitude 414 non-null float64 6 House price of unit area 414 non-null float64 dtypes: float64(5), int64(1), object(1) memory usage: 22.8+ KB None

The dataset contains 414 entries and 7 columns, with no missing values. Below is a summary of the columns:

• Transaction date: Indicates when the house sale occurred (stored as an object, so you may need to convert it or extract features such as year and month).
• House age: Represents the number of years since the house was built (float).
• Distance to the nearest MRT station: Measures the distance (in meters) to the closest mass rapid transit station (float).
• Number of convenience stores: Denotes how many convenience stores are within walking distance (integer).
• Latitude: Specifies the north-south geographic coordinate (float).
• Longitude: Specifies the east-west geographic coordinate (float).
• House price of unit area: Provides the price per unit area of the house (float), and serves as our target variable for prediction.

Since we are dealing with a regression problem, the next steps will involve data preprocessing and selecting appropriate regression models to determine the best-performing solution.

Data Preprocessing

To begin, we’ll perform several preprocessing tasks. Here’s the plan:

1. Date Conversion: Since the transaction date is stored as a string, we need to convert it into a datetime object. Once converted, we can extract useful features like the transaction year and month.
2. Feature Scaling: Next, we’ll scale the continuous features so that they are all on a similar scale. This is especially crucial for algorithms such as Support Vector Machines and K-nearest Neighbours, which are sensitive to the differences in feature scales.
3. Dataset Splitting: Finally, we’ll divide the dataset into a training set and a testing set, typically allocating 80% of the data for training and the remaining 20% for testing.

Now, let’s implement these preprocessing steps:

from sklearn.model_selection import train_test_split

from sklearn.preprocessing import StandardScaler

import datetime

# convert "Transaction date" to datetime and extract year and month

data['Transaction date'] = pd.to_datetime(data['Transaction date'])

data['Transaction year'] = data['Transaction date'].dt.year

data['Transaction month'] = data['Transaction date'].dt.month

# drop the original "Transaction date" as we've extracted relevant features

data = data.drop(columns=['Transaction date'])

# define features and target variable

X = data.drop('House price of unit area', axis=1)

y = data['House price of unit area']

# split the data into training and testing sets

X_train, X_test, y_train, y_test =

train_test_split(X, y, test_size=0.2, random_state=42)

# scale the features

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)

X_test_scaled = scaler.transform(X_test)

X_train_scaled.shape

(331, 7)

Model Training and Comparison 
Next, we’ll move on to training multiple models and comparing their performance. We’ll consider several popular regression models:
Linear Regression: Serves as an effective baseline for regression tasks.
Decision Tree Regressor: Demonstrates the performance of a basic tree-based approach.
Random Forest Regressor: Uses an ensemble technique to enhance the performance of decision trees.
Gradient Boosting Regressor: Offers another robust ensemble approach for regression problems.
We will train each model on our training data and evaluate them on the test set using Mean Absolute Error (MAE) and R-squared (R²) metrics. These metrics will provide insights into the average prediction error and the extent to which each model explains the variance in the target variable.
Now, let’s begin by training these models and comparing their performance:
from sklearn.linear_model import LinearRegression
from sklearn.tree import DecisionTreeRegressor
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
from sklearn.metrics import mean_absolute_error, r2_score

# initialize the models
models = {
    "Linear Regression": LinearRegression(),
    "Decision Tree": DecisionTreeRegressor(random_state=42),
    "Random Forest": RandomForestRegressor(random_state=42),
    "Gradient Boosting": GradientBoostingRegressor(random_state=42)
}

# dictionary to hold the evaluation metrics for each model
results = {}

# train and evaluate each model
for name, model in models.items():
    # training the model
    model.fit(X_train_scaled, y_train)

    # making predictions on the test set
    predictions = model.predict(X_test_scaled)

    # calculating evaluation metrics
    mae = mean_absolute_error(y_test, predictions)
    r2 = r2_score(y_test, predictions)

    # storing the metrics
    results[name] = {"MAE": mae, "R²": r2}

results_df = pd.DataFrame(results).T  
print(results_df)
   
                      MAE        R²
Linear Regression   9.748246  0.529615
Decision Tree      11.760342  0.204962
Random Forest       9.887601  0.509547
Gradient Boosting  10.000117  0.476071

Below is the performance of each model on the test set, evaluated using Mean Absolute Error (MAE) and R-squared (R²):




Linear Regression achieved the lowest Mean Absolute Error (MAE) of 9.75 and the highest R² score of 0.53, 
making it the top performer among the models we tested. 
This indicates that even a simple Linear Regression model can be very effective for this dataset.
In contrast, the Decision Tree Regressor recorded the highest MAE of 11.76 and the lowest R² of 0.20, 
suggesting that it might be overfitting the training data and underperforming on the test set. 
Meanwhile, both the Random Forest Regressor and the Gradient Boosting Regressor posted comparable results, 
with MAEs of 9.89 and 10.00 and R² scores of 0.51 and 0.48, respectively. 
Although these ensemble methods performed slightly below Linear Regression, 
they still fared much better than the Decision Tree model.

Summary
In summary, this example illustrates how to train and compare multiple machine learning models using Python. 
By evaluating several models, you can select the one that provides the best trade-off between accuracy, 
complexity, and overall performance for your specific task.

Google Colab Code link: 
https://colab.research.google.com/drive/1uQM2D_Cu3lSIJZ1d2PFPgzmviiK908vb?authuser=1#scrollTo=LQFDvfS2vFoo