Deep Dive into Classifiers

This Notebook provides a structured breakdown of the classifiers.

For both 3 and 7 classes, we use the same approach, only that the data imported is different.

This overview includes:

• Importing dependencies

• Exploring the dataset

• Data preprocessing steps

• Training and evaluating models

---

1. Importing Dependencies

This section imports all required libraries for data manipulation, machine learning, and evaluation.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from imblearn.over_sampling import SMOTE

from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from sklearn.neural_network import MLPClassifier

from sklearn.metrics import accuracy_score, f1_score, confusion_matrix, ConfusionMatrixDisplay

from IPython.display import display

---

2. Dataset Exploration

Loading Dataset

For 3-Class

# Load dataset
df = pd.read_csv("class_3_dataset.csv")

Preliminary Information

Class Label Meaning:

0 = waiting/idle

1 = sharpening

2 = slicing

3 = cutting

4 = pulling

5 = reaching

6 = feeling

# Display dataset info
df.info()
print(df.shape)

# Define feature and label columns
X, y = df.iloc[:, :-1], df.iloc[:, -1]

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 11629 entries, 0 to 11628
Columns: 157 entries, acc_mean_x_right to class
dtypes: float64(128), int64(29)
memory usage: 13.9 MB
(11629, 157)

Class Distribution

It can be observed that certain classes are quite under-represented, showing how this is an imbalanced dataset.

With 3 classes, class 0 and 1 are under-represented, and with 7 classes, class 4 and 6 are under-represented.

# Function to plot class distribution
def plot_class_distribution(y, title):
    class_counts = y.value_counts()
    plt.figure(figsize=(6, 6))
    plt.pie(class_counts, labels=class_counts.index, autopct='%1.1f%%', startangle=140, colors=plt.cm.Pastel1.colors)
    plt.title(title)
    plt.show()

# Plot class distribution before and after SMOTE
plot_class_distribution(y, "Class Distribution Before Preprocessing")

Correlation Heatmap

It can be seen that there are various features that are highly correlated with each other, making them redundant. Moreover, features can be streamlined by removing those which are not highly correlated with the class.

Note: not all feature labels are shown on the axes due to the high number of features present.

# Compute correlation matrix
corr_matrix = df.corr()

# Mask upper triangle
mask = np.triu(np.ones_like(corr_matrix, dtype=bool))

# Plot the heatmap
plt.figure(figsize=(20, 16))
sns.heatmap(corr_matrix, mask=mask, cmap="coolwarm", annot=False, fmt=".2f",
            linewidths=0.1, cbar=True, square=True)

plt.xticks(rotation=90)
plt.title("Feature Correlation Heatmap")
plt.show()

---

3. Data Preprocessing

Standardizing

# Standardize features
X_scaled = StandardScaler().fit_transform(X)

SMOTE

SMOTE (Synthetic Minority Over-sampling Technique) helps handle imbalanced datasets. It generates synthetic samples for the minority class rather than duplicating existing ones. This balances class distributions, reducing bias towards the majority class.

By plotting the distribution, we can see the effect of SMOTE on class imbalances

# Apply SMOTE to handle class imbalance
X_resampled, y_resampled = SMOTE(random_state=42).fit_resample(X_scaled, y)

plot_class_distribution(pd.Series(y_resampled), "Class Distribution After SMOTE")

PCA

Principal Component Analysis (PCA) is used to transform a high-dimensional dataset into a lower-dimensional form while preserving as much variance as possible.

Improves Efficiency

• High-dimensional datasets can be computationally expensive and difficult to process.
• PCA reduces the number of features while retaining the most critical information.
• This speeds up training times for machine learning models and reduces memory usage.

Removes Redundant & Correlated Features

• Many real-world datasets contain features that are highly correlated.
• PCA transforms correlated variables into a set of uncorrelated principal components, eliminating redundancy. This makes models more stable and interpretable.
• Reducing the number of dimensions helps prevent overfitting by removing irrelevant features.

# Apply PCA to retain 95% variance
pca = PCA(n_components=0.95, random_state=42)
X_pca = pca.fit_transform(X_resampled)

# Display removed features
removed_features = np.argsort(np.abs(pca.components_).sum(axis=0))[X_pca.shape[1]:]
print(f"Removed features: {df.columns[removed_features].tolist()}")

Removed features: ['acc_max_xyz_left', 'gyro_auc_xyz_right', 'gyro_mean_xy_right', 'acc_std_yz_right', 'acc_max_x_left', 'gyro_auc_zx_right', 'gyro_auc_zx_left', 'gyro_std_yz_left', 'acc_mean_x_left', 'gyro_std_xy_left', 'gyro_std_yz_right', 'gyro_max_y_right', 'gyro_std_xy_right', 'gyro_std_z_left', 'gyro_std_xyz_left', 'acc_std_xy_right', 'acc_max_zx_left', 'gyro_mean_y_right', 'gyro_auc_y_right', 'acc_mean_x_right', 'gyro_auc_xy_right', 'gyro_max_zx_left', 'acc_auc_x_right', 'gyro_auc_yz_right', 'acc_mean_roll_right', 'acc_auc_xy_left', 'acc_auc_pitch_left', 'acc_mean_pitch_left', 'acc_auc_yz_left', 'acc_mean_y_right', 'gyro_std_x_left', 'acc_std_z_right', 'acc_max_yz_left', 'gyro_std_xyz_right', 'acc_max_y_right', 'acc_auc_y_left', 'acc_mean_pitch_right', 'acc_mean_xyz_left', 'acc_peak_xyz_right', 'gyro_max_zx_right', 'acc_std_x_right', 'acc_auc_roll_right', 'gyro_max_z_left', 'acc_auc_y_right', 'gyro_std_z_right', 'gyro_std_x_right', 'gyro_mean_x_right', 'gyro_std_zx_left', 'acc_auc_pitch_right', 'acc_max_xyz_right', 'gyro_auc_x_right', 'acc_std_roll_left', 'acc_peak_yz_right', 'gyro_mean_y_left', 'acc_max_xy_right', 'acc_max_zx_right', 'gyro_auc_z_left', 'acc_std_y_right', 'gyro_auc_y_left', 'gyro_std_zx_right', 'gyro_mean_z_left', 'acc_max_x_right', 'gyro_max_x_left', 'gyro_max_z_right', 'acc_std_roll_right', 'gyro_mean_z_right', 'acc_std_pitch_right', 'acc_mean_xyz_right', 'acc_max_yz_right', 'gyro_max_x_right', 'acc_max_y_left', 'acc_mean_xy_right', 'gyro_auc_z_right', 'gyro_peak_xyz_left', 'acc_mean_roll_left', 'gyro_peak_yz_left', 'acc_max_roll_right', 'acc_peak_zx_right', 'gyro_mean_x_left', 'acc_mean_y_left', 'acc_max_roll_left', 'gyro_peak_xy_left', 'acc_auc_z_right', 'acc_auc_roll_left', 'acc_max_pitch_right', 'gyro_auc_x_left', 'acc_max_z_left', 'gyro_peak_yz_right', 'acc_auc_z_left', 'gyro_peak_xy_right', 'acc_peak_xy_right', 'acc_mean_z_right', 'gyro_peak_zx_left', 'acc_mean_xy_left', 'acc_auc_zx_right', 'acc_auc_xyz_right', 'acc_max_pitch_left', 'acc_max_z_right', 'acc_auc_yz_right', 'acc_peak_x_right', 'gyro_peak_xyz_right', 'acc_peak_y_right', 'acc_mean_z_left', 'gyro_peak_y_left', 'gyro_peak_zx_right', 'acc_std_pitch_left', 'acc_auc_xy_right', 'gyro_peak_z_left', 'acc_peak_z_right', 'acc_mean_yz_right', 'acc_mean_zx_right', 'gyro_peak_y_right', 'acc_mean_zx_left', 'gyro_peak_x_left', 'acc_mean_yz_left', 'gyro_peak_x_right', 'gyro_peak_z_right']

---