Quantum-Enhanced Anomaly Detection of User Behaviour for Cybersecurity

This repository contains the complete code and instructions to reproduce the results for the dissertation titled "Quantum-Enhanced Anomaly Detection of User Behaviour for Cybersecurity: A Comparative Analysis on the CSE-CIC-IDS2018 Dataset."

The experiments are designed to be run in a Kaggle Notebook environment.

Project Overview

This project conducts a comparative analysis between classical machine learning models (SVM, ANN) and quantum-enhanced models (QSVM, VQC) for network intrusion detection using the CSE-CIC-IDS2018 dataset. This README provides a step-by-step guide to set up the environment, run the experiments, and generate all the figures presented in the dissertation.

Prerequisites

A Kaggle Account.
The CSE-CIC-IDS2018 Dataset added to your Kaggle Notebook. You can find it here: https://www.kaggle.com/datasets/solarmainframe/ids-intrusion-csv

How to Reproduce the Results

To ensure a clean and stable environment, it is crucial to follow these steps in the exact order specified. The primary challenge in this project is managing library dependencies; these steps are designed to create a stable, compatible environment from the start.

Step 1: Factory Reset the Kaggle Notebook

Before running any code, you must start with a completely clean environment to avoid library conflicts.

Create a new Kaggle Notebook and add the CSE-CIC-IDS2018 dataset as an input.
In the top menu, navigate to Run > Factory reset.
Confirm the action. This will wipe any pre-existing conflicting libraries and give you a clean slate.

Step 2: Install All Required Libraries

After the factory reset, copy the entire code block from CELL 1 below into the first cell of your notebook. This single command installs a complete set of libraries with specific, compatible versions that are known to work together for this entire project.

Step 3: Restart the Session

After the installation in Cell 1 is complete, you must restart the session. This is a critical step that forces the notebook to load the new libraries you just installed.

In the top menu, navigate to Run > Restart & clear cell outputs.

Step 4: Execute the Main Code

After restarting, copy the code from CELL 2 below into a new, separate cell in your notebook. This single, consolidated script contains all the steps to load and clean the data, train all four models, and generate all the figures from Chapter 4 and Chapter 5 of the dissertation.

You can now run this cell. The entire process will take a significant amount of time to complete (likely over an hour) due to the quantum model simulations.

CELL 1: Environment Setup and Installations

# This single cell should be run first in a new session after a factory reset.
# It installs a complete, stable, and compatible environment for the entire project.

print("Installing a stable, modern environment for classical and quantum analysis...")
!pip install \
    numpy==1.26.4 \
    scikit-learn==1.4.2 \
    imbalanced-learn==0.12.0 \
    matplotlib==3.8.4 \
    seaborn==0.13.2 \
    pandas==2.2.2 \
    tensorflow==2.16.1 \
    qiskit==1.0.2 \
    qiskit-algorithms==0.3.0 \
    qiskit-machine-learning==0.8.0 \
    pylatexenc==2.10 \
    shap==0.45.0 \
    -q

print("\nInstallation complete.")
print("IMPORTANT: Please go to 'Run > Restart Session' in the menu before running the next cell.")

CELL 2: Main Experimental Pipeline (Run After Restart)

# --- Step 1: Import all necessary libraries ---
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.ensemble import RandomForestClassifier
from imblearn.over_sampling import SMOTE
from sklearn.svm import SVC
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout
from sklearn.metrics import confusion_matrix, roc_curve, roc_auc_score
import matplotlib.pyplot as plt
import seaborn as sns
import time
import shap

# Import modern Qiskit components
from qiskit.circuit.library import ZZFeatureMap, RealAmplitudes
from qiskit_algorithms.optimizers import COBYLA
from qiskit_algorithms.utils import algorithm_globals
from qiskit_machine_learning.algorithms import VQC, QSVC
from qiskit_machine_learning.kernels import FidelityQuantumKernel
from qiskit.primitives import Sampler

print("All libraries imported successfully.")

# --- Step 2: Load, Clean, and Prepare Data ---
print("\n--- Loading and Cleaning Dataset ---")
file_path = '/kaggle/input/ids-intrusion-csv/02-14-2018.csv'
df = pd.read_csv(file_path)
df.replace([np.inf, -np.inf], np.nan, inplace=True)
df.dropna(inplace=True)
df['Label'] = df['Label'].str.strip()
print("Dataset loaded and cleaned successfully.")

# --- Feature Selection ---
print("\n--- Performing Feature Selection ---")
X_full = df.drop(['Label', 'Timestamp'], axis=1)
y_full = df['Label'].apply(lambda x: 0 if x == 'Benign' else 1)
rfc = RandomForestClassifier(n_estimators=100, random_state=42, n_jobs=-1)
rfc.fit(X_full, y_full)
importances = pd.Series(rfc.feature_importances_, index=X_full.columns)
top_20_features = importances.nlargest(20).index.tolist()
print("Top 20 features selected.")
final_df = df[top_20_features + ['Label']]

# --- Data Preparation for Classical Models ---
print("\n--- Preparing Data for Classical Models ---")
X = final_df.drop('Label', axis=1)
y = final_df['Label'].apply(lambda x: 0 if x == 'Benign' else 1)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
smote = SMOTE(random_state=42)
X_train_resampled, y_train_resampled = smote.fit_resample(X_train_scaled, y_train)
print("Classical data preparation complete.")

# --- Step 3: Train and Evaluate Classical Models ---
# SVM
print("\nTraining SVM...")
svm_model = SVC(kernel='rbf', probability=True, random_state=42)
svm_model.fit(X_train_resampled, y_train_resampled)
y_pred_svm = svm_model.predict(X_test_scaled)
y_prob_svm = svm_model.predict_proba(X_test_scaled)[:, 1]
cm_svm = confusion_matrix(y_test, y_pred_svm)
print("SVM training complete.")
plt.figure(figsize=(8, 6)); sns.heatmap(cm_svm, annot=True, fmt='d', cmap='Blues', xticklabels=['Benign', 'Attack'], yticklabels=['Benign', 'Attack']); plt.title('Figure 4.8: Confusion Matrix for SVM'); plt.show()

# ANN
print("\nTraining ANN...")
tf.random.set_seed(42)
ann_model = Sequential([Dense(64, activation='relu', input_shape=(X_train_resampled.shape[1],)), Dropout(0.5), Dense(32, activation='relu'), Dropout(0.5), Dense(1, activation='sigmoid')])
ann_model.compile(optimizer='adam', loss='binary_crossentropy')
ann_model.fit(X_train_resampled, y_train_resampled, epochs=20, batch_size=64, verbose=0)
y_prob_ann = ann_model.predict(X_test_scaled, verbose=0).ravel()
y_pred_ann = (y_prob_ann > 0.5).astype("int32")
cm_ann = confusion_matrix(y_test, y_pred_ann)
print("ANN training complete.")
plt.figure(figsize=(8, 6)); sns.heatmap(cm_ann, annot=True, fmt='d', cmap='Greens', xticklabels=['Benign', 'Attack'], yticklabels=['Benign', 'Attack']); plt.title('Figure 4.9: Confusion Matrix for ANN'); plt.show()

# --- Step 4: Prepare Data for Quantum Models ---
print("\n--- Preparing Data for Quantum Models ---")
benign_df = final_df[final_df['Label'] == 'Benign'].sample(n=200, random_state=42)
attack_df = final_df[final_df['Label'] != 'Benign'].sample(n=200, random_state=42)
df_q_sample = pd.concat([benign_df, attack_df])
X_q = df_q_sample.drop('Label', axis=1)
y_q = df_q_sample['Label'].apply(lambda x: 0 if x == 'Benign' else 1)
scaler_q = StandardScaler(); X_q_scaled = scaler_q.fit_transform(X_q)
pca = PCA(n_components=4); X_q_pca = pca.fit_transform(X_q_scaled)
X_train_q, X_test_q, y_train_q, y_test_q = train_test_split(X_q_pca, y_q.values, test_size=0.2, random_state=42, stratify=y_q)
print("Quantum data preparation complete.")

# --- Step 5: Train and Evaluate Quantum Models ---
algorithm_globals.random_seed = 42
sampler = Sampler()
feature_map = ZZFeatureMap(feature_dimension=4, reps=2)

# VQC
print("\n--- WARNING: Training VQC. This will be very slow. ---")
ansatz = RealAmplitudes(num_qubits=4, reps=3)
optimizer = COBYLA(maxiter=100)
vqc = VQC(sampler=sampler, feature_map=feature_map, ansatz=ansatz, optimizer=optimizer)
vqc.fit(X_train_q, y_train_q)
y_pred_vqc = vqc.predict(X_test_q)
cm_vqc = confusion_matrix(y_test_q, y_pred_vqc)
print("VQC training complete.")
plt.figure(figsize=(8, 6)); sns.heatmap(cm_vqc, annot=True, fmt='d', cmap='Oranges', xticklabels=['Benign', 'Attack'], yticklabels=['Benign', 'Attack']); plt.title('Figure 4.10: Confusion Matrix for VQC'); plt.show()

# QSVM
print("\n--- WARNING: Training QSVM. This will also be very slow. ---")
quantum_kernel = FidelityQuantumKernel(feature_map=feature_map)
qsvc = QSVC(quantum_kernel=quantum_kernel, random_state=42)
qsvc.fit(X_train_q, y_train_q)
y_pred_qsvc = qsvc.predict(X_test_q)
cm_qsvc = confusion_matrix(y_test_q, y_pred_qsvc)
print("QSVM training complete.")
plt.figure(figsize=(8, 6)); sns.heatmap(cm_qsvc, annot=True, fmt='d', cmap='Purples', xticklabels=['Benign', 'Attack'], yticklabels=['Benign', 'Attack']); plt.title('Figure 4.11: Confusion Matrix for QSVM'); plt.show()

# --- Step 6: SHAP Analysis for ANN ---
print("\n--- Performing SHAP Analysis for ANN ---")
print("Calculating SHAP values... This may take several minutes.")
background = shap.sample(pd.DataFrame(X_train_resampled, columns=top_20_features), 100)
explainer = shap.KernelExplainer(lambda x: ann_model.predict(x, verbose=0), background)
shap_values = explainer.shap_values(pd.DataFrame(X_test_scaled, columns=top_20_features).sample(200, random_state=42))
print("SHAP values calculated.")
print("Generating SHAP summary plot for Figure 4.13...")
shap.summary_plot(shap_values[0], pd.DataFrame(X_test_scaled, columns=top_20_features).sample(200, random_state=42), show=False)
plt.title("Figure 4.13: SHAP Summary Plot for ANN Model")
plt.show()

edulakants/Dissertation_15172929

About

Resources

Stars

Watchers

Forks

Releases

Languages