Investigating Machine Learning Approaches for Detecting Misleading Online News

Vincent Nathanael | 14913994 | Coventry University
Module: 6003CMD | Supervisor: Ms. Anuja Talekar | Ethics: P190734

Project Overview

This project investigates whether machine learning models can statistically distinguish between misleading and credible political statements using the LIAR dataset (Wang, 2017). Four models are compared: Logistic Regression, Naive Bayes, SVM, and BERT.

The study is framed as a pattern-recognition task, not truth verification. Results and limitations are discussed critically in the dissertation.

Results Summary

Model	Accuracy	Precision	Recall	F1-Score
BERT	0.6196	0.6147	0.6196	0.6137
Logistic Regression	0.6085	0.6020	0.6085	0.5969
SVM	0.5975	0.5925	0.5975	0.5930
Naive Bayes	0.5848	0.5776	0.5848	0.5768

All models evaluated on the held-out LIAR test set (1,267 instances).

Repository Structure

6003CMD-FAKENEWS/
├── notebooks/
│   ├── 01_data_exploration.ipynb
│   ├── 02_preprocessing.ipynb
│   ├── 03_classical_models.ipynb
│   ├── 04_bert_local.ipynb
│   └── 05_evaluation.ipynb
├── results/
│   ├── final_results_summary.csv
│   ├── classical_models_results.csv
│   ├── bert_results.csv
│   ├── all_models_comparison.png
│   ├── grouped_comparison.png
│   ├── classical_models_comparison.png
│   ├── bert_training_curve.png
│   ├── all_confusion_matrices.png
│   ├── cm_bert.png
│   ├── cm_logistic_regression.png
│   ├── cm_naive_bayes.png
│   ├── cm_svm.png
│   ├── label_distribution.png
│   ├── statement_length.png
│   └── top_tfidf_features.png
├── .gitignore
├── requirements.txt
└── README.md

data/, models/, and venv/ are excluded from the repository. The dataset is downloaded automatically by notebook 01.

How to Reproduce

1. Clone the repository

git clone https://github.coventry.ac.uk/nathanaelv/6003CMD-dissertation.git
cd 6003CMD-dissertation

2. Create and activate a virtual environment

python3 -m venv venv
source venv/bin/activate

3. Install dependencies

pip install -r requirements.txt

4. Run the notebooks in order

01_data_exploration.ipynb   — downloads LIAR dataset, explores label distribution and statement length
02_preprocessing.ipynb      — text cleaning, tokenisation, lemmatisation, TF-IDF vectorisation
03_classical_models.ipynb   — trains and evaluates Logistic Regression, Naive Bayes, SVM
04_bert_local.ipynb         — fine-tunes BERT (MPS / CUDA / CPU)
05_evaluation.ipynb         — final comparison charts and summary table

The LIAR dataset is downloaded automatically in notebook 01. No manual download needed.

5. BERT training note

Notebook 04 detects the available device automatically:

Apple Silicon (M1/M2/M3): uses MPS — approximately 8-9 minutes for 3 epochs
NVIDIA GPU: uses CUDA
CPU only: significantly longer (30-60 minutes)

Dependencies

See requirements.txt. Key libraries:

scikit-learn — classical models and evaluation
torch + transformers — BERT fine-tuning
nltk — tokenisation and lemmatisation
pandas, numpy, matplotlib, seaborn

Dataset

LIAR dataset (Wang, 2017)
Source: https://github.com/tfs4/liar_dataset
Downloaded automatically by notebook 01.

References

Wang, W. Y. (2017). "Liar, Liar Pants on Fire": A new benchmark dataset for fake news detection. Proceedings of ACL 2017. https://doi.org/10.18653/v1/P17-2067

nathanaelv/6003CMD-dissertation

About

Resources

Stars

Watchers

Forks

Releases

Languages