5004CMD — Big Data Programming Project

Analysis of National Mobility Data Using Serial and Parallel Computing

Module: 5004CMD Data Science | Coventry University | 2025–26
Student: Pradeepa Periyasamy (15574787)
Module Leader: Dr. Katerina Stamou

Project Overview

This project implements a complete data science pipeline to analyse national mobility data from the U.S. Bureau of Transportation Statistics (BTS). It compares serial processing (Pandas) against parallel processing (Dask) using 10 and 20 worker configurations across four analytical questions.

Repository Contents

File	Description
`5004CMD_Big_Data_Coursework.ipynb`	Main Jupyter Notebook — all 18 cells, serial and parallel
`Trips_Full Data.csv`	Small dataset (Week 32, 2019)
`Trips_by_Distance.csv`	Large dataset (Jan 2019 – mid 2021)
`outputs/`	All generated PNG charts (QA, QB, QC, QD, performance)

Questions Addressed

Question	Description
A	Weekly stay-at-home population + mean trips by distance
B	Dates with >10M trips in 10–25 mile and 50–100 mile bands
C	Linear regression: predict Trips 5–10 from Trips 1–25 Miles
D	Travellers by distance category — bar, pie, horizontal charts

Key Results

Serial total time: 7.97 seconds
Parallel 10W total time: 10.69 seconds
Parallel 20W total time: 8.32 seconds
Regression R²: 0.9457 | RMSE: 1,981,455
Trips under 25 miles account for over 93% of all national daily trips

How to Run

Install dependencies: pip install pandas dask distributed scikit-learn matplotlib
Place both CSV files in the same folder as the notebook
Open in Jupyter Notebook (Anaconda recommended)
Run all cells top to bottom (Cell 1 → Cell 18)
Dask dashboard available at the URL printed after Client initialisation

Technologies Used

Python 3.9+ | Jupyter Notebook | Pandas | Dask | Scikit-learn | Matplotlib

periyasamp/Pradeepa-Periyasamy-5004CMD-Big-Data-Project

About

Resources

Stars

Watchers

Forks

Releases

Languages