BTS National Mobility Analysis: Hybrid Computational Framework

Project Overview

This repository contains the technical implementation for the analysis of national mobility patterns using the Bureau of Transportation Statistics (BTS) dataset. The project evaluates the scalability and efficiency of distributed computing by comparing sequential Pandas processing with parallelized Dask architectures to identify trends in population movement and trip distribution.

Technical Methodology

The implementation addresses the core research objectives through a hybrid processing pipeline:

Data Ingestion: Optimized loading of large-scale CSV files using explicit Dask schemas and dtypes.
Exploratory Data Analysis (EDA): Comparative analysis of national trip volumes by distance categories.
Predictive Modelling: Linear Regression analysis correlating local mobility with regional travel patterns ($R^2 = 0.8608$).
Data Cleaning: Systematic handling of missing values (NaNs) and temporal data conversion for high-integrity modelling.

Performance Benchmarking

A comparative study was conducted across three configurations to evaluate scalability and identify computational bottlenecks:

Sequential: Single-worker baseline using Pandas for standard data manipulation.
Parallel (10 Workers): Optimized distributed configuration using Dask to reduce execution time.
Parallel (20 Workers): High-worker configuration utilized to demonstrate the impact of Scheduler Overhead on lower-complexity tasks.

Repository Structure

The source code and assets are organized as follows:

8 Notebooks: Individual .ipynb files containing implementation logic for data acquisition, cleaning, EDA, and predictive modelling.
12 Visuals: High-resolution PNG exports of all data visualizations, including grouped bar charts and performance benchmarks.
Documentation: Project requirements.txt and .gitignore for environment reproducibility and repository integrity.

Data Availability

Note: Due to the substantial file size of the Bureau of Transportation Statistics (BTS) mobility dataset, the raw CSV files are not hosted in this repository.

To replicate the results, the raw data must be sourced directly from the BTS national mobility database and placed in the appropriate directory as specified in the implementation notebooks.

Author

Naomi Adella Tan
Student ID: 15660518
Module: 5004CMD - Data Science
Coventry University

tann8/Naomi-Adella-Tan-5004CMD-Data-Science-Assignment

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Languages