Project Name

Dissertation Code

Description

This is my implementation of the deep learning models as described in the dissertation. Below is a description of the structure, dependencies, and example usage.

Structure

---

multi_lettuce

This directory includes the code for the deep lettuce detection model. It is split into the following subdirectories:

datasets:

This directory includes the original data used in training in YOLO format, in the subdirectories:

• datasets/data/training: Training data
• datasets/data/validation: Testing data

multi_lettuce_model.ipynb:

Includes the code to define, instantiate, train, test, and validate the model. The first half of the code, which includes the training, takes a long time to run. For this reason, the resulting model is saved in the parent directory (yolov8s.pt). This can be loaded in the second part of the code, which loads the saved model's weights for training and evaluation. This code is faster to run; however, its outputs are already included in the parent directory.

identified_lettuces_rgb_images:

Includes extracted RGB single lettuce images based on predicted detections from the last run.

identified_lettuces_depth_images:

Includes extracted depth single lettuce images based on predicted detections from the last run.

---

single_lettuce

This directory includes the code for the deep CNN Regressor. It is split into the following subdirectories:

images:

This directory contains the preprocessing directory:

• preprocessing:
  • This directory contains the code for the image preprocessing conducted before the model's training.
  • depth_images: Contains the original single-lettuce depth images
  • rgb_images: Contains the original single-lettuce RGB images
  • depth_images_cropped: Contains the cropped depth images
  • rgb_images_cropped: Contains the cropped RGB images
  • depth_images_cropped_grayscale: Contains the cropped depth images which were converted to grayscale
  • depth_to_grayscale.ipynb: Contains the code to convert the cropped depth images to grayscale. If rerun, it will overwrite the depth_images_cropped_grayscale directory
  • image_cropping.ipynb: Contains the code to crop both RGB and depth images based on a manually specified center. If run, it will overwrite the cropped images, requiring manual re-specification of 388 lettuce images centers.

training_logs:

Contains the training event logging from each fold which are used to visualize the training using TensorBoard.

single_lettuce_cv_model.ipynb:

Contains the code to define, initialize, train, test, and evaluate the model. Training is performed during cross-validation and it takes substantial time. To avoid this, the models from each fold are saved along with the visualization images regarding the model's predictions when compared to actual labels. If the training code is re-run, training_logs will be overwritten, as well as the saved models. Conducting the cross-validation also consumes a large amount of memory.

Dependencies

The code for each model was run on different Anaconda environments, each with certain installed packages due to collisions on the typing-extensions versions between tensorflow and torch.

Environment 1: single_lettuce code (deep CNN Regressor model)

Package Version

---

absl-py 2.1.0 asttokens 2.4.1 astunparse 1.6.3 backcall 0.2.0 cachetools 5.3.3 certifi 2024.2.2 charset-normalizer 3.3.2 colorama 0.4.6 comm 0.2.1 contourpy 1.1.1 cycler 0.12.1 debugpy 1.8.1 decorator 5.1.1 executing 2.0.1 filelock 3.13.3 flatbuffers 23.5.26 fonttools 4.49.0 fsspec 2024.3.1 gast 0.4.0 google-auth 2.28.1 google-auth-oauthlib 1.0.0 google-pasta 0.2.0 graphviz 0.20.3 grpcio 1.62.0 h5py 3.10.0 idna 3.6 importlib-metadata 7.0.1 importlib_resources 6.1.2 ipykernel 6.29.3 ipython 8.12.3 jedi 0.19.1 Jinja2 3.1.3 joblib 1.3.2 jupyter_client 8.6.0 jupyter_core 5.7.1 keras 2.13.1 kiwisolver 1.4.5 labelImg 1.8.6 libclang 16.0.6 lxml 5.1.0 Markdown 3.5.2 MarkupSafe 2.1.5 matplotlib 3.7.5 matplotlib-inline 0.1.6 mpmath 1.3.0 nest-asyncio 1.6.0 networkx 3.1 numpy 1.24.3 oauthlib 3.2.2 opencv-python 4.9.0.80 opt-einsum 3.3.0 packaging 23.2 pandas 2.0.3 parso 0.8.3 pickleshare 0.7.5 pillow 10.2.0 pip 23.3.1 platformdirs 4.2.0 prompt-toolkit 3.0.43 protobuf 4.25.3 psutil 5.9.8 pure-eval 0.2.2 py-cpuinfo 9.0.0 pyasn1 0.5.1 pyasn1-modules 0.3.0 pydot 2.0.0 Pygments 2.17.2 pyparsing 3.1.1 PyQt5 5.15.10 PyQt5-Qt5 5.15.2 PyQt5-sip 12.13.0 python-dateutil 2.9.0.post0 pytz 2024.1 pywin32 306 PyYAML 6.0.1 pyzmq 25.1.2 requests 2.31.0 requests-oauthlib 1.3.1 rsa 4.9 scikit-learn 1.3.2 scipy 1.10.1 seaborn 0.13.2 setuptools 68.2.2 six 1.16.0 stack-data 0.6.3 sympy 1.12 tensorboard 2.13.0 tensorboard-data-server 0.7.2 tensorflow 2.13.0 tensorflow-estimator 2.13.0 tensorflow-intel 2.13.0 tensorflow-io-gcs-filesystem 0.31.0 termcolor 2.4.0 thop 0.1.1.post2209072238 threadpoolctl 3.3.0 torch 2.2.2 torchvision 0.17.2 tornado 6.4 tqdm 4.66.2 traitlets 5.14.1 typing-extensions 3.10.0.2 tzdata 2024.1 urllib3 2.2.1 wcwidth 0.2.13 Werkzeug 3.0.1 wheel 0.41.2 wrapt 1.16.0 zipp 3.17.0

Environment 2: multi_lettuce code (deep YOLOv8 lettuce detection model)

Package Version

---

absl-py 2.1.0 anyio 4.2.0 argon2-cffi 23.1.0 argon2-cffi-bindings 21.2.0 arrow 1.3.0 asttokens 2.4.1 astunparse 1.6.3 async-lru 2.0.4 attrs 23.2.0 Babel 2.14.0 beautifulsoup4 4.12.3 bleach 6.1.0 blinker 1.7.0 cachetools 5.3.2 certifi 2023.11.17 cffi 1.16.0 charset-normalizer 3.3.2 click 8.1.7 colorama 0.4.6 comm 0.2.1 contourpy 1.2.0 cycler 0.12.1 debugpy 1.8.0 decorator 5.1.1 defusedxml 0.7.1 exceptiongroup 1.2.0 executing 2.0.1 fastjsonschema 2.19.1 filelock 3.13.3 Flask 3.0.0 Flask-Login 0.6.3 Flask-SQLAlchemy 3.1.1 flatbuffers 23.5.26 fonttools 4.49.0 fqdn 1.5.1 fsspec 2024.3.1 gast 0.5.4 google-auth 2.27.0 google-auth-oauthlib 1.2.0 google-pasta 0.2.0 graphviz 0.20.3 greenlet 3.0.1 grpcio 1.60.0 h5py 3.10.0 idna 3.6 ipykernel 6.29.0 ipython 8.21.0 ipywidgets 8.1.1 isoduration 20.11.0 itsdangerous 2.1.2 jedi 0.19.1 Jinja2 3.1.2 joblib 1.4.0 json5 0.9.14 jsonpointer 2.4 jsonschema 4.21.1 jsonschema-specifications 2023.12.1 jupyter 1.0.0 jupyter_client 8.6.0 jupyter-console 6.6.3 jupyter_core 5.7.1 jupyter-events 0.9.0 jupyter-lsp 2.2.2 jupyter_server 2.12.5 jupyter_server_terminals 0.5.2 jupyterlab 4.0.12 jupyterlab_pygments 0.3.0 jupyterlab_server 2.25.2 jupyterlab-widgets 3.0.9 keras 2.15.0 kiwisolver 1.4.5 libclang 16.0.6 Markdown 3.5.2 MarkupSafe 2.1.3 matplotlib 3.8.3 matplotlib-inline 0.1.6 mistune 3.0.2 ml-dtypes 0.2.0 mpmath 1.3.0 nbclient 0.9.0 nbconvert 7.14.2 nbformat 5.9.2 nest-asyncio 1.6.0 networkx 3.3 notebook 7.0.7 notebook_shim 0.2.3 numpy 1.25.0 oauthlib 3.2.2 opencv-python 4.9.0.80 opt-einsum 3.3.0 overrides 7.7.0 packaging 23.2 pandas 2.2.1 pandocfilters 1.5.1 parso 0.8.3 pillow 10.2.0 pip 23.3.2 platformdirs 4.2.0 prometheus-client 0.19.0 prompt-toolkit 3.0.43 protobuf 4.23.4 psutil 5.9.8 pure-eval 0.2.2 py-cpuinfo 9.0.0 pyasn1 0.5.1 pyasn1-modules 0.3.0 pycparser 2.21 pydot 2.0.0 Pygments 2.17.2 pyparsing 3.1.1 python-dateutil 2.8.2 python-json-logger 2.0.7 pytz 2024.1 pywin32 306 pywinpty 2.0.12 PyYAML 6.0.1 pyzmq 25.1.2 qtconsole 5.5.1 QtPy 2.4.1 referencing 0.33.0 requests 2.31.0 requests-oauthlib 1.3.1 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rpds-py 0.17.1 rsa 4.9 scikit-learn 1.4.2 scipy 1.12.0 seaborn 0.13.2 Send2Trash 1.8.2 six 1.16.0 sniffio 1.3.0 soupsieve 2.5 SQLAlchemy 2.0.23 stack-data 0.6.3 sympy 1.12 tensorboard 2.15.1 tensorboard-data-server 0.7.2 tensorflow 2.15.0 tensorflow-estimator 2.15.0 tensorflow-intel 2.15.0 tensorflow-io-gcs-filesystem 0.31.0 termcolor 2.4.0 terminado 0.18.0 thop 0.1.1.post2209072238 threadpoolctl 3.4.0 tinycss2 1.2.1 tomli 2.0.1 torch 2.2.2 torchvision 0.17.2 tornado 6.4 tqdm 4.66.2 traitlets 5.14.1 types-python-dateutil 2.8.19.20240106 typing_extensions 4.8.0 tzdata 2024.1 ultralytics 8.1.45 uri-template 1.3.0 urllib3 2.2.0 wcwidth 0.2.13 webcolors 1.13 webencodings 0.5.1 websocket-client 1.7.0 Werkzeug 3.0.1 wheel 0.42.0 widgetsnbextension 4.0.9 wrapt 1.14.1