godrive

README

GoDrive

Description

GoDrive is a lane following autonomous driving robot implementation that uses image classification as a form of imitation learning. sum-product networks (SPNs) are used as a way to compute exact inference in linear time, allowing for accurate and fast uncertainty measuring in real-time.

This code is part of my undergraduate thesis Mobile Robot Self-Driving Through Image Classification Using Discriminative Learning of Sum-Product Networks. The full final thesis can be read here. Both prediction and training of SPNs are done through the GoSPN library.

Objectives

The primary objectives of this implementation are twofold: both as a comparative study on different SPN architectures and learning methods, and also as a preliminar work on SPNs for self-driving and their feasibility as a real-time prediction model. A third secondary objective was to compare SPNs with state-of-the-art multilayer perceptrons (MLPs) and convolutional neural-networks (CNNs).

Lane following as self-driving

Ours is a primitive approach to self-driving, mainly that of lane following through imitation learning. The robot's objective is to remain inside a designated lane whilst still moving forward. Since the robot is not allowed to stop, constantly moving forward, the prediction model must be both accurate - identifying lane markings and making the necessary heading corrections - and fast.

The robot was allowed to execute three different operations: go forward, turn left, or turn right. Prediction output encoded these three commands as a single byte.

Hardware

We experimented with the Lego Mindstorm NXT, nicknamed Brick. A Raspberry Pi Model 3, nicknamed Berry, was attached to the bot, together with a low cost webcam. The Berry was then used for image capturing, processing, and label prediction, sending the predicted label to the Brick, who was only tasked with executing corresponding motor commands.

Prediction

Prediction was done in real-time. The implementation contained in this repository takes advantage of the Berry's four CPU cores. Three cores are dedicated to computing each label concurrently. The fourth core is used for image capturing, processing and sending the predicted byte to the Brick.

Training

Training was done separately on a desktop computer. You can read more about training and validation here

Results

Results are available both in the thesis and also in video.

Code structure

The code is structured as follows:

• root:
  • contest.go: connection test for testing if the bot is visible
  • main.go: main file for executing training or self-driving
• bot:
  • bot.go: bot loop, communication, prediction and image processing
  • usb.go: USB communication handling from Berry to Brick
• camera:
  • writer.go: writer inferface for recording camera feed
  • camera.go: camera processing and capturing
  • trans.go: image transformations
• data:
  • data.go: pulling data from dataset
• models:
  • model.go: interface for prediction models
  • accuracy.go: validation code for accuracy measuring
  • dennis.go: Dennis-Ventura architecture inference, learning and serialization
  • gens.go: Gens-Domingos architecture inference, learning and serialization
• java:
  • Remote.java: Brick-side code for interpreting messages as motor power