DIY Self-Driving Car – Collecting Data – What do we need?

To train a driving model, we need driving data. Let’s break down what exactly we’re interested in.

A fundamental requirement: We don’t want to do any manual labeling. We will need at least a couple hours of driving, which means 100.000s of frames recorded. We can’t possibly label those by hand within the scope of a hobby project.

We want our system to predict the required steering wheel angle to stay in lane, based on a camera image. So record those and we’re good to go, right? Not quite.

Although „Staying in lane“ is the main thing we do during driving, there are other actions we take every now and then, and we don’t want the model to worry about those:

1. Taking turns
2. Changing lanes
3. Overtaking/Passing cyclists/car/construction equipment etc.
4. Navigating independently of lanes (e.g. parking lots)

Fortunately there are measurable indicators that allow us to detect those actions with sufficient reliability and to remove the corresponding samples from our dataset before training. No manual labeling required!

For each of those actions at least one of three things should be true:

Driving action	Indicator light	Large SWA	Low speed
Turning	X	X	X
Lane change	X
Passing	X
Open space		X	X

If none of those is true, we assume what’s happening is „Follow Lane“ and we can use it for training our model. If however any of these apply, we assume something else is happening and just skip the sample.

Doing it this way might be more restrictive than necessary, as it will also remove scenarios where we just follow a lane at low speeds or follow tight curves. Might rethink this logic later.

Now indicators can only be either on or off, so that’s an easy one, but what exactly is a „large“ steering wheel angle and what counts as „low“ speed?

We’ll decide that after recording some data and visualizing it.

TL;DR We should also record vehicle speed and whether or not any indicator light is active.

We will also record the GPS position so we can ensure that road sections that we drive a lot won’t be overrepresented in our training data. We want our model to drive well on all roads, not just on those that are e.g. close to our frequent parking spots.

For validation we’ll also record GPS speed. If the speed signals provided by the vehicle CAN bus and GPS don’t match up within a reasonable margin of error, we know that at least one one these data sources is not reliable and we can’t trust and shouldn’t use the sample. Recording the currently used number of GPS satellites serves the same purpose.

Summary: We’ll save camera frames to individual .jpg files and create an accompanying .csv-file that contains:

• Camera frame filename
• Steering wheel angle
• Vehicle speed (CAN)
• Left indicator (on/off)
• Right indicator (on/off)
• GPS longitude
• GPS latitude
• GPS speed
• GPS sat count

-> Coming soon: Part 4: Collecting Data – Data Analysis