Edge AI Anomaly Detection Part 3 - Machine Learning on Raspberry Pi

2020-06-08 | By ShawnHymel

License: Attribution Raspberry Pi

In the previous tutorial, we identified the most important features and trained two separate machine learning models. For the features, we computed the median absolute deviation (MAD) of every sample (consisting of 128 measurements of an accelerometer in the X, Y, and Z domain).

For the first model, we calculated the mean and covariance matrices of all of the training samples. We can then use the Mahalanobis Distance (MD) to measure how far from the mean of the training data cluster a new sample is. If the MD is too high, we can assume an anomaly has occurred.

For the second model, we used an autoencoder, which is a special type of neural network, to attempt to recreate the input MAD values at its output. We use the mean squared error (MSE) to judge how well the autoencoder accomplished its job. If the MSE is too high, we can say that an anomaly has occurred.

In this tutorial, we will re-train the models using just normal samples (no anomaly samples) and use those models to predict anomalies in real time. The ESP32 will run the same code from the first tutorial, which captures raw data from an MSA301 3-axis accelerometer attached to a ceiling fan. The ESP32 sends each set of measurements to our Raspberry Pi, which will attempt to predict anomalies by using one of the machine learning techniques discussed above.

See here if you would like to watch the accompanying video for this tutorial:

Overview

We want to use the same code to capture accelerometer data and send it to our server as we did in the first tutorial (Data Collection). Instead of storing those samples to separate files, we will first compute the MAD of each new sample (128 measurements) in real time. Then, we will use our machine learning model to predict whether the sample is an anomaly or not.

Deploy machine learning model to Raspberry Pi

While we can use any computer, I specifically wanted to show how we might use an edge device or single board computer to perform these calculations. As it turns out, creating a Python script to run our server and make predictions is not much different than what we’ve done in previous tutorials.

Just like in the Data Collection tutorial, we will use the same REST protocol to query the server and send over a group of measurements in JSON format.

Collect accelerometer data with an ESP32

The sensor node will send a GET request asking if the server is ready. The server will respond with a 1 (assuming it is ready to receive new data), and the node will send over a group of 200 samples.

Training for a State

From my experience with the system, both the Mahalanobis Distance and Autoencoder seem to only work in one, very particular state. If the ceiling fan is moved (even slightly!) or the accelerometer is adjusted, the model is no longer valid. After that, every new sample will appear as an anomaly.

One way to fix that is to train a more robust model. We’ll save that for the next tutorial.

For now, let’s try working within the limitation that we can’t move the fan or accelerometer. So, we’ll set the sensor node on the ceiling fan and run the data collection process again. From there, we’ll train a new model (no need to compare it to anomalous data, as we’ve already done that) and copy it over to our Raspberry Pi.

Retrain model for a particular device state

The Pi will then run inference to make predictions on whether new samples indicate an anomaly or not. Note that between training and deployment, we have not moved the fan nor the accelerometer.

For an actual product to use a system like this, you might need to test each unit at the end of the assembly process to train a model unique to that unit. Alternatively, you could have the end user train a model using a remote server and/or phone app. The model would only be good for that particular unit in that particular environment.

Customer training machine learning model

Training Steps

A note about the code for this tutorial: I will not show all of the code, as it can be quite lengthy. Source code can be found in this repository. Please download the repository and examine the code if you would like to follow along or try things out for yourself.

First up, head to the data_collection folder and run the esp32_accel_post on your ESP32. Make sure that you change the SSID and password for your WiFi network and change the IP address of the server to match that of the Rasbperry Pi. Tape or attach your ESP32 and accelerometer to your ceiling fan. Have your fan run in whatever you want to consider “normal” operation (low setting for me).

ESP32 and accelerometer on ceiling fan

Next, run http_accel_server.py on your Raspberry Pi to collect some data. I found that about 20-30 min of data worked well enough. I recommend copying the collected data files back to your computer for training (as we’ll need Jupyter Notebook and TensorFlow).

To create a new Mahalanobis Distance training model, run the anomaly-detection-training-mahalanobis.ipynb Notebook (note that you can also run the anomaly-detection-training-md-deploy.py script, which essentially does the same thing, but it does not require Jupyter Notebook). You’ll want to use the files we just collected to perform the training steps, and you can leave out the anomaly samples.

Run the training script, and it should produce a .npz file containing the mean and covariance matrices of our model. It should also give you a recommended threshold to use in your deployment program.

Perform the same steps with the autoencoder/anomaly-detection-training-autoencoder.ipynb Notebook. This should produce a .h5 keras file and a recommended threshold. Then, open the anomaly-detection-tflite-conversion.ipynb Notebook and run the first 5 cells to convert the .h5 file to a .tflite file.

At this point, it’s important to not move the fan or the accelerometer! If you do, you will need to redo the collection and training steps, as the model will no longer be valid.

Deploy Mahalanobis Distance Model

Copy the .npz file containing your mean and covariance matrices to your Raspberry Pi along with mahalanobis_distance/http_server_anomaly_detection_md.py. Open the Python script and set the threshold to whatever you discovered in the training step. Make sure that the MD_MODEL_FILE variable points to your .npz model file. Note that you might need to tweak the threshold based on system performance.

Run the Python script, and you should see samples start coming in. The script will compute the MAD values of each set of samples and then calculate the Mahalanobis Distance between those MAD values and the model’s mean. If the MD is over the threshold, the console should inform you as such.

Try lightly touching the fan blades or taping a few quarters to the end of one blade to see if you can make the system detect an anomaly.

Rasbperry Pi console showing output of MD script

Deploy Autoencoder Model

Copy the .tflite file you created during training over to your Raspberry Pi. Head to https://www.tensorflow.org/lite/guide/python and download the .whl file for your Raspberry Pi (check your CPU and Python version--the ARM32, Python 3.7 installer worked for me on a Raspberry Pi model 3B+). Install it with:

Copy Code

python -m pip install <name_of_wheel_file>.whl

From the GitHub repository, open auotencoder/http_server_anomaly_detection_tflite.py on your Raspberry Pi. Adjust the threshold as necessary (start with the recommended threshold from the training step) and make sure that TFLITE_MODEL_FILE points to your .tflite file. Save and run the script.

You should see samples come in, and the script should compute the MAD values for each set of samples. From there, it will run inference using your model and compute the MSE. If the MSE is over the threshold, it will report an anomaly.

Raspberry Pi console output of TFLite model

Try lightly touching the fan blades to see if you can generate an anomaly! Is it considered an anomaly when you turn the fan off?

Resources and Going Further

Remember that this only works when the fan is in one particular state! In the next tutorial, we’ll try creating a slightly more robust model (at the expense of some accuracy) so that we can move the fan slightly and not have to worry about re-training the model.

All code for this tutorial can be found here: https://github.com/ShawnHymel/tinyml-example-anomaly-detection

This is a great discussion about the Mahalanobis Distance: https://stats.stackexchange.com/questions/62092/bottom-to-top-explanation-of-the-mahalanobis-distance

Here is another great article on Autoencoders: https://www.jeremyjordan.me/autoencoders/

Recommended Reading