Raspberry Pi Face Recognition-Based Door Lock

2020-12-30 | By Maker.io Staff

License: See Original Project

The face recognition-based phone unlocking system launched by Apple on its iPhones in 2017 took the world by storm and has been a disruptive technology since then in the smartphone industry. Similarly, home security, monitoring and automation devices have recently become an integral part of many people’s lifestyle. A previous maker.io blog explained the complete ecosystem, the common sensors in existing commercial products, and the common concepts for the maker community.

This project builds a door unlocking system using a face-recognition system as a password. It uses a Raspberry Pi 4 board along with a Raspberry Pi V2 camera module and with a 7-inch touch screen display for visualization and to configure the key faces.

BOM

Raspberry Pi 4B
Raspberry Pi Camera Module V2
Jumper Cables
Servo Motor
LED Touch Screen 4.3 inches
Raspberry Pi4 USB-C Power Supply

Introducing the Project

Computer vision technology has been an area of active research for decades because the possibilities of applications are innumerable. The common applications of computer vision and image processing include text recognition, depth perception, visual odometry, object detection and recognition, pose estimation, human face detection and recognition, among many others.

Detecting and recognizing objects in a camera’s field of view is a very important task with applications in automation, robotics, and manufacturing industries.

The Difference Between Detecting and Recognizing a Face

Face Detection is the application of image processing techniques to determine if the image is/has a human face or not. It is analogous to object detection, but the distinct and consistent features of the human face make face detection a more informed task achievable by hand-tuned feature descriptors and conventional image processing methods. Face detection is the precursor to face recognition. Face detection in a scene with multiple people or objects introduces challenges in localization of the bounding box for the face.

The basic idea behind this is to look for common features of the human face in the image. The common human face features include:

Two eyes (blackish blobs) some distance apart horizontally
A nose in between the two eyes, placed vertically below
A flat rectangular/elliptical surface above the eyes
A pair of lips below the nose stretching horizontally

While there are many more such features, a common human face detection alone is a moderately complex algorithm.

Face Recognition is a more complex task that needs to classify if the human face detected matches the faces the software has already seen. It is essentially an object-classification-cum-object-detection task for human faces. The additional complexities arise when the intent is to make the face recognition algorithm robust enough to work efficiently even on images with different orientations, partial views, variable expressions, and facial hair changes.

Common Techniques for Face Detection & Recognition

The Viola-Jones algorithm for face detection using Haar Feature-based cascade classifier is a popular image processing technique. Histogram of Gradient (HOG)-based feature descriptors are widely used in the Haar cascade method. OpenCV, the largest open-source computer vision library, has an inbuilt implementation of the Haar Cascade-based face detection algorithm with a pre-trained classifier for human face classification.

face_1

Common feature detectors and their corresponding convolution on the human face (Image Source - OpenCV Tutorial Website)

A neural network is the most useful concept of machine learning that can learn to perform tasks based on training samples. Convolutional neural networks (CNNs), a supervised learning technique, use the concept of convolution of a feature detector over an input image to learn the common features descriptors in the image. A CNN is a type of deep neural network (a neural network with lots of constituent multi-perception layers) that has the potential to learn various features from multiple training images (labeled with classes) and, thus, can detect any incoming image belonging to any of the pre-trained image classes.

Project Outline

The project uses deep learning techniques for face recognition, and if the observed face matches the key faces configured in the application, it sends a message out to the door to unlock. The application uses a Raspberry Pi camera module V2 to continuously observe live video feed and detect human faces in it. If the human face matches the configured key human face, then it unlocks the door.

The application trains itself using these images stored in the project folder annotated with the names of the person. If the image in the live feed has a face matching the trained image encodings, the password is accepted.

The complete application is developed in the Python programming language. Using off-the-shelf libraries for face recognition and the Python library for servo motor control, the complete project is implemented.

system_2

Face Recognition-based Door Unlocking System (Image Source - Original)

User Interface

The user interface for the application is created using the PyQt5 library in Python. The GUI has a simple display where the live feed of the camera is presented, and the status of door locked or unlocked is also mentioned.

Users are encouraged to try updating the GUI with more features for automated new password face addition and different views in the UI.

Setting up the Hardware

The hardware interfaces include the following steps:

1. Connect the Raspberry Pi Camera Module V2 with the Raspberry Pi board using the 40 pin ribbon cable
2. Connect the Touch Screen display board with the Raspberry Pi board using another 40 pin ribbon cable
3. Connect the servo motor to the Raspberry Pi board using the GPIO pins

a. Connect the servo motor’s PWR and GND pins to the Vcc and GND pins on the RPi
b.Connect the signal pin to pin 17 of the RPi. Since the RPi uses software PWM, different pins can be used to generate the PWM signals for the motor.

4. Power up the Raspberry Pi board using an AC-DC power converter providing 5 V power

interfaced_3

Raspberry Pi Interfaced with the LCD Display (Image Source -Original)

schematic_4

Schematic for the connections (Image Source - Original Made in DK SchemeIt)

Setting up the Software

The complete project is documented and codes are available in the GitHub repository here. The repository has all codes to reproduce the project. The code architecture is shown below.

class_5

Class Architecture for the Face Recognition Door Lock Project (Image Source - Original)

To configure the Raspberry Pi with the OS and install the appropriate LCD packages and kernel, refer to the docs folder in the code repository.

The most important libraries used in the project are:

dlib - An efficient machine learning and deep learning library written in C++ with bindings for Python
face_recognition - A Python package with dlib backend running dlib’s state-of-the-art face recognition library
RPi.GPIO - A library to control Raspberry Pi GPIO pin outputs
PyQt5 - Python bindings for the Qt C++ library for developing user interfaces and display tools
OpenCV - Computer Vision library primarily used to read the image and update annotations on it

Execution Steps

After setting up the hardware as shown above, execute the following steps to run the code and the project.

Setup the Python virtual environment with all tools, libraries, and packages using the command make install-deps in the terminal.
Open the repository and store an image containing the face of the person in the saved_images folder.
Run make run to run the program. It will open up a window showing the live feed of the camera. When it sees a face matching the faces in the saved_images folder, it executes a Raspberry Pi GPIO instruction and rotates the servo motor, representative of unlocking the door.

cover_6

Working Example of the Program (Image Source - Original)

Conclusion

Raspberry Pi is the favorite maker platform for single-board computers used by hobbyists and makers. While this project runs on a battery, ideally using a 5 V/1 A AC-DC power source is the most sustainable and cost-effective solution.

Human face detection and recognition have similarly been a very important topic for security and tracking operations. While the tools and algorithms used in this library are not as robust and safe as the commercial systems, it has a lot of potential to be a substitute with a few design and implementation tweaks (beyond the scope of this article). Users are encouraged to try enhancements to the project and let us know what you find!