AuraLock: A Facial Recognition Home Security Device
This project began as a solution to a fairly common problem: One’s inability to quickly open their home door while carrying multiple items. Whether we are carrying two trips worth of groceries, or simply fumbling around with a phone, entering our home can become a serious annoyance. AuraLock is a project designed to alleviate the need for keys or even fingerprints, resulting in a truly hands-free entry experience, all while maintaining 24/7 security for you and your family.
Abstract
This project centers around a door lock that implements facial recognition and smart features via mobile app integration. The lock hardware itself will retract a deadbolt when a valid face is detected, allowing the user to simply push the door open on their way in, providing ease of access second only to unsecured automatic sliding doors. In addition, a user will be able to download an application onto their Android mobile device that provides them greater features and control over their door locking device. These features include the ability to create a user account, unlock the device remotely, add new users, retrieve a Live Capture Image, and allow the lock to be opened by any face for a set period if the user is expecting company. This project is displayed working on a miniaturized door mockup with a functioning mobile app.
Hardware
To begin, the brain for this project is a Raspberry Pi 4, and the peripherals are seen in the following image, along with the direction of data transfer. As you can see, Google FireBase is used as a communication medium between the Android Mobile app and the Raspberry Pi itself.
The previous wiring diagram was then converted to a PCB design using EagleCAD. This PCB was then milled out on a Bantam desktop PCB mill via subtractive manufacturing.
The following images represent the 3D modeling for the hardware housing. All 3D modeling was done using SolidWorks, however Fusion360 was implemented when preparing the deadbolt to be milled on the Tormach CNC.
Here one can see a collection of images displaying the process of manufacturing the previous 3D models using FDM 3D printing and CNC.
The wiring schematic seen here illustrates the connections between each of the hardware peripherals used. A MCP3008 ADC chip is also implemented in order to measure resistance on the servo in the form of voltage spikes. This allows the device to halt the servo only when the deadbolt has been fully extended or retracted.
The following images display the assembled hardware module.
Device Software
This state diagram illustrates the function of the on-device software running on the Raspberry Pi.
This state diagram is sectioned into threads, labeled in red. Each thread runs simultaneously, however some threads will place a mutex lock on the processes of others at times so they do not conflict with one another.
Threads are shown communicating with the Firebase database.
-
The Facial Recognition & Capture Thread is responsible for taking an image every 0.5 seconds and processing it to discern whether or not there are authorized faces within it. If there is an authorized face, the door will unlock and an image and name of the user will be sent to Firebase. If the user has specified a delayUnlock, the lock will simply open for any face while there is delayTime remaining. This thread is also responsible for taking an image any time the user requests a LiveCapture, in which case an image will be uploaded to Firebase as LiveCapture.jpg, where it will be downloaded and displayed by the mobile app.
-
The Unlock Thread is responsible for checking whether the user has requested a remote unlock, in which case the door will unlock.
-
The Delayed Unlock Thread allows a user to specify an interval of time in which the door will open for any face. This is useful when a user is expecting company or hosting a gathering where very few of the guests will be registered users.
-
The Encode New Face Thread is responsible for receiving the images of new users from the app and encoding that image for recognition by the RPi. Encodings are dynamically added to a dictionary array.
-
The Door Sense Thread simply detects if the door has been opened and then waits to re-lock it once closed. This handles the case when a user manually unlocks the door from inside while exiting.
Database
Google Firebase was used as the cloud data transfer service for this project. Here one can see the data structure for the Firebase database that allows for communication between the Android app and Raspberry Pi. Each variable is either read or written to in order to transfer information.
For example, the "Unlock" variable is set to "True" when a user requests to unlock the door from the mobile app. The variable is then immediately reset to "False" by the Raspberry Pi once it has been read, and the door unlocked.
Similarly, the "DelayUnlock" variable is set to "True" by the mobile app when a user requests to have the door remain unlocked for a set period of time indicated by the "delayTime" variable. During this time the door will unlock for any face, authorized or not. The RPi reads this variable and immediately resets the variable to "False" while also reading the "delayTime" variable. The RPi then updates the "timeRemaining" variable every minute until time has elapsed, allowing a user to see the time remaining from their mobile app.
There also exists two image folders in Firebase Storage from which the RPi and Android app push and pull images of users. There is also a third file that stores the LiveCapture.jpg image displaying the current view from the lock.
Mobile Application
This image is the Home Screen of the mobile application. Here the user may select any of the available features, including Remote Unlock, Live Capture, Fixed Unlock, Add Face, or History.
By clicking the About page, the user may view information describing each of these features, however they are described here for your viewing:
Unlock: The Unlock feature allows a user to remotely unlock their AuraLock from anywhere in the world, as long as there is an internet connection.
Live Capture: This feature allows a user to retrieve an image capture displaying what the AuraLock camera sees at that moment.
Fixed Unlock: This feature is described as the "DelayUnlock" above. This feature is what allows a user to specify a period of time during which anyone (authorized or not) is able to unlock the door using facial recognition. This feature would be used in the situation of parties or gatherings where many people would be entering that are not registered users.
Add Face: The Add Face feature is how a user may add a new face to be encoded by the Raspberry Pi.
History: The history feature allows a user to view the most recent unlock made. This includes an image of who unlocked the door, the name of who unlocked the door, and the time at which this unlock was made.
Some of these features are illustrated in GIFs below.
Complete Project
The following images display the completed prototype installed in the simulated door. This should allow one to gain a better understanding of how the hardware is situated in the frame of the door, rather than the door itself.
Final Demonstration
Below you can see a complete demonstration of the Remote Unlock feature. The user selects the Unlock button on the Mobile Application, the Firebase variable "Unlock" is updated, and the door unlocks for 5 seconds. Since the door was not opened, it simply re-locks after the 5 seconds has passed.