Isn’t it fascinating if shoppers pay the bill at time of checkout by just presenting a face and no cash or card required? Also, what if digital billboards show advertising content based on shopper demographics and show loyalty programs and promotional offers based on shopper’s past purchase. Further, it is astonishing for retailers if they can analyse shopper’s mood and facial expressions at different SKUs, automate shopping experience with limited workforce, and prevent theft and shoplifting by increasing in-store security.
Let us see how facial recognition technology can increase operational efficiency for retailers and elevate in-store customer experience.
1. Self-Checkout with Facial Recognition based Payments
American Express Global Customer Service study, states that customers on an average have to wait for 15 minutes before they get served. The study also states that customers are more loyal where they experience faster checkout and less waiting time. Payment through various payment methods such as cash, card or QR code requires different POS terminal setup and higher staff involvement. Face recognition is a more convenient payment method as it eliminates various hardware.
Shoppers from retail stores can easily pay by presenting their face at tablet-like devices or at kiosks. It is easier, faster, accurate and most importantly secure.
Let us look at how facial recognition payment system works:
a. Scan Items and Present Face:
When shoppers are ready for checkout, they visit a self-pay kiosk where a tablet-like device is placed. Shoppers can scan the purchased items to add to a cart for payment. Digital carts are also now available in many retail stores where selected items from the shelf are automatically added to digital carts. After adding items to the cart, shoppers show their faces to the device. The system can discover and spot human images or video frames. It uses procedures such as convolutional neural networks (CNNs), Haar cascades, and deep learning methods to discover facial landmarks and segregate faces from the background or other things.
b. Characteristics Extraction
Whenever the system identifies a face, all the key characteristics such as the parting between the eyes, the structure of the nose, and the arch of the jawline get drawn. These characteristics are then used for creating facial signatures or faceprints.
C. Authenticate
Thanks to Artificial Intelligence which can identify and recognize faces from millions of user databases. Validating facial resemblance with the enrolled user’s faceprint is a primary part of authentication. Computer Vision-based Machine Learning algorithm that can find the shopper’s face from the entire database.
d. Approve
The system decides the person’s facial recognition depending on similar results and prearranged threshold. The user is given the way in or allowed to complete the transaction if the similarity scores match up to the requirement for authentication. After successful authentication, facial recognition devices will show payment and user details to approve.
A couple of companies such as 7-Eleven in Japan have started trials with facial recognition payments and Alibaba has also rolled out a “Smile to Pay” pilot project for facial recognition payments in China.
Retailers may create a private database by enrolling customers during the KYC process or they can use a public database owned by the government. With different permissions and various API methods, facial data can be accessed from government databases for authentication.
2. Enhance Customer Experience with Personalized and Customer Centric Offering
For personalised experience, facial recognition plays a key role to identify the shoppers based on demographics (age, gender, location etc.) and then present content and offers pertaining to them. After facial identification of shoppers, a deep learning algorithm presents various purchasing recommendations on digital signage and shelf based on past purchases.
CaliBurger, a Californian burger chain of restaurants has installed kiosks with face recognition technology. Customers scan their face while visiting the store and with the help of Facial Recognition, the kiosk screen will display their loyalty program with past and favourite purchases. This will help the retail chain to speed up the ordering process with limited workforce requirements.
With past purchase information, retailers can further send different personalised deals to customers via SMS, email or push notifications on mobile application. This also prevents retailers to bombard all the deals to customers in which they are never interested.
3. Improve In-store Customer Service with Tailored Assistance
One of the surveys states that 91% of the non-engaged customers simply leave the store without complaining. Facial recognition with embedded machine learning algorithms helps retailers to understand different moods and expressions of shoppers in response to various discounts and promotional offers. Facial expressions analytics such as excited, depressed, surprised, and happy guide retailers to design effective promotional campaigns.
Deep learning analyses when shoppers check the same item multiple times and linger around it. That indicates that the shopper has interest in purchasing that but needs some assistance to decide. With such video analytics, mobile apps in staff’s mobile or dashboard at reception notifies staff to attend to shoppers immediately and provide required assistance.
Walmart has patented a technology, which identifies facial expressions of shoppers from checkout lines with AI based algorithms to measure the service satisfaction index. This has helped them to improve the satisfaction level of shopper and in-store experience.
4. Theft and Shoplifting Prevention
Shoplifting has been a continuous and major issue in organized retail. The National Retail Federation (NRF) states that theft, fraud, and losses from retail has increased to $50.6 billion in 2018, from $46.8 billion in 2017. Retailers are also finding innovative ways to identify and stop shoplifters from stealing in the store. Facial recognition can be a boon to the retail industry to discover shoplifters from stores in real-time. It identifies previous offenders live and matches it with the shoplifter database. It notifies the security team when a match is found to take necessary action.
Let us see how facial recognition identifies face of shoplifter:
- Facial image is captured from live video stream
- Facial features are extracted from the image
- Image is converted into grayscale and cropped with face
- Converted into template that is used by the algorithm to search larger database
- Compare the database with template and notified regarding fraud
5. Workforce Productivity- Who, Where and When
With facial recognition deployed into the store, it is easier for retailers to manage workforce productivity. It stores employee attendance records, provides secure access to employees during and after working hours, restricts unauthenticated access to critical and restricted areas in the store, and enables quicker check-in and check-outs. It also improves staff productivity by monitoring real-time interactions and record their engagement with shoppers, which will help in identifying improvement areas to streamline the process.
Important Face Recognition System Components:
For facial recognition algorithms to yield precise and useful outcomes, they need to incorporate many crucial elements that function in tandem.
1. Obtaining Images: In this stage, facial pictures or video frames are collected from various devices, such as mobile phones, cameras, and surveillance footage.
2. Preliminary processing: Preliminary processing aims to standardize and enhance the quality of raw pictures for optimal feature extraction. Methods like filtration, alignment, and normalization are employed at this point.
3. Face Recognition: The face recognition module locates and identifies faces in previously processed photos using algorithms. It ensures that only relevant face regions are subjected to further processing to extract features.
4. Feature Extraction: The identified face is processed in this step to extract distinguishing characteristics and facial landmarks. The face signature that is utilized for comparison is built on these elements.
5. Face Database: The facial recognition technology matches pre-registered faceprints of authorized users or people of interest with a database for identification purposes.
6. Facial matching: The resulting facial signature is compared with the faceprints in the database by the matching algorithm to find potential matches.
7. Taking Decisions: To decide if the face matches or not, the algorithm uses a threshold applied to the similarity score.
8. User Interface: By providing feedback and authentication results, the user interface facilitates communication between the system and humans.
By including these important components facial recognition can discover and validate people correctly, making a path for faster and more secure facial recognition payment systems.
Wrapping up
Retail industry has always surprised shoppers with innovative technologies to enhance in-store customer experience. Be it shopping with AR, smart mirrors for virtual trial, digital shelves, cashier less checkouts and many more. We at eInfochips helps our retail customers to expand their technology landscape with product engineering services. We have executed various technology transformation initiatives for our clients like connected retail stores, video management software development, 360° camera design, video analytics and many others. We have comprehensive expertise in various areas such as Device Engineering, Digital Engineering and Quality Engineering.
Get in touch with us to know more about how eInfochips can help the retail spectrum in Facial Recognition, Deep Learning and Artificial Intelligence space.
https://linkedin.com/pulse/facial-recognition-payment-system-future-secure-patrick-mutabazi/
