In the Shopic user experience, shoppers clip the Shop-E device onto a standard shopping cart handle, transforming it into a smart cart. The smart cart automatically identifies each item as its placed in the cart, and displays the item and price on the Shop-E screen. The technology that recognizes items as they are placed in the cart is a type of artificial intelligence (AI) called computer vision.
Computer vision's roots date back to the 1950s and '60s. In 1966, MIT scientists linked a camera to a computer and tried to get the computer to describe what it saw. While those efforts met limited successes, they established the groundwork that computer vision is based on today. In 2012, a team from the University of Toronto at the ImageNet Large Scale Visual Recognition Challenge entered a deep neural network called AlexNet that correctly identified objects 83% of the time, 10% better than any efforts to that date.
In the decade since, computer vision has advanced significantly. Computers with cameras can correctly identify objects and people with 99% accuracy, often recognizing items faster than people can.
Computer vision is used by autonomous vehicles to recognize their surroundings. Healthcare professionals use it to automate tasks like detecting cancerous skin moles, and security personnel use it for facial recognition. And of course, computer vision is used in the supermarket.
Last June, Amazon implemented computer vision in its Amazon Fresh grocery stores. Amazon outfitted its stores with cameras and sensors. The Computer Vision identified every item that customers placed in their carts, and when customers finished shopping, they could automatically pay and just walk out of the store.
In addition to the shopper’s experience, wall-to-wall computer vision provided Amazon with inventory data that they could use to optimize their ordering and ensure that there were always enough products on the shelf.
This was the first implementation of Computer Vision in a supermarket, and while the technology worked, the implementation was rather inelegant. The surveillance infrastructure in the store followed consumers' every step, and consumers had no idea what was being done with the data. This raised concerns about privacy and is just one more example of what Business Insider calls "Amazon’s Empire of Surveillance.”
Shopic takes a much smaller, more efficient approach to computer vision. Rather than outfitting entire stores with cameras and sensors, Shopic’s computer vision is focused solely on the shopping cart. The Shop-E is currently equipped with two cameras and monitors everything that goes in and out of the cart.
Rather than outfitting entire stores with cameras and sensors, Shopic’s computer vision is focused solely on the shopping cart.
Customers place items in their cart the same way they have always done, and computer vision takes care of everything. It identifies the product and adds it to the running list of products on the device’s screen. Computer vision can even capture items that are tossed into the cart from across the aisle (not that we advocate throwing merchandise that is heavier than marshmallows around the store).
Future versions of the Shop-E will include two additional cameras placed on each side of the device. These cameras will scan the shelves, identify products that need to be restocked, and alert store employees when items are sitting on the wrong shelves.
Like the cameras facing the cart, these cameras aren't being used to capture consumer data. They eliminate many of the privacy concerns that are inherent in the Amazon system.
Unlike the AlexNet, which accurately identified 83% of items back in 2012, Shopic’s computer vision accurately identifies 98.5% of all items that it has been trained on. Furthermore, it detects when an item is added or removed from the cart with 99.4% accuracy.
According to Eran Kravitz, Shopic’s CTO and co-founder, “We need to differentiate between thousands of items, some of which are very similar to one another in an environment with many edge cases. The system is incredibly accurate and user-friendly – it kind of feels like magic. Our algorithms recognize the item itself based on its visual features; all shoppers have to do is place them in the cart as they usually would.”
When new, untrained items are added to the cart, the system alerts customers, who scan a bar code on the device. Scanning and adding the product into the cart seamlessly train the system to recognize new products.
As a result of this high degree of accuracy, supermarkets can trust the carts with unsupervised checkout and payment tasks.
Computer vision is ideal for supermarkets. Customers can place their groceries into bags within their cart. When they finish shopping, they can pay directly on the cart, remove the Shop-E device from their cart, and bring their groceries to the parking lot.
It’s a vastly improved experience over loading a shopping cart, waiting in line, unloading it at checkout, bagging all the items, and then placing the bags back in the cart.
