ABOUT

We have built a unique machine learning touch table for Charlotte Tilbury that uses computer vision, deep learning algorithms, and machine learning techniques to detect and classify products placed on the table in real-time.



The machine learning touch table is a prototype that can easily be expanded to detect many other products, making it a versatile technology that can be applied to a wide range of products, from makeup to perfumes and beyond. With the ability to detect groups of products and the potential moving forwards for one-camera detection, this technology has the potential to transform the way we interact with physical products and the information that accompanies them.



To achieve this initial prototype, we have used three compact and cost-effective cameras to view and detect products in three specific areas of the screen, with each focused on one area. However, through experimentation during the prototype phase, we were also able to reduce these three cameras down to just one single camera with a wide angle view which also uses IR depth to understand the customer's interactions and allow it to work in many different environments without the background of the environment having such an impact, subject to further development.



Our machine learning model is built using a technique called supervised learning. We use machine learning algorithms to enable the system to recognize and classify products placed on the table. Specifically, we use a deep learning algorithm called YOLO (You Only Look Once) to detect the products in the live camera feed. YOLO is a state-of-the-art object detection algorithm that can process images in real-time and provides accurate results.



To help assist in the training of our YOLO model, we used Roboflow, a platform that simplifies the process of creating and managing datasets for computer vision. We used Roboflow to generate bounding boxes for our product images and to create variations of those images that were used to train the model. This helps to improve the accuracy and generalization of the model.



Once our YOLO model has identified a product in the camera feed, it outputs the result via OSC (Open Sound Control) in Python to a front-end GUI that we have created in Touchdesigner. This GUI displays relevant information about the product, such as animations, text, videos, and images.



The customer interacts with the touch table through the front-end GUI, using a touch interface to browse through the product information and multimedia content.



Overall, the machine learning touch table is a cutting-edge technology that brings together the latest advances in computer vision, machine learning, and deep learning to provide customers with an interactive and informative experience. It is a one of a kind prototype that can easily be expanded to detect many other products, making it a versatile and powerful tool for a wide range of industries.