ABOUT

What if digital technology can understand emotions like humans can?

Computers and AI systems may be smart in terms of IQ, but are lacking in
EQ – or emotional intelligence. When we interact online with campaigns,
tutorials, and most other digital content, we are in an emotionally blind
world in which we miss natural human feedback.

A-dapt has launched an Emotion Interface Design format called A-dapt Vision that can analyse people in real-time and adapts digital content such as video based upon human factors such as attention, positivity, and learning engagement. It is powered by a unique emotion analysis algorithm that uses the device camera as a sensor, working safely and securely in the browser. The technology is not to be confused with facial recognition and no biometric data is sent to the server for analysis.

Concept

The concept for the A-dapt Vision project was to develop an emotionally responsive and innovative user interface responsive design solution to demonstrate key features and possible applications of our innovation. We wanted to give creators and innovators the ability to understand the format quickly, then pick and run with it for their own creative projects.

A-dapt Vision enhances the interaction between digital content and a user
by allowing digital content to respond to human factors in real time, such
as attention and emotions, enabling new levels of engagement.

We determined the following UI overlays would enable people to explore what they could do and envision with the format.

• Basic Graph: The dot position varies based upon low to high control and obstructive to conductive; text overlay shows predominant value with two key emotional affects e.g. Impressed / Hopeful; and with Attention, Positivity, Energy, Mood in UI bar.
• Advanced graph: Adds demographic estimates (age / gender); Head Pose; Up to 40 Features.
• Emotion levels: Adds percentage readings for the six emotional states plus neutral (Anger, Disgust, Joy, Fear, Surprise, Sadness, Neutral).
• Emoji: Generates emoji to mirror the main emotion.
• Generative AI: Demonstrates how emotional responses can interface with generative AI systems such as ChatGPT by gauging reaction to a joke.
• Positive Impression: Displays heart animation when user shows positive reaction.
• Sample Report: Manifests sample data in a PDF from a time limited 30 second session.
• New features such as ‘Emotion Mirror’ and Large Language Model (LLM) are under development.

Additionally, we wanted to show how users could upload any video for analysis and allow them to paste in a YouTube link and see their reaction to video content. No other Emotion AI UI overlays the biometric analysis over a real-time video stream in a secure web-browser.

The animating dot shows the user where ‘they are’ in real-time and how they can move the dot to achieve higher states of control, depending upon their selected view.

Execution

A-dapt Vision face analysis works in the browser by using the device camera as sensor. The AI is a 750Kb Deep Neural Network auto download which is highly responsive with much lower latency, compared to server-side technologies. Not to be confused with facial recognition, no biometric data is sent to the server, and it does not allow for racial profiling.

A-dapt Vision has the following features:

• Uses the device camera as sensor (enabled by the user)
• Built in Native JavaScript
• Works in browser / browser based Deep Neural Network
• No App required
• Low latency / highly responsive
• Attention, positivity, pose, energy, motivation
• Demographic estimates
• 6 emotions + neutral
• 100+ affects / expressions / features
• No facial recognition / no racial profiling
• No biometric data sent to server
• Anonymised data analysis & reports
• Can be integrated into any website, App digital video or digital campaign

A-dapt Vision incorporates a unique combination of technical features.

• Firstly, the size of the Deep Neural Network (DNN) is significantly smaller than other DNN-based solutions for face analysis. it is less than 1MB in size, making it highly efficient and capable of running on a wide range of devices, including mobile phones and laptops. This is a significant advantage over other solutions that require more powerful hardware to run, or that are limited to running on server-side infrastructure.
• Secondly, it is highly accurate, achieving similar results to competitors who use server-side algorithms. This is due in part to the proprietary architecture of the DNN, which is multi-task and has a basic part in common to all face analysis modules. This allows for reduced processing time and memory footprint, which in turn results in faster and more efficient analysis of facial expressions and features.
• Thirdly is its ability to analyse over 100 facial expressions and features, which is a higher number than many other solutions. This means that it can detect a wider range of emotions and expressions, which is useful in applications such as education, marketing, entertainment, and healthcare.
• Fourthly, it is designed specifically for analysing facial expressions and features in real-time directly on the user’s browser, which is another unique feature. This means that the data is processed locally on the user's device, rather than being sent to a server for analysis. This allows for faster processing times and eliminates the need for costly server infrastructure. Moreover, the real-time analysis allows for the detection of fast-changing expressions, which is useful in applications such as user experience testing and market research.

Overall, the DNN is unique in its combination of small size, high accuracy, multi-task architecture, wide range of facial expressions and features analysed, and real-time analysis directly on the user’s browser. These features make a powerful and efficient tool for analysing facial expressions and features in real-time, with applications in a wide range of industries.

Additionally, these two main technological barriers have been addressed by A-dapt Vision.

1. Latency. A-dapt Vision works in the browser on the user’s device. Because it uses the device CPU without having to be engineered into the CPU, this provides an Emotion AI breakthrough in terms of low latency, performance, and scalability. It is also good for the environment as it does not require a server to be available 24/7 using around 8% of the device CPU as and when it is required.

2. Safety. Because A-dapt Vision Emotion AI works on the user’s device no biometric data or face images need to be sent to a server for analysis and is therefore not to be confused with facial recognition. A-dapt Vision software treats the humans as anonymous and adapts to them, just like in real life when humans meet for the first time and don’t require each other’s personal data to interact and adapt.

A-dapt has implemented additional measures to communicate the safety and privacy features of the technology to users. This involves providing more detailed information about how the technology works, including the fact that it is client-side and does not send any private data to servers. Adapt have developed ways to train users about the technology and its features, helping to build trust and confidence in the product.

Results

The simplest way to assess the innovation is to see A-dapt Vision working on yourself.

The innovation incorporates both Artificial Intelligence (AI) and Emotion Intelligence (EI). AI is the process of programming a computer to mimic human intelligence - to create a machine that can learn like a human. Emotion Intelligence is the ability to understand and manage emotions. It is more natural, nuanced and innate and is better at managing human relationships. We often hear that EI is more important than IQ when it comes to being successful in life.

Adaptive Media France was commissioned by the French Government Agency France Travail to develop the technology across 20 campuses in the Nice region of France to help employability outcomes for young unemployed.

A-dapt is developing innovations in healthcare, including remote cognitive therapy assessment videos that can be used to detect early signs of dementia.

A-dapt is currently working with two leading agency networks on campaigns using A-dapt Vision AI.


A-dapt Vision delivers a new paradigm in interface design, for the first time enabling new methods for emotionally aware screen-based learning, marketing and entertainment.

A-dapt Vision was nominated for the SXSW Innovation Awards 2024 for Interface Design.

The future of interface design is emotional intelligence!