Virtual Reality is now. If most of the advertising industry hasn’t jumped on the VR ship yet, you can bet we will soon – or at least, as soon as those incredibly expensive headsets become a bit more affordable (or attractive?) for the general public. Attractiveness is especially important – at the moment, Virtual Reality is mostly used in games, and not everyone has an interest in gaming or immersive experiences… for now.
That is where Augmented Reality kicks in. When it comes to approaching the average user, AR is way easier to tackle. All you need is a smartphone with a camera and powerful enough hardware to render 3D shapes in a virtual space, and there you have it – Pokémon GO, Google Lens, even a Google Maps integration. AR is proving particularly useful in the education and heritage fields too, with museums and education apps using the power of AR to explain difficult concepts in a 3D space.
However, there is something on the horizon which seems to promise much more than Augmented Reality can already offer. It is the next step in immersiveness and experience – the future of consumer relationships, big data and technology, all at the same time.
What in the world is Mixed Reality?
You may not be entirely familiar with the term Mixed Reality, and it makes sense. It is something especially discussed in the realm of sci-fi and slightly niche tech, relating to the use of technological devices to augment the actual, physical space where we walk and live. Where Virtual Reality requires a bulky headset to immerse you in a virtual space, and where Augmented Reality requires the intermission of a camera to visualise shapes through a screen, Mixed Reality is the perfect blend of both, and it would require the use of smart glasses, smart contact lenses or (in a far, far away future) even brain implants.
The close proximity between your eye and the screen leads to a perfect blend of reality and data, one that big tech firms aspire to make as seamless as possible. In other words, there is a discernible difference between having to open an app dedicated entirely to AR, and wearing a pair of glasses designed specifically for Mixed Reality. The main difference with VR headsets is that your entire body and perception is not projected into a virtual space; rather, it is the virtual space that is overlapped with your perception of reality.
That is all cool and interesting, sure. How does this relate to the creative industries?
Image credit: Neil Williams
The endless potential of MR
So far, smart glasses have been either incredibly bulky, incredibly specialised or incredibly expensive. The Microsoft Hololens are the state of the art in Mixed Reality, but they are priced at $3,500 (and are also a proper headset rather than easy, every-day wearable glasses); the Bose Frames are (obviously) focused on music; whereas the Amazon Echo Frames offer a degree of integration with the Alexa voice assistant.
In other words, so far smart glasses have not really made it in the hands of the average and mainstream consumers. They are perceived as an experimental piece of technology, and there is no way they could replace smartphones – in some cases, they can’t even complement them.
There are exceptions, of course. The Vuzix Blade smart glasses are rather unassuming, feature a powerful camera and offer good AR features – but still, they don’t seem to attract the average consumer. Things may change with the release of the Apple Glasses, set to launch in 2022 or 2023 (not that far ahead then!). With Apple’s reach and leverage on its gigantic user base, and a rumoured integration with iPhone and iOS, the Apple Glasses may be just what sci-fi writers have been envisioning for ages, and they may kickstart a new era of consumer experience and smart devices, in advertising and beyond.
There used to be a time when computers and the cyber space were perceived as pure science fiction. Then technology caught up with expectations, VR and AR came about, and although we are still to see their full potential, the present is already increasingly digitised and relying on big data to provide meaningful customer experiences. Mixed Reality has the potential to go the extra mile, sending us straight into an episode of Black Mirror – hopefully without the cynical endings.
All the trends and predictions we are seeing now about VR and AR can easily be applied to Mixed Reality as well. Customers will be able to attend immersive online events, experience improved customer care and immersive marketing. Advertisers will find new, creative ways to offer unparalleled storytelling experiences, and brands can truly aim to integrate with the lifestyle of their target audience, perhaps through the use of first-party people-based targeting.
Forget Behavioural or Contextual; the future of marketing flows through the potential of Mixed Reality and an age of smart glasses/lenses. And I’m personally, incredibly excited to see how it will pan out.
So far I’ve been rather abstract in discussing the power of MR. How would Mixed Reality actually work?
A day in Mixed Reality
Image credit: AKQA for Discord
Smarter Voice Assistants
You are employed in a production or creative/digital agency of the future. You work from home most days, the hybrid work model now being the norm across the entire industry. Today is one of your days in the office. You put on your smart glasses and they instantly boot up, thanks to a pair of skin sensors on the temple tips. Siri, Alexa or any other voice assistant wishes you a good day, showing you the weather forecast and discussing your reminders for the day. This could happen through a pair of bluetooth headphones, earbuds or two speakers embedded into the eyeglasses themselves.
You have been complaining about this brand of cereals for about a week now. You really want a change in your breakfast habits – or perhaps you’ve already tried that, and it just didn’t click. Your trusted voice assistant, with your consensus, has been listening to that sentiment and is now offering you to purchase a new brand, with a small thumbnail notification on your right lens. You gesture it away for now, and through a pair of special rings on your right hand, the smart glasses understand your command and execute straight away.
Personalised Digital Outdoors
It’s getting late. You ask your trusted voice assistant to start the latest episode of your favourite podcast and you are on your way. As you walk, Netflix tells you there’s been a problem with PayPal and asks you to update your payment method. You do that easily, by hooking up your Apple/Android Pay card to the app. This takes you no more than a few flicks of your fingers as you walk towards the bus stop.
Walking around town, you are attracted by a number of virtual, animated billboards begging you for your attention. Coincidentally, you’ve been wanting to buy a new vacuum cleaner for a while, and there is one billboard advertising just that. You focus on it to see a spectacle of colours and shapes exploding all around you, a unique experience tailored just for your interest.
You see a bunch of these virtual billboards as you walk around town, while on the bus and before reaching your office. You watch a few, ignore most others. Mind, you could remove your glasses or turn off the AR features whenever – it’s just more entertaining not to do so.
Your first commitment of the day is the usual weekly meeting: an overview of the agency’s live projects and some monthly new business opportunities. Some members of the team work from another town entirely, so they can never be physically present – but this is not a problem for your smart glasses. The empty seats in the room soon get filled by virtualised 3D avatars, figures with human features and genuine body language that make it easier to read the room.
Remember the struggles of Zoom meetings, during which reading intentions and body language was essentially impossible? Mixed Reality defies the challenges of remote distance by putting a virtual avatar in the room with you. It is still not quite like being in person, and there is a lot of “You’re on mute” still – but it’s a start. You’ve heard that the new model of smart glasses will feature a photorealistic engine and avatar creation anyways. That will put an end to this problem.
Image credit: Wunderman Thompson for BT
Presenting Models, Products, Physical Spaces
You work through the rest of the day, as usual. During your lunch break, your trusted voice assistant reminds you about that brand of cereals and that vacuum cleaner you saw earlier. You decide to check them out. A 3D model of both products appears in front of your eyes, and while you certainly can’t taste the cereals themselves, you are able to read the box and find nutritional information exactly where you’d expect it to be. It reads well, so you add it to your Sainsbury’s cart. The vacuum cleaner model is also on a 1:1 scale, meaning you are able to place it in the space around you and see if it would fit your home.
With Mixed Reality, you’re essentially able to try out a product in full before you even buy it. This makes for more informed decision-making and more personalised targeting, as the collected data from your purchases can then be used to suggest suitable products for you in a much more customised manner. Additionally, advertisers know this very well, so they pour a lot of budget into the 3D models to make them as appealing as possible. Perhaps they even put some catchy tagline or call-to-action in the preview screen.
As the end of your work day approaches, you are called in for another meeting, this time with a freelance 3D modeller that your agency has hired to build product pages. Years ago, the only way you could visualise a 3D model was through a screen, but Mixed Reality enables the freelancer to project the shape in the physical space, making real-time changes and polishing details per your instructions. No matter how many times you’ve seen this, it’s still incredibly fascinating to see an artist build a 3D model in front of your eyes.
X-Ray Data on Video Content
As you walk back home, you decide to turn off the AR features for a while. You’re tired and really not in the mood for an explosion of colourful virtual billboards. Of course, you can’t turn off all of them – some advertisers have paid for premium spaces, meaning some billboards will still remain on and can be visualised through your glasses. You don’t have to watch the whole show, of course – but they are there if you want to.
In fact, you might as well check this one out. There’s this actress you’re sure you’ve seen before, but you’re not entirely certain where. In an instant and with a special gesture, you conjure up the X-Ray screen – something Amazon used to do with Prime Video. It connects to the Internet Movie Database and pulls information on the participating cast, actors, director and writers of the ad. The X-Ray screen also shows you a range of related products, ads and films where you will be able to find the same actress. Some of them you can purchase or rent instantly for later use.
Your mom video-calls you. You’re still walking, so you’re not ready to go full-screen, and push her in a corner. You chat on your way home and she keeps you company while you hop off the bus, enter home and start cooking dinner – all on video, and without the use of a smartphone. Multi-tasking at its best.
When she hangs up, a notification in your left lens informs you that the latest episode of Muppets Now is available on Disney Plus. Without the need for a separate screen (though you can use it if you want to), you open it up and keep it in a corner while cooking. You probably wouldn’t do this with a film, but for a comedy show, it’s good to have some company.
With the smart glasses, Disney Plus, Netflix and every other major platform has enabled X-Ray as well. You’re able to access an overload of information with just a simple gesture. Of course, Disney is also using the app’s ad space to show you related products, such as Disney Store toys, gadgets, blu-rays and the latest offer on Disneyland tickets. All of these are easily accessible with a gesture.
Endlessly Entertained, Never Alone
You live by yourself, so cooking dinner and eating can be a very solitary process. Not with your smart glasses. You have spread a number of virtual monitors around the room that enable you to access podcasts, TV shows, YouTube and other platforms with relative ease, so you’re never alone while walking around the house. Some of these you can keep on your lenses, others you can project on a wall, the floor or the table, so that you are constantly immersed in a virtual environment that suits your needs.
If you want, you can easily open up an AR game to have some colourful fun in the very space of your kitchen. In-game ads will pop-up here and there, as they would, but the experience is overall fun and entertaining. You go to bed relaxed, ready for your next work day.
Revolution of Performance Shows, Experiential & Live Events
Ads, billboards and meetings are not the only things affected by Mixed Reality. Performance shows have been highly revolutionised as well, at least the high-end ones. You probably won’t be able to find beautiful AR and holograms at a live music event in the pub down the road, but a concert? Definitely. A gaming convention? Absolutely.
Event managers and experiential agencies are now able to visualise installations and experiential spaces through the use of smart glasses, and they can easily build models of booths and shows before they even send them to production. Every performance show, every live event, every experience is polished and refined to guarantee customers have the most meaningful time they can. Some do it better than others, of course. Those are the ones you will remember.
Physical retail is probably dead by now, but if it isn’t, it was transformed entirely by the popularisation of Mixed Reality glasses. You can physically visualise clothing on your mirrored image through the use of AR and big data. You can see the appearance of an accessory before the shop assistant grabs it from the stockroom. You can examine the packaging, read the labels, try different colour combinations, all through a seamless, streamlined experience that was not even remotely imaginable before the advent of Mixed Reality.
The world of consumer experience and advertising is changed forever. The only way forward is up.
New metrics, new results, new goals
Clearly the era of Mixed Reality comes with a number of innovations and changes in the marketing and advertising space. Mixed Reality will essentially be a much more powerful storytelling tool for advertisers and agencies all around the world. Every customer interaction will aim to be meaningful and leave an impact, there will be new metrics and KPIs, and the priorities of brands and agencies will be fundamentally different.
While the focus on brand awareness and growth is not going away, Mixed Reality has the potential to shift our entire approach to advertising and introduce more immersive ways to track user activity. Without third-party cookies, Contextual Targeting and People-Based Targeting will take the centre stage. All data will be shared intentionally to build a privacy-first world, one in which consumer consensus is at the core of every brand strategy. One in which building fewer, meaningful relationships with a brand will be more important than establishing a million more.
This is what Mixed Reality may mean for the future of advertising. We’re not too far from at least seeing a glimpse of it come true. It may take a full decade to see such a massive change in society, or perhaps much less; but we can’t deny that the future of advertising is bright, meaningful, powerful and innovative. And I can’t wait to see what happens next.