Mercedes-Benz heralds a new era for the user interface with human-like virtual assistant powered by generative AI
- MBUX Virtual Assistant transforms the car-driver relationship with natural conversation, paving the way for a hyper-personalized user experience
- MBUX Surround Navigation seamlessly marries route guidance and driving assistance in real time, brought to life by powerful 3D graphics made with Unity's game engine
- Constant expansion of 3rd-party app portfolio brings customers even wider choice of entertainment, productivity and more
At CES 2024, Mercedes-Benz is showcasing a raft of developments that define its vision for the hyper-personalized user experience of the future – in-car and beyond. Headlining those is the new integrated MBUX Virtual Assistant. It uses advanced software and generative AI to create an even more natural and intuitive relationship with the car, with proactive support that makes the customer's life more convenient. This game-changing development takes the 'Hey Mercedes' voice assistant into a whole new visual dimension with Unity's high-resolution game-engine graphics. Running on the Mercedes-Benz Operating System (MB.OS) developed in-house, its rollout starts with the vehicles on the forthcoming MMA platform (Mercedes-Benz Modular Architecture). The Concept CLA Class, which celebrates its North American premiere at CES 2024, is based on this platform and likewise provides a preview of MB.OS.
"Through advances in artificial intelligence, tomorrow's Mercedes-Benz will know its driver like never before. It will enhance and complement our customers' lives – in their cars and in other areas too. We have already made strong progress on this journey. The latest proof points on show at CES provide an exciting glimpse of what lies ahead."
Markus Schäfer, Member of the Board of Management at Mercedes-Benz Group AG, Chief Technology Officer
MBUX Virtual Assistant makes life easier with natural interaction and proactive intelligence
The class-defining MBUX Virtual Assistant running on MB.OS is the most human-like interface with a Mercedes-Benz yet. Combining the intelligent systems of MBUX into a single entity, it presents a new face to the customer with natural and empathetic interaction. The MBUX Virtual Assistant is a further development of the system first showcased in the VISION EQXX. It uses generative AI and proactive intelligence to make life as easy, convenient and comfortable as possible. For instance, it can offer helpful suggestions based on learned behavior and situational context.
"The Mercedes-Benz user experience of tomorrow will be hyper-personalized. With generative AI, our MBUX Virtual Assistant brings more trust and empathy to the relationship between car and driver. Thanks to our MB.OS chip-to-cloud architecture, our future vehicles will provide customers with exactly what they need when they need it."
Magnus Östberg, Chief Software Officer, Mercedes-Benz AG
Four characteristics ensure the assistant is always in tune with customer needs and preferences
Four 'personality traits' underpin the MBUX Virtual Assistant – Natural, Predictive, Personal and Empathetic – blended to create a fully rounded whole. The underlying principle of each personality trait is to offer seamless interaction that leverages the best of voice and graphic interfaces. The customer can choose to speak with the assistant with or without using the keyword 'Hey Mercedes'. Being predictive, the assistant can also offer proactive suggestions and routines based on situational context, learned behavior and using generative AI. At the same time, a more emotional neural voice that can express different speaking styles generates a sense of empathy. To ensure it is always a very personal assistant, the driver can tailor it to their own specific preferences via a range of options in a central hub.
Visual communication made more intuitive and emotional through powerful 3D graphics
The holistic approach pairs more natural dialogue with visual feedback through advanced 3D graphics from the Unity game engine. In the new MBUX Virtual Assistant, a 'living' star avatar uses animations to express different moods and states of being. Emotions range from calm to excited and even sensitive. Further animations indicate whether the assistant is speaking, listening, thinking, making a suggestion or providing a warning. Movement, brightness, intensity and color interact seamlessly to communicate intuitively with the driver.
MBUX Surround Navigation marries assistance with route guidance for a seamless experience
Integrated visual communication also enters a new dimension with MBUX Surround Navigation. Facilitated by the domain integration of MB.OS, this advanced feature pairs a real-world view of the car's surroundings with its reactions to it in real time. The driver benefits from significantly enhanced situational awareness by being able to 'see what the car sees'. The seamless incorporation of information into the driver display using 3D graphics from the Unity game engine provides a highly intuitive overview. It offers everything the driver needs to know at a single glance. For instance, it shows the type of traffic that is ahead by indicating another car, van, truck or even a cyclist. It also shows pedestrians close to the roadside and other potential hazards and superimposes route guidance into a realistic representation of the surroundings – particularly helpful in busy urban environments. As a result, the customer will be able to clearly see where exactly their next turn will take them.
More productivity, entertainment and convenience with new 3rd-party apps and content1
Over-the-air (OTA) updates ensure customers have access to the latest content and features tailored to their wishes and regional preferences. The new E-Class, for example, is already delivering these benefits – its infotainment domain is a precursor of MB.OS technology. Starting Q1 2024, productivity in the E-Class will be further elevated through the addition of the new Meetings app, where customers can participate in TEAMS meetings. This, paired with existing offerings with Zoom and WebEx, covers all market leader for video conferencing. The extensive update also includes the addition of further apps for everyday convenience. These include Booking.com, getAbstract, Just Eat Takeaway.com and more.2 In entertainment, there are plans to introduce cloud-based retro gaming from Antstream Arcade and further immersive experiences.
Video streaming offering to be enhanced with Sony Pictures Entertainment's RIDEVU3
In an exciting development, the video streaming offering will be enhanced by an integration of Sony Pictures Entertainment's (SPE) cutting-edge RIDEVU service. RIDEVU promises to elevate the new vehicle entertainment experience for Mercedes-Benz customers, granting them access to a diverse array of video entertainment directly within their vehicles. RIDEVU provides users with advanced controls and features designed for the car. It includes a curated library of films available for streaming, enriching the Mercedes-Benz entertainment bundle.
Hyper-personalization through digital art with MBUX Collectibles and more
MBUX Collectibles is Mercedes-Benz's first in-car app to showcase digital art and collectibles based on NFTs. It enables the creation of a private art gallery by connecting the customer's personal NFT wallet to the user interface. Users can also browse a digital art exhibition curated by Mercedes-Benz. On display at CES are NFT collections from Mercedes-Benz NXT within the app. The collection Mercedes-Benz NXT Icons Era 1 'Luxury' is the first in a series of seven exploring the rich Mercedes-Benz design heritage. At CES, there is also a first sneak peak of Era 2 'Technology', which is yet to be launched. And as an homage to last year's CES, visitors can enjoy a first glimpse of the Mercedes-Benz NXT Superdackel NFT. A limited edition of 1000 Superdackel NFTs by Mercedes-Benz NXT will drop soon.
Concept CLA Class showcases MB.OS as the foundation for the hyper-personalized Mercedes-Benz
The first vehicles to run on MB.OS will be the family based on the forthcoming MMA platform. The Concept CLA Class, which celebrates its North American premiere at CES 2024, offers a close-to-production insight into the future range of four vehicles. To stress the central role played by MB.OS in shaping the customer experience, the interior of the show car offers a striking visualization of it. The complex multi-faceted architecture is represented by one of its many hardware components, the high-performance water-cooled chip from collaboration partner NVIDIA. The advanced three-display MBUX Superscreen hints at the role of the Mercedes-Benz of the future as a hyper-personalized part of the customer's digital life. MB.OS provides the foundation for achieving this. It comprises four domains – Infotainment, Automated Driving, Body & Comfort and Driving & Charging – and the connectivity module linked to the Mercedes-Benz Intelligent Cloud. The chip-to-cloud architecture ensures deep integration into the vehicle and full control of all sensors and actuators. Furthermore, by decoupling the software and hardware innovation cycles, the company can ensure its vehicles are constantly up to date.
# # #
About Mercedes Benz
Mercedes-Benz USA (MBUSA), headquartered in Atlanta, is responsible for the distribution, marketing and customer service for all Mercedes-Benz products in the United States. MBUSA offers drivers the most diverse lineup in the luxury segment with 18 model lines ranging from the sporty GLA SUV to the flagship S-Class and the dynamic all-electric vehicles from Mercedes-Benz. MBUSA is also responsible for Mercedes-Benz Vans in the U.S. More information on MBUSA and its products can be found at www.mbusa.com and www.mbvans.com.
- The use of the Digital Extras requires the permanent acceptance of the Terms of Use for Digital Extras and the Mercedes me ID Service Terms in their applicable versions, the permanent pairing of the vehicle with the Mercedes me user account and - if applicable - the activation of the Digital Extras. Upon expiry of the term, the Digital Extras can be renewed for a fee in the Mercedes me Store provided they are still offered for the corresponding vehicle at that point. In addition, for the use of some Digital Extras there may be further usage requirements or limitations specifically separate customer-owned contracts with third parties (e. g., streaming, charging, conclusion of a data contract covering comfort data volume), selected third-party products (e.g. smartphone, smartwatch) or they are only available for specific customer groups. Alternatively, to comfort data volume, depending on the generation of your multimedia system, data volume must be available via Internet in the car or, where possible, your own data volume will need to be used via tethering. Information on personal data processed for the use of Digital Extras may be found in the Mercedes me connect privacy policy.
- Specific app availability could vary by market.
- Specific RIDEVU offering in U.S. to be confirmed closer to launch
