Forward-looking: The service department can seem like the slowest part of a dealership, especially when it's your car getting worked on. But Mercedes-Benz is infusing its dealerships with AR technology to speed up the diagnosis and repair of tricky and complex issues with its Virtual Remote Support, powered by Microsoft's HoloLens 2 and Dynamics 365 Remote Assist. I visited one of Mercedes' showcases last week to check out the implementation and get some hands-on time with the HoloLens 2.

For customers that bring in a hard-to-solve problem, it was common for Mercedes-Benz to call in a flying doctor, an expert from HQ that would fly in and get hands-on with an issue. That could take days to arrange, not to mention the costs and environmental impacts associated with flying in these technical specialists from all over the country. All the while, the customer is left without their luxury automobile.

While many shoppers go with a Mercedes for the styling and performance, it's the service experience that impacts their future purchases. Long waits could see someone changing brands in the future.

Mercedes is arming all 383 dealerships across the U.S. with this Virtual Remote Assistant, hoping to improve the efficiency of their service bays and keep customers happy.

It all starts with the HoloLens 2, an augmented reality headset with four visible light cameras for head tracking, two infrared cameras for eye tracking, a depth sensor, a five-channel microphone array, and built-in speakers.

// Related Stories

The local technician puts on the HoloLens and can then pull up wiring diagrams, schematics, and other details on the headset, so they can refer to them while they're working on the customer's vehicle.

If the problem needs assistance from one of the experts, they can call them and even provide them a visual feed from his headset. The experts see what the technician does, and their face is broadcast in a small portion of the screen. It's like a futuristic Zoom or Teams meeting call happening in your eyeglasses. Ever helpful, those experts can add arrows, circles, and other notes to help point out important issues.

Using the headset requires little orientation. All you do is put it on and do a short calibration. It follows your eyes and can recognize your hands. You look down and tap your wrist to bring up a menu that allows you to start your work or teleconference. It's projected onto the glass of the headset but looks like it's floating in front of you. You can reach out and interact and manipulate the windows with a variety of gestures. It all feels similar to the movie Minority Report and was pretty intuitive to use.

When I was finished goofing around with the headset, Mercedes-Benz representatives explained that the technology has the potential to completely change the service industry. A career that is typically associated with trades work, can now become more digital and high-tech, attracting a whole new demographic of students and employees.

Mercedes boasts that their cars today have more lines of software code than the Space Shuttle, an F-35 fighter jet, and a pacemaker combined. This much functionality means more complexity. So to chase down and resolve any complex issues, the automaker has turned to the HoloLens. Referencing a wiring diagram or schematic that's projected in 3D space onto the car you're working on, will help cut down on wasted time.

Furthermore, while automakers boast that electric vehicles will feature fewer moving parts and require less maintenance, these advanced vehicles may still have issues, likely with wiring and digital components. Technicians using HoloLens can chase down these issues quicker than those having to go back and forth between the computer and the car.

Mercedes-Benz isn't a newbie to AR. The automaker offers an augmented reality navigation system, which broadcasts navigation instructions on top of a video feed from a front-facing camera, all on the dashboard to make it difficult to get lost. With the new S-Class, that information is overlaid right on the windshield with an augmented reality windshield. The automaker sees more opportunities for HoloLens 2. It can be used for sales training or even with customers on the showroom floor, customizing their car before their eyes.

Since the introduction of augmented reality, it seemed destined for this kind of application – infusing hands-on work and know-how with data to make complex tasks easier. Now that Mercedes is putting it into action, we're looking forward to see just how effective it can be.