XR-Based Decision Making in Automotive Development

This article provides an overview of extended reality (XR), highlights the benefits of 3D visualization and XR, and explains different use cases of XR in automotive development.

What Is XR

Extended reality (XR) is an umbrella term for virtual reality, augmented reality, and mixed reality. The 'X' can be considered as a variable representing any current or future spatial computing technology.

Virtual reality (VR) is fully immersive. It’s all virtual, with no connection to the real world. An example would be an application that requires you to wear a head-mounted display (HMD) or sit in a chair while experiencing the VR application. In addition to industrial solutions, games are also a popular example of how VR is used.

Augmented reality (AR) is an overlay to the real world. The most famous example of an AR application is probably Pokémon GO, a game where you collect virtual Pokemons, captured in real life. Another example is a smartphone display where a 3D image pops out in front of a 2D catalog. Head-up-displays used for navigation in cars, plane cockpits, or integrated into a helmet also use AR.

Mixed Reality (MR) is the combination of the real and virtual worlds. One example is a seating buck (a frame to which a steering wheel and a seat are attached) that allows you to see the virtual car and get haptic feedback from a real steering wheel. In the case of the Varjo XR-3 device, mixed reality can also involve your own hands visible in VR in front of your HMD. Mixed reality can also be a virtual car next to a real car, seen through a tablet.

Related – Immersive Design: Transforming Architecture with Virtual Reality

Benefits of XR and 3D Visualization

The general benefits of 3D visualization are cost reduction, increased efficiency, and greater flexibility.

Cost Reduction
In the virtual design validation process, you can reduce the amount of clay and cubing models by using virtual prototypes instead. Clay models are required to secure the design process but are very expensive. Cubing models are necessary to secure the engineering process but are even more expensive. Generally, you need fewer prototypes and hardware if you rely on 3D visualization.

Increased Efficiency
Increase the efficiency of your processes with 3D visualization. You don't have to wait for the hardware to be built; you can decide based on your 3D visualization. The data is created with 3D software anyway and is already available.

Greater Flexibility
It’s possible to visualize all possible product variants in one single file, as well as different proposed design and engineer solutions during the development process. You can simulate the front and rear light of the exterior, the ambient light in the interior, and complex light animations such as welcome scenarios when unlocking a door. Designs can be compared in different environments such as landscapes and interiors. It can also be lit in a photo studio scenario to explore marketing potentials. 3D visualization can be accessed by anyone via real-time applications or rendered images.

Benefits of XR Compared to Conventional 3D

Thanks to a true 3D representation of a product in XR, more realism is achieved. You can better judge surfaces and joints and get a realistic idea of how the product will look. As a user, you are more involved and get a better emotional understanding of the product. Thanks to fully immersive tracking technology, you can conduct realistic ergonomic studies. Compared to a traditional CAVE, for example, XR is more cost effective, requires less space, and can be used more flexibly—for example, at trade shows to gather important feedback from visitors or to present new designs. During a virtual collaboration, you can combine the real world with the virtual and take advantage of both for better decision making.

Use Cases of XR in Automotive

Evaluation Using VR

When using XR for automotive evaluation, the setup is a head-mounted display connected to a high-end PC with a good graphics card, which allows for easy tracking of body position and head movements. It can be inside-out tracking technology, which takes up less space and can be set up more flexibly, or outside-in tracking with additional external trackers like the HTC Vive with the Lighthouse base station. VR controllers that track hand position and display hands in VR are optional, but this leads to an even better immersion. Possible use cases include the evaluation of surfaces, materials, and lights in daytime and nighttime scenarios.

Evaluation Using MR - Interior

Starting with MR to evaluate interior design, the setup is an HMD, complex tracking of body, head movements, hands, and a seating buck. Additional 3D-printed parts of the vehicle, such as a dashboard, center console, and doors can be mounted on the seating buck. This enables a haptic representation of the surfaces in MR, so that the user can touch and feel the design.

Use cases are the evaluation of materials and surfaces, HMI concepts, ambient lighting, and ergonomic studies. For HMI design, you can test a click-dummy programmed with HTML 5 or Qt Designer directly in your VRED scene. With hand tracking enabled and a physical representation of your vehicle, such as 3D-printed dashboard and door surfaces, you can explore the entire interior, including display content. The ambient light and illuminated icons of bottoms can also be viewed in MR if there is a night scenario of your design prepared in VRED. You can also calculate realistic light simulation as well as animated welcome scenarios. It is possible to test the accessibility of gearshifts, sun visors, and storage compartments, all virtually and with realistic materials.

Evaluation Using MR - Exterior

The setup of an exterior evaluation looks similar to that of an interior evaluation but you don’t need a seating buck. The need for a physical representation depends on your use case. Hands are less important in exterior evaluations and the hand-tracking can be done by VR controllers. Use cases include comparing a real car with a virtual one. You can view and evaluate virtual facelift parts on a real, existing vehicle before the design changes go into production. A virtual vehicle can be viewed in a real environment, or you can evaluate virtual materials on a physical representation of the car such as a clay model. With advanced hand tracking methods enabled by Varjo and Ultraleap, you can also view your real hands in VR.

Virtual Collaboration

For a virtual collaboration, you need a high-end PC or laptop, an HMD for joining the collaboration in VR, and of course a stable Internet connection. The benefits include:

Easy access for any persona. Even non-visualization experts can access a virtual collaboration with a prepared data set.
Participants can access from anywhere no matter where they are located to exchange ideas.
Several experts from different departments can work on the same data set at the same time. It is possible to take another's point of view and explore the product together. This enables closer collaboration.

Cloud-Based Workflows

Apart from the hardware used, a fast wireless Internet connection such as a 5G mobile network, is the most important aspect for cloud-based workflows. The main advantage is that expensive local hardware is no longer needed. Users can access from multiple devices such as tablets, cell phones, HMDs, or laptops. Untethered devices free you from annoying cables. The content can be streamed directly to a wireless HMD. Benefits include no stumbling over cables, no complex setup with external trackers / a special mounting on the ceiling to deal with the cables, and no space limitations so the user doesn't get pulled back by the cable attached to his or her HMD.

Cloud-based workflows enable broad participation. Users can access from anywhere, given they have high-speed WiFi, and are enabled to participate from a home office. Your vision can be shared with anyone such as customers, colleagues, or other stakeholders. The possibilities of sharing your designs are endless. Render power can be scaled on demand. If you want to evaluate your VRED scene with realistic reflections, lighting, and shading, you just need to unlock more render nodes and you can evaluate your design with real-time GPU ray tracing. Or if you have an important presentation, you can render an offline animation overnight and present it the next day.

Related AU Sessions

Learn more with the full class.

Tillmann Dorsch is a product owner for visualization (VRED) in the Product Management team at Autodesk. He has a background in 3D visualization and was working over the past 10 years as a 3D artist, visualization expert, and consultant, mostly in Automotive, to create visuals, secure virtual processes, and manage car configurators. He was working for different customers and in different companies before he joined Autodesk. Tillmann studied Virtual Design and holds an MBA in International Management.