Virtual and mixed realities converge in new driving simulator

Portobello, a brand new driving simulator developed by researchers at Cornell Tech, blends digital and combined realities, enabling each drivers and passengers to see digital objects overlaid in the true world.

This expertise opens up new prospects for researchers to conduct the identical person research each within the lab and on the highway—a novel idea the staff calls “platform portability.”

The analysis staff, led by Wendy Ju, affiliate professor on the Jacobs Technion-Cornell Institute at Cornell Tech, introduced their paper, “Portobello: Prolonged Driving Simulation from the Lab to the Highway,” on the ACM Conference on Human Factors in Computing Systems (CHI) in Might. The paper earned honorable point out on the convention.

Co-authors included doctoral college students Fanjun Bu, Stacey Li, David Goedicke, and Mark Colley; and Gyanendra Sharma, an industrial adviser from Woven by Toyota.

Portobello is an on-road driving simulation system that allows each drivers and passengers to make use of mixed-reality (XR) headsets. The staff’s motivation for growing Portobello stemmed from its work on XR-OOM, an XR driving simulator system. The instrument may merge facets of the bodily and digital worlds, but it surely had limitations.

“Whereas we may stage digital objects in and across the automobile—similar to in-car digital shows and digital dashboards—we had issues staging digital occasions relative to things in the true world, similar to a digital pedestrian crossing on an actual crosswalk or having a digital automobile cease at actual cease indicators,” Bu stated.

This posed a major impediment to conducting significant research, significantly for autonomous driving experiments that require exact staging of objects and occasions in mounted areas throughout the atmosphere.

Portobello was conceived to beat these limitations and anchor on-road driving simulations within the bodily world. Throughout the design part, researchers make the most of the Portobello system to generate a exact map of the examine atmosphere. Inside this map, they will strategically place digital objects based mostly on real-world components (inserting digital pedestrians close to cease indicators, for instance). The automobile operates throughout the similar mapped atmosphere, seamlessly mixing simulation and actuality.

With the profitable integration of Portobello, the staff has not solely addressed the restrictions of XR-OOM however has additionally launched platform portability. This innovation permits researchers to conduct an identical research in each managed laboratory settings and real-world driving situations, enhancing the precision and applicability of their findings.

“Individuals deal with in-lab simulators as visible approximations of real-world situations, virtually a performative expertise,” Bu stated. “Nevertheless, individuals deal with on-road simulators as practical approximations. [They] felt extra stress in on-road simulators and felt their choices carried extra weight.”

Bu stated Portobello may facilitate the “twinning of research”—operating the identical examine throughout completely different environments. This, he stated, not solely makes findings extra sensible, but in addition helps uncover how different elements would possibly have an effect on the outcomes.

Ju stated, “We imagine that by going past operating pristine research and permitting some variability from real-world to bleed by way of, analysis outcomes might be extra relevant to real-world settings.”

Hiroshi Yasuda, a human-machine interplay researcher at Toyota Analysis Institute (TRI), additionally contributed to the analysis.