Digital and blended realities converge in new driving simulator

Portobello, a brand new driving simulator developed by researchers at Cornell Tech, blends digital and blended realities, enabling each drivers and passengers to see digital objects overlaid in the true world.

This know-how opens up new prospects for researchers to conduct the identical consumer research each within the lab and on the street—a novel idea the staff calls “platform portability.”

The analysis staff, led by Wendy Ju, affiliate professor on the Jacobs Technion-Cornell Institute at Cornell Tech, introduced their paper, “Portobello: Prolonged Driving Simulation from the Lab to the Street,” on the ACM Convention on Human Elements in Computing Methods (CHI) in Might. The paper earned honorable point out on the convention.

Co-authors included doctoral college students Fanjun Bu, Stacey Li, David Goedicke, and Mark Colley; and Gyanendra Sharma, an industrial adviser from Woven by Toyota.

Portobello is an on-road driving simulation system that allows each drivers and passengers to make use of mixed-reality (XR) headsets. The staff’s motivation for creating Portobello stemmed from its work on XR-OOM, an XR driving simulator system. The instrument may merge elements of the bodily and digital worlds, but it surely had limitations.

“Whereas we may stage digital objects in and across the automotive—resembling in-car digital shows and digital dashboards—we had issues staging digital occasions relative to things in the true world, resembling a digital pedestrian crossing on an actual crosswalk or having a digital automotive cease at actual cease indicators,” Bu mentioned.

This posed a major impediment to conducting significant research, notably for autonomous driving experiments that require exact staging of objects and occasions in fastened areas throughout the atmosphere.

Portobello was conceived to beat these limitations and anchor on-road driving simulations within the bodily world. Through the design part, researchers make the most of the Portobello system to generate a exact map of the research atmosphere. Inside this map, they will strategically place digital objects based mostly on real-world components (putting digital pedestrians close to cease indicators, for instance). The automobile operates throughout the similar mapped atmosphere, seamlessly mixing simulation and actuality.

With the profitable integration of Portobello, the staff has not solely addressed the restrictions of XR-OOM however has additionally launched platform portability. This innovation allows researchers to conduct similar research in each managed laboratory settings and real-world driving eventualities, enhancing the precision and applicability of their findings.

“Individuals deal with in-lab simulators as visible approximations of real-world eventualities, virtually a performative expertise,” Bu mentioned. “Nevertheless, individuals deal with on-road simulators as purposeful approximations. [They] felt extra stress in on-road simulators and felt their selections carried extra weight.”

Bu mentioned Portobello may facilitate the “twinning of research”—operating the identical research throughout completely different environments. This, he mentioned, not solely makes findings extra reasonable, but additionally helps uncover how different elements would possibly have an effect on the outcomes.

Ju mentioned, “We consider that by going past operating pristine research and permitting some variability from real-world to bleed by, analysis outcomes might be extra relevant to real-world settings.”

Hiroshi Yasuda, a human-machine interplay researcher at Toyota Analysis Institute (TRI), additionally contributed to the analysis.