Mixed reality

Mixed reality (MR) is the merging of real and virtual worlds to produce new environments and visualizations, where physical and digital objects co-exist and interact in real time. Mixed reality does not exclusively take place in either the physical or virtual world, but is a hybrid of reality and virtual reality, encompassing both augmented reality and augmented virtuality via immersive technology.[2]

For Windows Mixed Reality, see Windows Mixed Reality. For the Gin Blossoms album, see Mixed Reality.
An example of mixed reality, showing virtual characters mixed into a real-world live stream[1]

The first immersive mixed reality system that provided enveloping sight, sound, and touch was the Virtual Fixtures platform, which was developed in 1992 at the Armstrong Laboratories of the United States Air Force. The project demonstrated that human performance could be significantly amplified, by overlaying spatially registered virtual objects on top of a person's direct view of a real physical environment.[3]

Differences in terminology
The reality-virtuality continuum

Mixed reality refers to a continuum that encompasses both virtual reality (VR) and augmented reality (AR):

  • Virtual reality immerses users in a fully artificial digital environment.
  • Augmented reality overlays virtual objects on the real-world environment with spatial registration that enables geometric persistence concerning placement and orientation within the real world. Prior technologies that overlaid data or images not spatially registered to real-world geometries are referred to as heads-up display technologies.

Definition
Mediated reality continuum (horizontal axis: virtuality; vertical axis: mediality). Four points are shown for augmented reality, augmented virtuality, mediated reality, and mediated virtuality.

Virtuality continuum and mediality continuum

In 1994, Paul Milgram and Fumio Kishino defined a mixed reality as "...anywhere between the extrema of the virtuality continuum" (VC),[2] where the virtuality continuum extends from the completely real through to the completely virtual environment, with augmented reality and augmented virtuality ranging between. The first fully immersive mixed reality system was the Virtual Fixtures platform, which was developed in 1992 by Louis Rosenberg at the Armstrong Laboratories of the United States Air Force. It enabled human users to control robots in real-world environments that included real physical objects and 3D virtual overlays ("fixtures") that were added enhance human performance of manipulation tasks. Published studies showed that by introducing virtual objects into the real world, significant performance increases could be achieved by human operators.[4][5]

The continuum of mixed reality is one of the two axes in Steve Mann's concept of mediated reality as implemented by various welding helmets, wearable computers, and wearable photographic systems he created in the 1970s and early 1980s.[6][7][8][9][10] The second axis was the mediality continuum, which includes Diminished Reality, such as implemented in a welding helmet or eyeglasses that can block out advertising or replace real-world ads with useful information.[11][12]

"The conventionally held view of a Virtual Reality (VR) environment is one in which the participant-observer is immersed in, and able to interact with, a completely synthetic world. Such a world may mimic the properties of some real-world environments, either existing or fictional; however, it can also exceed the bounds of physical reality by creating a world in which the physical laws ordinarily governing space, time, mechanics, material properties, etc. no longer hold. What may be overlooked in this view, however, is that the VR label is also frequently used in association with a variety of other environments, to which total immersion and complete synthesis do not necessarily pertain, but which fall somewhere along a virtuality continuum. In this paper, we focus on a particular subclass of VR related technologies that involve the merging of real and virtual worlds, which we refer to generically as Mixed Reality (MR)."

Interreality physics

In a physics context, the term "interreality system" refers to a virtual reality system coupled with its real-world counterpart.[13] A 2007 paper describes an interreality system comprising a real physical pendulum coupled to a pendulum that only exists in virtual reality.[14] This system has two stable states of motion: a "Dual Reality" state in which the motion of the two pendula are uncorrelated, and a "Mixed Reality" state in which the pendula exhibit stable phase-locked motion, which is highly correlated. The use of the terms "mixed reality" and "interreality" is clearly defined in the context of physics, but may be slightly different in other fields.

Mixed Reality
A person is in a virtual environment, with a camera acting as a third-person point of view.[15]

Augmented virtuality (AV) is a subcategory of mixed reality that refers to the merging of real-world objects into virtual worlds.[16]

As an intermediate case in the virtuality continuum, it refers to predominantly virtual spaces, where physical elements (such as physical objects or people) are dynamically integrated into and can interact with the virtual world in real time. This integration is achieved with the use of various techniques, such as streaming video from physical spaces, like through a webcam,[17] or using the 3D digitalization of physical objects.[18]

The use of real-world sensor information, such as gyroscopes, to control a virtual environment is an additional form of augmented virtuality, in which external inputs provide context for the virtual view.

Applications

Mixed reality has been used in applications across fields including art, entertainment, and military training.

Interactive product content management (IPCM)

Moving from static product catalogs to interactive 3D smart digital replicas, this solution consists of application software products with scalable license models.

Simulation-based learning (SBL)

Moving from e-learning to s-learning, simulation-based learning includes VR-based training and interactive, experiential learning. This also includes software and display solutions with scalable licensed curriculum development model.

Military training

Combat reality is simulated and represented in complex, layered data through HMD. Military training solutions are often built on commercial off-the-shelf (COTS) technologies, such as Virtual Battlespace 3 and VirTra, both of which are used by the United States Army. As of 2018, VirTra is being used by both civilian and military law enforcement to train personnel in a variety of scenarios, including active shooter, domestic violence, and military traffic stops.[19][20] Mixed reality technologies have been used by the United States Army Research Laboratory to study how this stress affects decision-making. With mixed reality, researchers may safely study military personnel in scenarios where soldiers would not likely survive.[21]

In 2017, the U.S. Army was developing the Synthetic Training Environment (STE), a collection of technologies for training purposes that was expected to include mixed reality. As of 2018, STE was still in development without a projected completion date. Some recorded goals of STE included enhancing realism and increasing simulation training capabilities and STE availability to other systems.[22]

It was claimed that mixed-reality environments like STE could reduce training costs,[23][24] such as reducing the amount of ammunition expended during training.[25] In 2018, it was reported that STE would include representation of any part of the world's terrain for training purposes.[26] STE would offer a variety of training opportunities for squad brigade and combat teams, including Stryker, armory, and infantry teams.[27] STE is expected to eventually replace the U.S. Army's Live, Virtual, Constructive – Integrated Architecture (LVC-IA).[28]

Remote working

Mixed reality allows a global workforce of remote teams to work together and tackle an organization's business challenges. No matter where they are physically located, an employee can wear a headset and noise-canceling headphones and enter a collaborative, immersive virtual environment. As these applications can accurately translate in real time, language barriers become irrelevant. This process also increases flexibility. While many employers still use inflexible models of fixed working time and location, there is evidence that employees are more productive if they have greater autonomy over where, when, and how they work. Some employees prefer loud work environments, while others need silence. Some work best in the morning; others work best at night. Employees also benefit from autonomy in how they work because of different ways of processing information. The classic model for learning styles differentiates between Visual, Auditory, and Kinesthetic learners.[29]

Machine maintenance can also be executed with the help of mixed reality. Larger companies with multiple manufacturing locations and a lot of machinery can use mixed reality to educate and instruct their employees. The machines need regular checkups and have to be adjusted every now and then. These adjustments are mostly done by humans, so employees need to be informed about needed adjustments. By using mixed reality, employees from multiple locations can wear headsets and receive live instructions about the changes. Instructors can operate the representation that every employee sees, and can glide through the production area, zooming in to technical details and explaining every change needed. Employees completing a five-minute training session with such a mixed-reality program has been shown to yield the same results as reading a 50-page training manual.[30]

Functional mockup

Mixed reality can be used to build mockups that combine physical and digital elements. With the use of simultaneous localization and mapping (SLAM), mockups can interact with the physical world to utilize features like object permanence.

Consciousness

It has been hypothesized that a hybrid of mixed and virtual reality could pave the way for human consciousness to be transferred into a digital form entirely—a concept known as Virternity, which would leverage blockchain to create its main platform.[31][32]

Medical

Mixed reality can combine smartglasses with surgical processes.[33] Mixed reality headsets (Microsoft HoloLens, ...) allow for sharing of information between doctors, meaning fewer doctors need to be in close proximity to the patient. This can, in some situations (i.e. patient infected with contagious disease), improve doctor safety and reduce PPE use.[34]

Display technologies

Here are some commonly used MR display technologies:

Notable applications
gollark: Hmm, what an uncool hash function.
gollark: What hash function are you using?
gollark: `raise Exception("Bees or something")`
gollark: Why are you doing `return 1//0` to throw an error?!
gollark: You're doing a ridiculously high-stakes presentation on using Git instead of superior SQLite?

See also

References
  1. R. Freeman, A. Steed and B. Zhou, Rapid Scene Modelling, Registration and Specification for Mixed Reality Systems Archived 2007-02-06 at the Wayback Machine Proceedings of ACM Virtual Reality Software and Technology, pp. 147-150, Monterey, California, November 2005.
  2. P. Milgram and A. F. Kishino (1994). "Taxonomy of Mixed Reality Visual Displays". IEICE Transactions on Information and Systems. pp. 1321–1329. Retrieved 2013-10-17.
  3. Rosenberg, Louis B. (1992). "The Use of Virtual Fixtures As Perceptual Overlays to Enhance Operator Performance in Remote Environments". Technical Report AL-TR-0089, USAF Armstrong Laboratory, Wright-Patterson AFB OH, 1992.
  4. L. B. Rosenberg. The Use of Virtual Fixtures As Perceptual Overlays to Enhance Operator Performance in Remote Environments. Technical Report AL-TR-0089, USAF Armstrong Laboratory, Wright-Patterson AFB OH, 1992.
  5. Rosenberg, Louis B. (1993). "Virtual fixtures as tools to enhance operator performance in telepresence environments". Telemanipulator Technology and Space Telerobotics. 2057: 10–21. Bibcode:1993SPIE.2057...10R. doi:10.1117/12.164901.
  6. Steve Mann, "Campus Canada", ISSN 0823-4531, p55 Feb-Mar 1985, pp58-59 Apr-May 1986, p72 Sep-Oct 1986
  7. Impulse, Volume 12, Number 2, 1985
  8. Quantigraphic camera promises HDR eyesight from Father of AR, by Chris Davies, SlashGear, Sep 12th 2012
  9. IEEE Technology & Society 31(3)
  10. Through the Glass, Lightly, IEEE Technology & Society, Volume 31, Number 3, Fall 2012, pages 10-14
  11. Mann, S., & Fung, J. (2001). Video orbits on EyeTap devices for deliberately diminished reality or altering the visual perception of rigid planar patches of a real-world scene. Proceedings of the Second IEEE International Symposium on Mixed Reality, pp 48-55, March 14–15, 2001.
  12. 关于智能眼镜 (About Smart Glasses), 36KR, 2016-01-09
  13. J. van Kokswijk, Hum@n, Telecoms & Internet as Interface to Interreality Archived 2007-09-26 at the Wayback Machine (Bergboek, The Netherlands, 2003).
  14. V. Gintautas, and A. W. Hubler, Experimental evidence for mixed reality states in an interreality system Phys. Rev. E 75, 057201 (2007).
  15. LIV. "Mixed Reality Studio - The LIV Cube". LIV. Retrieved 2017-04-08.
  16. P. Milgram, and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays IEICE Transactions on Information and Systems, E77-D(12), pp. 1321–1329, 1994.
  17. "The Dive Home Page". 30 June 2012. Archived from the original on 2012-06-30.
  18. "Introduction - Teleimmersion Lab". UC Berkeley.
  19. Inc., VirTra. "VirTra's Police Training Simulators Chosen by Three of the Largest U.S. Law Enforcement Departments". GlobeNewswire News Room. Retrieved 2018-08-22.
  20. "How do police use VR? Very well | Police Foundation". www.policefoundation.org. 2017-08-14. Retrieved 2018-08-22.
  21. Patton, Debbie; Marusich, Laura (2015-03-09). 2015 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision. pp. 145–150. doi:10.1109/COGSIMA.2015.7108190. ISBN 978-1-4799-8015-4.
  22. Eagen, Andrew (June 2017). "Expanding Simulations as a Means of Tactical Training with Multinational Partners" (PDF). A thesis presented to the Faculty of the U.S. Army Command and General Staff College.
  23. Bukhari, Hatim; Andreatta, Pamela; Goldiez, Brian; Rabelo, Luis (2017-01-01). "A Framework for Determining the Return on Investment of Simulation-Based Training in Health Care". INQUIRY: The Journal of Health Care Organization, Provision, and Financing. 54: 0046958016687176. doi:10.1177/0046958016687176. ISSN 0046-9580. PMC 5798742. PMID 28133988.
  24. Smith, Roger (2010-02-01). "The Long History of Gaming in Military Training". Simulation & Gaming. 41 (1): 6–19. doi:10.1177/1046878109334330. ISSN 1046-8781.
  25. Shufelt, Jr., J.W. (2006) A Vision for Future Virtual Training. In Virtual Media for Military Applications (pp. KN2-1 – KN2-12). Meeting Proceedings RTO-MP-HFM-136, Keynote 2. Neuilly-sur-Seine, France: RTO. Available from: http://www.rto.nato.int/abstracts.asp {{Webarchive|url=https://web.archive.org/web/20070613170605/http://www.rto.nato.int/Abstracts.asp |date=2007-06-13 }}
  26. "STAND-TO!". www.army.mil. Retrieved 2018-08-22.
  27. "Augmented reality may revolutionize Army training | U.S. Army Research Laboratory". www.arl.army.mil. Retrieved 2018-08-22.
  28. "Army Shoots for Single Synthetic Training Environment". GovTechWorks. 2015-11-17. Retrieved 2018-08-22.
  29. Sena, Pete. "How The Growth Of Mixed Reality Will Change Communication, Collaboration And The Future Of The Workplace". TechCrunch. Retrieved 2017-05-16.
  30. The Manufacturer. "Manufacturers are successfully using mixed reality today". www.themanufacturer.com.
  31. "Potential of Blockchain Technology as Protocol of Universal Virtual Reality". TurboFuture. Retrieved 2018-10-18.
  32. Shirazi, Sina. "Pairing Augmented Reality with Virtual Reality". Cite journal requires |journal= (help)
  33. "Taipei hits highs in Medica 2017". healthcare-in-europe.com. Retrieved 2019-04-05.
  34. Mixed-reality headsets in hospitals help protect doctors and reduce need for PPE

Further reading

Media related to Mixed reality at Wikimedia Commons

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.