A brave new world of virtual, augmented, hybrid, hyper, and diminished realities
My mind is currently churning with thoughts pertaining to the potential uses, possibilities and potentialities, and advantages and disadvantages of various forms of virtual and augmented realities (VR and AR).
All this was sparked by my recent acquisition of a mind-boggling virtual reality system comprising an Oculus Rift coupled with a processor-graphics combo that makes one drool with desire and squeal in delight (see also It's (Virtual Reality) Life Jim, But Not As We Know It).
The bay outside my office -- I never realized I had so many friends
(Click Here to see a larger image. Source: Max Maxfield)
Virtual Reality (VR)
Now, before we plunge headlong into the fray with gusto and abandon, it may be a good idea to take a deep breath and start by considering what we mean by virtual reality. Actually, even the concept of reality is a little fluffy around the edges. For example, the Holographic principle suggests that the entire universe can be seen as two-dimensional information presented on the cosmological horizon (my dear old mother, by comparison, suggests that the Holographic principle is a load of twaddle).
Personally, I would argue that virtual reality is anything that allows us to experience a different reality to the one we perceive through our senses. On this basis, ancient humans sitting around campfires listening to their storytellers weave fantastic tales were experiencing a form of virtual reality. Similarly, fictional books allow us to envision alternative realities, as do traditional two-dimensional (2D) and three dimensional (3D) movies.
Today's computer-simulated immersive multimedia virtual reality systems take things one step further, but that step is a humongous one. It's impossible to convey the experience of using an Oculus Rift-based virtual reality system -- you really have to try it for yourself. (In a future column I will be comparing and contrasting a variety of systems, including the Oculus Rift and the HTC Vive.) Once you don the Oculus headset with its integrated headphones, you are transported to a different world with associated sound effects. It's more than having the ability to look up at the ceiling or down at the floor or all around you -- if you tilt your head to one side, crouch down, stand on your tippy-toes, or move forward, backward, or sideward, your perspective changes accordingly.
The closest thing I can use to describe this experience is being on a Star Trek Holodeck, and we're still only at the beginning of this technology. For example, I'm currently using the Xbox one controller that came with my Rift, but I'm anxiously awaiting the availability of Oculus Touch controllers, which will add a new dimension to the proceedings by allowing you to "see" virtual representations of your hands in your virtual world and precisely manipulate objects in your virtual environment.
In the future, we can expect a wide variety of haptic ("touch") technologies and devices to appear on the scene (no pun intended). These little scamps will dramatically augment the user's visual and auditory experience with physical components.
As a simple haptic example, one of the free applications that comes with the Oculus Rift is the Oculus Dreamdeck, which allows you to sample a suite of virtual experiences (just listen to the commentator's excitement when he reaches the 3:20 mark in this video).
If you are short of time, skip forward to the dinosaur scene that starts at the 9:20 mark. This is one I always use to introduce someone to the Oculus experience. Assume you are in the bay outside my office having your first Rift experience. Under my tutelage you select the dinosaur option. The scene commences with you standing in a corridor of a museum (I can see a simplified 2D version of what you are experiencing on the computer monitor next to the workstation). A humongous T-Rex appears at the far end of the corridor and stomps its way towards you. At around the 10:10 mark it opens its mouth and roars at you with spittle flying all over the place. At exactly the same time, I take a cup of water I have stashed nearby and flick drops of water into your face.
This is my poor-man's version of a haptic interface. When I did this to the guy who owns the building in which I maintain my office, he afterwards said that he experienced a "Total WTF moment" (of course, being young and innocent, I have no idea what this means LOL).
Augmented and Diminished Realities
An augmented reality is one in which a live direct or indirect view of a physical, real-world environment is augmented (or supplemented) by computer-generated sensory input such as sound, video, text, graphics, and/or GPS data. For example, consider this video depicting Magic Leap technology.
The information that is overlaid on the real-world view and the objects therein can be virtual or real in nature, and it can include data that wouldn’t be possible to perceive by normal means.
There are aspects to all of this that aren’t immediately apparent. For example, machine vision and artificial intelligence technologies based on deep neural networks (DNNs) and deep learning have evolved spectacularly over the past few years. By enhancing augmented reality technology with machine vision and object detection and recognition, information about the real world surrounding the user becomes interactive and digitally manipulable.
What this means is that augmented realities are really only a subset of mediated realities, in which a view of reality is modified or mediated in some way. In turn, this means that, as opposed to (or in addition to) being augmented, a real-world view might also be diminished in some way. As one example, suppose you are perambulating around a city like San Francisco, viewing the world through some form of mediated reality system that detects things you'd rather not see -- like trash or graffiti or homeless people -- and simply removes them from the scene (or replaces them with something more palatable).
I've called this diminished reality, but other applicable terms might be decreased, deleted, or deletive reality (I'd be very interested to hear your views on this).
Hybrid and Hyper Realities
Other terms one sometimes hears are hybrid reality and hyper reality. It's easy to have a knee-jerk reaction that someone is just coining new terms for the sake of it and/or for marketing purposes, but it may be that we do need to consider these additional nomenclatures.
Let's start by defining virtual reality (in today's broadly accepted context) to be a completely fabricated view of an artificial world. Let's further define augmented reality to be a real-world view annotated with textual, graphical, etc. information overlays.
Now, suppose we were to take a real-world view and add artificial world overlays -- not information per se, but artificial constructs such as buildings, machines, statues, flora, and fauna (including mythical creatures like dragons, for example). You could call this augmented reality, but at some stage I think it might be fair to classify the result as a hybrid reality.
Here's another situation that just popped into my mind. Do you recall the old Mystery Science Theater 3000 television program. The idea was that you watched a cheesy old science fiction film that was presented as though you were sitting at the back of a cinema. In the front row of the cinema (appearing as silhouettes from your perspective) were three characters passing disparaging comments about the film you all were watching. The films were selected for their awfulness, but they were made palatable by the commentary. I recall one dreadful black-and-white movie in which, half way through, a man was seen running down a corridor carrying a huge stack of paper, and one of the characters at the bottom of the screen exclaimed, "Thank God, the script's arrived!"
The reason I mention this here is that there is an Oculus Rift app that allows you to watch your favorite movies, or YouTube videos, in the comfort of a virtual reality cinema (you can select between various theater settings). Now suppose that your headset was equipped with binocular, forward-facing cameras and an associated machine vision system. Further suppose that you have a bowl of popcorn on your knee. Assume this system were such that you could enjoy the experience of a virtual cinema -- with rows of seats in front of you and everything -- but, when you looked down, you saw the real-world bowl of popcorn and your real-world hands. In this case, I think the term hybrid reality would be apposite.
What about the term hyper reality? Well, all I can say is that you should take a look at this Vimeo video and then tell me what you think. Once again, I suppose you could class this as augmented reality, but it certainly seems "hyper" to me.
To be continued...
I could waffle on about all of this stuff for ages, but I think I may have waffled enough for this column. In my next article we will consider some of the technologies that can be brought into play to present virtual and mediated realities, from smartphones and tablet computers to devices like Google Glass (RIP) to... but let's not get carried away here.
We'll also be considering applications for these alternative realities, from entertainment to education to commercial to industrial.
In the meantime, as noted above, I'd be interested to hear what you think about the term mediated realities being used to encompass augmented realities and diminished realities. I'd also be interested to know if you like diminished reality in this context, or if you prefer decreased, deleted, or deletive reality. Finally, do you agree with my assessment that hybrid reality deserves its place in the nomenclature? And is hyper reality its own beast, or is this just another hybrid incarnation?