For the purpose of this chapter, Cross Reality or XR refers to technologies and applications that involve combinations of mixed reality (MR), augmented reality (AR), virtual reality (VR), and virtual worlds (VWs). These are technologies that connect computer technology (such as informational overlays) to the physical world for the purposes of augmenting or extending experiences beyond the real. Especially relevant to the definition of XR is the fact that this term encompasses a wide range of options for delivering learning experiences, from minimal technology and episodic experiences to deep immersion and persistent platforms. The preponderance of different terms for slightly different technologies indicate that this is a growth area within the field. Here we provide a few definitions of these technologies.
MR—Mixed reality refers to a blend of technologies used to influence the human perception of an experience. Motion sensors, body tracking, and eye tracking interplay with overlaid technology to give a rich and full version of reality displayed to the user. For example, technology could add sound or additional graphics to an experience in real time. Examples include the Magic Leap One and Microsoft HoloLens 2.0. MR and XR are often used interchangeably.
AR—Augmented reality refers to technology systems that overlay information onto the real world, but the technology might not allow for real-time feedback. As such, AR experiences can move or animate, but they might not interact with changes in depth of view or external light conditions. Currently, AR is considered the first generation of the newer and more interactive MR experiences.
VR—Virtual reality, as a technological product, traces its history to approximately 1960 and tends to encompass user experiences that are visually and auditorily different from the real world. Indeed, the real world is often blocked from interacting with the virtual one. Headsets, headphones, haptics, and haptic clothing might purposely cut off all input except that which is virtual. In general, VR is a widely recognizable term, often found in gaming and workplace training, where learners need to be transported to a different time and place. VR experiences in STEM often consist of virtual labs or short virtual field trips.
VW—Virtual worlds are frequently considered a subset of VR with the difference that VWs are inherently social and collaborative; VWs frequently contain multiple simultaneous users, while VRs are often solo experiences. Another discrimination between virtual reality and virtual worlds is the persistence of the virtual space. VR tends to be episodic, with the learner in the virtual experience for a few minutes and the reality created within the experience ends when the learner experience ends. VWs are persistent in that the worlds continue to exist on computer servers whether or not there are active avatars within the virtual space (Bell 2008). This discrimination between VR and VW, however, is dissolving. VR experiences can be created to exist for days, and some users have been known to wear headsets for extended periods of time. Additionally, more and more VR experiences are being designed to be for game play, socialization, or mental relaxation. The IEEE VR 2020 online conference and the Educators in VR International Summit 2020 offered participants opportunities to experience conference presentations in virtual rooms as avatars while interacting with presenters and conference attendees (see Sect. 2.5 for more information).
CVEs—Collaborative virtual environments are communication systems in which multiple interactants share the same three-dimensional digital space despite occupying remote physical locations (Yee and Bailenson 2006).
Embodiment—Embodiment is defined by Lindgren and Johnson-Glenberg (2013) as the enactment of knowledge and concepts through the activity of our bodies within an MR (mixed reality) and physical environment
Human-Centered Design philosophy that involves putting human needs, capabilities, and behavior first (Jerald 2018: 15). XR provides the opportunity to experience just-in-time immersive, experiential learning that uses concrete yet exploratory experiences involving senses that result in lasting memories. Here we discuss opportunities for social applications with XR.
XR learner activities are usually created for individual use, which may or may not need to be simultaneously experienced as a class together at the same time or place with the instructor. Activities can be designed into instruction with VR headsets, high-resolution screens, smartphones, or other solo technological devices for use inside and outside of the classroom.
Do we address the challenges in the grant proposal?
some learners will be held back from full XR activity by visual, physical, and social abilities such as stroke, vertigo, epilepsy, or age-related reaction time. It should also be noted that the encompassing nature of VR headsets might create some discomfort or danger for any learners as they can no longer fully see and control their body and body space.
Tech CEOs keep talking about “the metaverse.” Mark Zuckerberg insists that Facebook will be seen as a “metaverse company” instead of a social media company. Satya Nadella proclaims Microsoft is creating a “metaverse stack” for the enterprise.
Author Neil Stephenson coined the term “metaverse” in Snow Crash, a dystopian cyberpunk novel published in 1992.
In the novel, the metaverse is a sort of 3D virtual world. It’s not simply a virtual reality game but is a persistent, shared virtual world. Or rather, the metaverse is a whole universe of shared virtual spaces seemingly linked together—you could, essentially, teleport between them.
If you think this all sounds a bit like Ready Player One or a higher-tech version of Second Life, you’re right.
virtual reality (VR) and not augmented reality (AR) was necessary for that kind of vision
To Zuckerberg and other tech CEOs, the concept of “the metaverse” seems to have more in common with “Web 2.0.” It’s a bunch of new technologies: VR headsets! Presence! Persistent digital worlds
Microsoft’s vision of the metaverse seems to take the form of rambling, buzzword-heavy talk about “digital twins” and “converging the physical with the digital” with “mixed reality.” Microsoft’s Azure cloud can do it!
The terms ‘extended reality’ or ‘cross reality’ refer to “technologies and applications that involve combinations of mixed reality (MR), augmented reality (AR), virtual reality (VR), and virtual worlds (VWs)” (Ziker, Truman, & Dodds, 2021, p. 56). Immersive learning definitions draw from Milgram and Kishino’s key taxonomy (1994) emphasizing the continuum of experiences that range from where a computer adds to a learner’s reality with overlays of information, or a computer experientially transports a learner to a different place and time by manipulating sight and sound.
VR Design Model
three different design models (see Figure 3): the ADDIE Design Model (Branson, 1978), Design Thinking (Brown & Wyatt, 2010) from user experience (UX), and the 3D Learning Experience Design Model (Kapp & O’Driscoll, 2009).
Serrat (2008) defines storytelling as “the vivid description of ideas, beliefs, personal experiences, and life-lessons through stories or narratives that evoke powerful emotions and insights” (p.1).
The foundational theory for most XR experiences is experiential learning theory. In cases where users create within XR, constructivist learning theory also applies.
XR experiences can include a story arc (See Appendix D), a tutorial of user affordances, intentional user actions, and place the user into first or third person experiences (Spillers, 2020).
The use of AR/VR in educational settings is on the rise, paving the way for new careers and a workforce trained to embrace technology.
If projections stay on track, the global spending on educational AR/VR is expected to rise from $1.8 billion to $12.6 billion over the next four years.
the International Data Corporation (IDC) released a report indicating that the pandemic has fueled an impressive forecast of worldwide expenditures on AR/VR, which are expected to grow from $12 billion in 2020 to $72.8 billion by 2024.
rom completing spinal surgery to training at a high-tech facility, such as the University of Nebraska Medical Center’s Davis Global Center, which has AR/VR and holographic technologies among its many offerings.
They’ve had huge success in China, but China is a pretty unique market. In late 2019, they expanded to Korea
the company is partnered with Deutsche Telekom and LG Uplus in Germany and Korea respectively, so it’s not too far a stretch to wager that the US branches of those carriers (T-Mobile and LG Electronics) will form at least part of their US distribution channels.
Finch Technologies makes body-tracking wearable devices to serve as controllers for XR applications and experiences including the recently announced FinchRing (this is the promised cherry on top).
The finger-worn hand tracker is designed to enable convenient and discreet 6-degree-of-freedom controls.
Tagesschau 2025 – an application based on a German news program that will present, among other features, holographic weather forecasts;
Lischer-Katz, Z. (2020). Archiving experience: An exploration of the challenges of preserving virtual reality. Records Management Journal, 30(2), 253–274. https://doi.org/10.1108/RMJ-09-2019-0054
Virtual reality (VR) technologies are complex configurations of computer hardware, software and file formats
In the context of digital materials, and all materials, given a long-enough time scale, preservation is an ongoing process, rather than a singular event that ensures eternal access to information.
alternative perspectives, e.g. phenomenology and media materiality, are necessary for
addressing the archival challenges of VR.
Since 1992 I have been teaching a course on how to write electronics section. My students include freshman, writing majors, and media lab graduate students.
As I watch the yearly growth in ingenuity among my students, I find myself anticipating a new kind of storyteller, one who is half hacker, half bard. The spirit of the hacker is one of the great creative wellspring Safari time, causing the in animate circuits to sync with ever more individualized and quirky voices; the spirit of the bard is eternal and irreplaceable, telling us what we are doing here in about we mean to one another.
Set 600 years from now, describes a society that science has dehumanized by eliminating love, parenthood, and the family in favor of generating engineering, test tube delivery, and state indoctrination. Books are banned, and science has come up with a substitute form of storytelling to delete the masses.
p. 20 Ray Bradbury Fahrenheit 451
p. 22 this accounts of a digital dystopia but eroticize and demonize the computer. Cyberpunks surfers are like Cowboys on the new frontier motorcycle hoodlums with a joystick in their hand instead of a motorcycle between their legs. They are outlaw pirates on an endless voyage of exploration throughout the virtual world riding and plundering among the invisible data hoards of the world in many states by the stronger pirate parents who reach in and reprogram their minds.
p. 22 William Gibson Neurmancer
p. 28 The Harbinger on the the Holodeck
The technical in economic cultivation of this freestyle new medium of communication has led to several new varieties of narrative entertainment. This new storytelling formats very from the shoot them up video game in the virtual dungeons of Internet role-playing games to the post modern literary hypertext. This wide range of narrative art holds the promise of a new medium of expression that is S varied SD printed book or the movie picture.
Books printed before 1501 or cold incunabula; the word is derived from the Latin for swaddling clothes and is used to indicate that this books are the work of a technology still in its infancy.
The garish video games in tangled websites of the current digital environment or part of a similar period of technical evolution, part of a similar struggle for the conventions of coherent communication.
p. 29 now, in the incunabular days of the narrative computer. We can see how 20th century novels, films, and please have been steadily pushing against the boundaries of linear storytelling. We therefore have to start our survey of the harbingers of the holiday back with a look at multiform stories, that is, linear narrative straining against the boundary of pre-digital media like a two dimensional picture trying to burst of its frame.
p. 30 The multiform story
Frank Capra’s It’s a wonderful life
p. 34 Robert Zemeskis Back to the Future
p. 35 Harold Ramis’s Groundhog Day
Multi forms stories often reflect different points of view of the same event. p. 37 Kurosawa Rashomon, the same crime is narrated by four different people: a rape victim; her husband, who is murdered; the bandit who attacked them; and a bystander.
p. 37 Milorad Pavic’s Dictionary of the Khazars
p. 37 Multi form narrative attempts to give a simultaneous form to this possibilities, to allow us to hold in our minds at the same time multiple contradictory alternatives.
p. 38 active audience
When the writer expands the story to include multiple possibilities, the reader assumes a more active role.
p. 40 although television viewers have long been accused of being less active engaged in readers or theatergoers, research on thin culture provides considerable evidence that viewers actively appropriate the stories of their favorite series. In addition to sharing critical commentary and gossip, fans create their own stories by taking characters and situations from the series and developing them in ways closer to their own concerns.
p. 42 role playing games or theatrical in a non-traditional but thrilling way. Players are both actors and audience for one another, and events the purple tree often have the media seat of personal experience.
p. 43 Live theater has been incorporating the same qualities of spontaneity and audience involvement for some time.
p. 43 MUDs have allowed distant players on the Internet to share a common virtual space in which they can chat with one another in real time. A the social psychologist Sherry Turkle has persuasively demonstrated, mods are intensely “evocative” environments for fantasy play that allow people to create and sustain elaborate fictional personas.
p. 44 movies three dimensions
p. 51 dramatic storytelling in electronic games
p. 55 story webs
p. 59 computer scientist as storytellers
p. 65 Chapter 3 From Additive to Expressive Form
beyond multimedia
Sept 28, 1895 Arrival of the Train at La Ciotat Station
p. 66 photoplays
p. 67 one of the lessons we can learn from the history of film is that additive formulations like photo play or the contemporary catshall ‘multimedia” or a sign that the medium is in an early stage of development and it is still depending on the format derived from earlier technologies instead of exploiting its own expressive power. Today the derivative mindset is apparent in the conception of cyber space is the place to view “pages” of print or “clips” of moving video end of cedar rooms is offering “extended books.”
p. 60 ELIZA, 1966 Joseph Weizenbaum
p. 71 the four essential properties of digital environments
Digital environments are procedural
Digital environments are participatory
Digital environments are spacial
Digital environments are encyclopedic
p. 90 Digital structures of complexity
p. 95 part to the aesthetics of the medium
chapter 4 immersion
definition
The experience of being transported to an elaborate please simulator please it’s pleasurable in itself regarding of the fantasy contact. we Refer to this experience as immersion. Immersion is a metaphorical term derived from the physical experience of being submerged in water. We seek the same feeling from a psychologically immersive experience that we do from a plunge in the ocean or swimming pool: the sensation of being surrounded by a completely other reality, as different is water is from air, that takes over all of our attention, our whole perceptual apparatus.
p/ 99 entering the enchanted place
my note: ghost in the machine The computer itself, even without any fantasy content, is an enchanted object. Sometimes it can act like an autonomous, animate being, sensing it’s environment in carrying out internally generated processes, yet it can also seem like an extension of our own consciousness, capturing our words through the keyboard in displaying them on the screen as fast as we can thank them.
p. 110 the active creation of belief
In digital environments we have new opportunities to practice this active creation of belief. For instance, in an interactive video program set in Paris that may research group designed in the 1980s for language learners, we included a working telephone, represented by a photograph of a phone who’s keypad could be clicked on .
p. 112 structuring participation with a mask .
p. 119 regulating arousal According to Winnicott, “the pleasurable element in playing Carris whit eight employee Kasian that the instructional a razzle is not excessive”; that is, the object of the imaginary world should not be too enticing, scary, or real let the immersive trance be broken. This is true in any medium. If a horror movie is too frightening, we cover our eyes or turn away from the screen.
p. 126 chapter Agency
Agency is the satisfying power to take meaningful action and see the results of our decisions and choices. We expect to feel agency on the computer when we double click on a file and seat open before us or when we enter numbers in a spreadsheet and see the totals readjust. However, we do not usually expect to experience agency within a narrative environment.
p. 129 the pleasures of navigation
One form of agency not dependent on the game structure yet characteristic of digital environment is spatial navigation. The ability to move through virtual landscapes can be pleasurable in itself, independent of the content of the spaces.
p. 130 the story of the maze
the adventure maze embodies a classic fair-tale narrative of danger and salvation. as a format for electronic narrative, the maze is a more active version of the immersive visit (chapter 4).
p. 134 Giving Shape to Anxiety
p. 137 The Journey Story and the Pleasure of Problem Solving
Among a myriad of other definitions, Noor (2016) describes Virtual Reality (VR) as “a computer generated environment that can simulate physical presence in places in the real world or imagined worlds. The user wears a headset and through specialized software and sensors is immersed in 360-degree views of simulated worlds” (p. 34).
Noor, Ahmed. 2016. “The Hololens Revolution.” Mechanical Engineering 138(10):30-35.
Weiss and colleagues wrote that “Virtual reality typically refers to the use of interactive simulations created with computer hardware and software to present users with opportunities to engage in environments that appear to be and feel similar to real-world objects and events”
Weiss, P. L., Rand, D., Katz, N., & Kizony, R. (2004). Video capture virtual reality as a flexible and effective rehabilitation tool. Journal of NeuroEngineering and Rehabilitation, 1(1), 12. https://doi.org/10.1186/1743-0003-1-12
Henderson defined virtual reality as a “computer based, interactive, multisensory environment that occurs in real time”
Rubin, 2018, p. 28. Virtual reality is an 1. artificial environment that’s 2. immersive enough to convince you that you are 3. actually inside it. ”artificialenvironment ” could mean just about anything. The photograph is an artificial environment of video game is an artificial environment a Pixar movie is an artificial environment the only thing that matters is that it’s not where are you physically are. p. 46 “VR is potentially going to become a direct interface to the subconscious”
p. 225 Virtual reality: the illusion of an all-enveloping artificial world, created by wearing an opaque display in front of your eyes.
From: https://blog.stcloudstate.edu/ims/2018/11/07/can-xr-help-students-learn/ : p. 10 “there is not universal agreement on the definitions of these terms or on the scope of these technologies. Also, all of these technologies currently exist in an active marketplace and, as in many rapidly changing markets, there is a tendency for companies to invent neologisms around 3D technology.” p. 11 Virtual reality means that the wearer is completely immersed in a computer simulation.
There is no necessary distinction between AR and VR; indeed, much research
on the subject is based on a conception of a “virtuality continuum” from entirely
real to entirely virtual, where AR lies somewhere between those ends of the
spectrum. Paul Milgram and Fumio Kishino, “A Taxonomy of Mixed Reality Visual Displays,” IEICE Transactions on Information Systems, vol. E77-D, no. 12 (1994); Steve Mann, “Through the Glass, Lightly,” IEEE Technology and Society Magazine 31, no. 3 (2012): 10–14.
++++++++++++++++++++++
Among a myriad of other definitions, Noor (2016) describes Virtual Reality (VR) as “a computer generated environment that can simulate physical presence in places in the real world or imagined worlds. The user wears a headset and through specialized software and sensors is immersed in 360-degree views of simulated worlds” (p. 34). Weiss and colleagues wrote that “Virtual reality typically refers to the use of interactive simulations created with computer hardware and software to present users with opportunities to engage in environments that appear to be and feel similar to real-world objects and events.” Rubin takes a rather broad approach ascribing to VR: 1. artificial environment that’s 2. immersive enough to convince you that you are 3. actually inside it. (p. 28) and further asserts “VR is potentially going to become a direct interface to the subconscious” (p. 46). Most importantly, as Pomeranz (2018) asserts, “there is not universal agreement on the definitions of these terms or on the scope of these technologies. Also, all of these technologies currently exist in an active marketplace and, as in many rapidly changing markets, there is a tendency for companies to invent neologisms.” (p. 10)
Noor, Ahmed. 2016. “The Hololens Revolution.” Mechanical Engineering 138(10):30-35.
Rubin, P. (2018). Future Presence: How Virtual Reality Is Changing Human Connection, Intimacy, and the Limits of Ordinary Life (Illustrated edition). HarperOne.
Weiss, P. L., Rand, D., Katz, N., & Kizony, R. (2004). Video capture virtual reality as a flexible and effective rehabilitation tool. Journal of NeuroEngineering and Rehabilitation, 1(1), 12. https://doi.org/10.1186/1743-0003-1-12
XR Storytellers: Learners Making Immersive Stories
LIVE NOW
Thursday, May 7, 2020 from 1:30 PM to 2:30 PM (CDT)
PRESENTATION
Our team will share lessons learned in collaborating to create immersive experiences that accelerate STEM education. Find out how students achieve classroom learning objectives by designing AR experiences. Watch a demonstration of how an immersive scientific story is co-created by students and teachers in a virtual learning environment. Explore novel techniques for supporting learners to demonstrate understanding and share knowledge using spatial technologies and storytelling principles. We invite guests to share their questions and perspectives on the possibilities and limitations of XR storytelling to facilitate relational connections to curriculum and instruction.
PRESENTERS: Sarah Cassidy | Janelle LaVoie | Quincy Wang | Poh Tan
We are a team of VR learners from the University of Saskatchewan and Simon Fraser University in Canada. Our research explores innovative uses of immersive technology for STEM education and pro-social change.
During Lab work on Jan 28, we experienced Video 360 cardboard movies
let’s take 5-10 min and check out the following videos (select and watch at least three of them)
F2F students, please Google Cardboard
Online students, please view on your computer or mobile devices, if you don’t have googles at your house (you can purchase now goggles for $5-7 from second-hand stores such as Goodwill)
Both F2F and online students. Here directions how to easily open the movies on your mobile devices:
Copy the URL and email it to yourself.
Open the email on your phone and click on the link
If you have goggles, click on the appropriate icon lower right corner and insert the phone in the goggles
Open your D2L course on your phone (you can use the mobile app).
Go to the D2L Content Module with these directions and click on the link.
After the link opens, insert phone in the goggles to watch the video
Videos: While watching the videos, consider the following objectives:
– Does this particular technology fit in the instructional design (ID) frames and theories covered, e.g. PBL, CBL, Activity Theory, ADDIE Model, TIM etc. (https://blog.stcloudstate.edu/ims/2020/01/29/im-690-id-theory-and-practice/ ). Can you connect the current state, but also the potential of this technology with the any of these frameworks and theories, e.g., how would Google Tour Creator or any of these videos fits in the Analysis – Design – Development – Implementation – Evaluation process? Or, how do you envision your Google Tour Creator project or any of these videos to fit in the Entry – Adoption – Adaptation – Infusion – Transformation process?
– how does this particular technology fit in the instructional design (ID) frames and theories covered so far?
– what models and ideas from the videos you will see seem possible to be replicated by you?
Find one F2F and one online peer to form a group.
Based on the questions/directions before you started watching the videos:
– Does this particular technology fit in the instructional design (ID) frames and theories covered. e.g. PBL, CBL, Activity Theory, ADDIE Model, TIM etc. (https://blog.stcloudstate.edu/ims/2020/01/29/im-690-id-theory-and-practice/ ). Can you connect the current state, but also the potential of this technology with the any of these frameworks and theories, e.g., how would Google Tour Creator or any of these videos fits in the Analysis – Design – Development – Implementation – Evaluation process? Or, how do you envision your Google Tour Creator project or any of these videos to fit in the Entry – Adoption – Adaptation – Infusion – Transformation process?
– how does this particular technology fit in the instructional design (ID) frames and theories covered so far?
– what models and ideas from the videos you will see seem possible to be replicated by you?
exchange thoughts with your peers and make a plan to create similar educational product
Evaluate the ability of the game you watched to be incorporated in the educational process
Assignment: In 10-15 min (mind your peers, since we have only headset), do your best to evaluate one educational app (e.g., Labster) and one leisure app (games).
Use the same questions to evaluate Lenovo DayDream:
– Does this particular technology fit in the instructional design (ID) frames and theories covered, e.g. PBL, CBL, Activity Theory, ADDIE Model, TIM etc. (https://blog.stcloudstate.edu/ims/2020/01/29/im-690-id-theory-and-practice/ ). Can you connect the current state, but also the potential of this technology with the any of these frameworks and theories, e.g., how would Google Tour Creator or any of these videos fits in the Analysis – Design – Development – Implementation – Evaluation process? Or, how do you envision your Google Tour Creator project or any of these videos to fit in the Entry – Adoption – Adaptation – Infusion – Transformation process?
– how does this particular technology fit in the instructional design (ID) frames and theories covered so far?
– what models and ideas from the videos you will see seem possible to be replicated by you?
