top of page

XR in Healthcare

1. Work Activity Affinity Diagram

2. Ideation & Brainstorming

3. Rapid Prototyping

4. HCD Theories

5. Systematic Review

6. Affordances & Embodiment

7. Prototyping & AR

8. Strengths & Challenges

1. Work Activity Affinity Diagram

Teambuilding:

Our process began by forming an interdisciplinary  team of 7 students from computer science, creative technologies and systems engineering to encourage innovative thinking, as emphasized in collaborative design principles [1]. 

Topic:

We chose immersive technologies for nursing education as our project topic, leveraging Extended Reality (XR) for realistic training aligned with technology-enhanced learning trends [10]. 

WAAD:

We started by creating a Workflow Activity Affinity Diagram, organizing medical activities like airway intubation and identifying pain  points faced in emergency trauma situations to inform the design. It helped us design features like adjusting the difficulty and environmental stress factors.

2. Ideation & Brainstorming

Next, we engaged in creative ideation techniques like group sketching, noting ideas for features like AI assistant bots, motion-controlled interactions, and collaborative training modules with multi-user support. This aligned with generative design thinking to produce solutions centered around users – in this case, nursing students.

3. Rapid Prototyping

We incorporated rapid digital prototyping using Figma to visualize conceptual designs and workflow. These ranged from AI assistant bots with attributes like personalized names and languages to specialty medical devices and motion-controlled interactions using virtual reality gloves. This enabled iterative testing using prototypes and wireframes, allowing for refinements like including authentication access and an intuitive interface with icons previewing module content.

4. HCD Theories

We analyzed concepts like activity theory, and place and space, to grasp the contextual nuances in XR nursing education.

Activity Theory:

If I were baking a cake, it would be more than about just mixing the ingredients and putting it in the oven. It would include factors such as why I want to bake a cake (for someone’s birthday), where I am baking it (my kitchen), have I baked it before, what does a cake symbolize in my culture, etc. Activity Theory takes all of this into consideration for a wholesome perspective of the process as a whole and not segmented tasks in isolation. It has 6 primary components:

  1. Subject: The person or group engaged in the activity (you, in our cake-baking example).

  2. Object: The goal or motive of the activity (a baked cake).

  3. Tools: The physical or symbolic instruments used to achieve the goal (mixer, recipe, oven).

  4. Rules: The norms, conventions, or constraints that guide the activity (baking times, the sequence of adding ingredients).

  5. Community: The larger group with a shared interest in the activity (all the people who love cakes or, more broadly, all who bake).

  6. Division of Labor: The distribution of tasks among community members (if you were baking with friends, one might mix while another decorates).

Place and Space:

Place is a cultural phenomenon that emerges from the patterns of use and behavior of people in a space. Space is the underlying opportunity for creating places.Space is often associated with physical dimensions, segmentation, subdivision, boundaries that provide structure enabling, or disabling action, movement or interaction. We found the following are the concepts of Places and Spaces that could be incorporated into HealXR to offer better learning experience:

  1. Create Media Spaces (Environments)

  2. Emotional place

  3. Turn Space into Place

  4. Adapting Technology

  5. Social and Cultural Dimensions

  6. Hybrid Spaces

  7. Social Interaction

  8. Digital Spaces

  9. Dynamic and Adaptable Space

  10. Spatial Metaphors

  11. Boundary Negotiation

5. Systematic Review

We searched research databases like SCOPUS to analyze existing immersive healthcare education solutions and identify potential gaps. Outlining inclusion criteria and measures to reduce bias was integral for an effective narrative review. Conducting searches across databases provided insights from 613 articles, directing evidence-based decisions in areas like modular difficulty adjustments and privacy compliance.

6. Affordances & Embodiment

Affordances:

Affordances refer to the design features that facilitate and enhance user interaction and learning within a virtual environment. These affordances are crafted to support cognitive, physical, sensory, functional, and emotional aspects of user experience, ensuring an immersive, accessible, and effective learning tool for nursing students.


Interaction Cycle:

The interaction cycle is a conceptual framework that describes the continuous process through which users engage with a system, encompassing the stages of goal formation, execution of actions to achieve those goals, interpretation of the system's feedback, and evaluation of outcomes relative to their objectives.


Embodiment:

Embodiment refers to how the platform leverages the principles of embodied cognition to enhance nursing education through virtual and augmented reality experiences. This approach recognizes that cognitive processes are deeply rooted in the body's interactions with the world.

7. Prototyping & AR

Our high-fidelity designs made in Figma were implemented and transformed into an interactive prototype and deployed into a Meta Quest 3 headset.

8. Strengths & Challenges

Strengths:

  1. Our team's diversity across cultures and expertise in various fields drove our success by fostering inclusive and comprehensive design solutions.

  2. Effective ideation and brainstorming on AR and VR concepts enriched our design process, enabling exploration of a wide range of possibilities.

  3. Using Figma for prototyping and wireframing facilitated iterative testing and refinement, ensuring HealXR was user-friendly and met nursing students' needs.

  4. Incorporating realistic spatial contexts into HealXR's UX design provided nursing students with a practical and immersive learning experience in a simulated trauma center.

Challenges:

  1. Team diversity enhanced perspectives but also added complexity and delays in decision-making due to differing cultural and professional viewpoints.

  2. While Figma and other tools aided prototyping, varying expertise levels and the need for specialized VR/AR hardware introduced inconsistencies and increased costs.

  3. Translating theoretical concepts like Activity Theory into practical UX elements was challenging, often necessitating extensive revisions and prolonging development.

  4. The time constraints of HealXR as a class project limited deep dives into processes, leading to a broader but more superficial exploration of design aspects.

Tools Practiced...

Dall-E 2

Miro

Overleaf

Figma

Credits...

Harshal Pilania, Researcher / User Interface Developer

Joshua Okoro, Researcher / Figma / AR Mockup

David Jones, Team Leader / Computer Science

Sunday Ubur, Researcher / Computer Science

Ajit Gopal, Researcher / Industrial & Systems Engineering

Ankit Sangwan,  Researcher / Industrial & Systems Engineering

Afe Akusu, Researcher / Creative Technologies

Immersive Installation

AR enabled Board Game

XR in Healthcare

for D-Day Memorial

Why A.I. Art stereotypes?

Azadi: Raise your Voice

for Smithsonian Museum

Building a Dog-House

for VT Corps of Cadets

A Fantastical World

Bloom: The urge to help

Campervan Concept

Playing with P5js

bottom of page