Head-mounted Displays & Smart Glasses in Healthcare

INTRODUCTION

With the global COVID-19 pandemic forcing healthcare systems around the world to reimagine how healthcare can be delivered remotely and safely, we have seen a rapid proliferation of telehealth and virtual care solutions. In reality, COVID-19 has provided a long-awaited and much-anticipated tipping point for the virtual care industry, which promises to continue to grow exponentially long after the crisis has passed. Indeed, many are hoping that far from being a temporary workaround, the transition to Virtual Care could potentially provide a solution to underlying challenges that have been endemic within the system for some time, including provider shortages, access issues for remote and underserved communities, clinician burnout, burgeoning costs, and declining operating margins.And all of this against a background of declining population health. ^1-4 Could virtual care be the silver bullet?

The global pandemic has not only highlighted challenges with traditional healthcare delivery but has also cast a spotlight on the medical education and medical device training industries. Medical students are struggling to get access to clinical environments for hands-on experience, and global travel restrictions are hampering the ability of medical device manufacturers to train and certify their staff on essential medical equipment.

As emerging technologies hold the promise of improving healthcare efficiency and effectiveness as well as enhancing patient and provider experience and engagement, new hardware and software applications have been flooding the market.

Collaborating with peers and extracting and exchanging actionable information through smartphones and tablets are commonplace in the private sector. Although healthcare has been relatively slow to adopt emerging technologies, they now promise to transform the healthcare landscape.⁶ From clinical care to medical education and training, Head-Mounted Displays (HMDs) and Smart Glasses (SGs) provide similar functionality to smartphones but are tailored to specific use cases in healthcare and provide the additional convenience of hands-free operation. The market for these devices is forecasted to grow at a CAGR of 36.8% between 2020 and 2023.⁷ With the increase in popularity of HMDs and SGs in healthcare, more peer-reviewed data on their use in care delivery and medical education is becoming available.

This paper examines the utilization of Head-Mounted Displays and Smart Glasses in healthcare and summarizes recent findings. It also explores the technologies that complement HMDs and SGs, highlights current and potential use cases, technology requirements, and implementation challenges, and points to the enormous potential of extending HMDs and SGs into the daily workflow of care delivery and medical education and training.

What Are Head-Mounted Displays and Smart Glasses?

Head-Mounted Displays (HMDs) and Smart Glasses (SGs) are categories of wearable technology. They are compact devices, placed over the head (head-mounted) equipped with a small display in front of one or both eyes, or worn like standard eyeglasses. Although some HMDs display only computer-generated imagery (virtual reality), other HMDs and SGs refer to see-through displays that seamlessly provide critical information within the user’s vision but without limiting the view (augmented or mixed realities). SG displays are also head-mounted or embedded in the glass itself
as a transparent heads-up display or augmented reality overlay. ⁸

In addition to their ability to display projected digital images, HMDs und SGs can capture information from the environment through cameras, thermal or GPS sensors, and microphones, and from internal sources such as connected devices or computers. Wireless technologies, like Wi-Fi, Bluetooth, LTE or 5G, and GPS, can support the utilization of a variety of mobile applications and online platforms.

Users of HMDs and SGs can access information, record, save and exchange audio and video files, make phone calls, or conference with peers through common platforms like Teams, Skype or Zoom. The navigation of the devices and their software application is increasingly occurring through voice commands, while other models may use touchpads or buttons, or track hand gestures or eye movement.

The Role of Advanced Technologies for HMDs and SGs in Healthcare

Extended Realities

On the reality-virtuality continuum, Head-Mounted Displays and Smart Glasses provide the option to extend the real-world view on one end of the spectrum to a completely Virtual Reality (VR) on the other. VR provides a 100% digital and computer-generated environment and therefore replaces the real world. Augmented Realities (AR) and Mixed Realities (MR), on the other hand, refer to the space between both ends of the spectrum, including a digital overlay of 2D or 3D imagery on the real-world view. Two minimum components necessary to create 3D Extended Realities (XR), which users can interact with, are position tracking of the user’s eyes or head, and the visualization of the virtual elements from the user’s perspective. ⁹Emerging technologies including Robotics, Deep Learning, Machine-Learning, Natural Language Processing, and Artificial Intelligence (AI) are contributing to the continuous refinement of XR applications and by extension, HMDs and SGs, and promise to disrupt and transform all areas of the healthcare landscape.

In medicine, early popular NASA applications of XR focused predominantly on displaying complex anatomy and planning and training for surgical procedures. Today, the XR domain comprises real-time medical education and proctoring, planning and AI-guided surgery, care team collaboration, and multiple therapeutic interventions.^{9, 10}

A review of academic and media publications from the last decade shows that, while both VR and AR can provide significant benefits to training and care provision for clinicians, medical VR applications are more often targeted at patients as the primary user. At the same time, AR has seen a broader application targeting clinicians as the primary user through their adoption of HMDs and SGs.⁹ Thus, more publications refer to VR applications in the field of “Knowing,” such as practicing communication skills, while the overlay technologies (AR) often refer to the field of “Showing,” like training or testing of practical clinical skills or care team collaboration.^{9, 11, 12}

Augmented Intelligence

Hospitals are already advancing their use of artificial intelligence – or “augmented intelligence” as the American Medical Association (AMA) prefers to call it – and hospitals are encouraged to develop services that leverage it. We are already seeing examples of AI for chatbots, machine-learning algorithms for imaging or patient scheduling, or models that predict adverse events from hospitalized patient data. Referring to physicians’ frustration with electronic medical records (EMRs) and their usability, the AMA emphasizes the demand for clinician-centered models in augmented intelligence applications. ¹³That is exactly where HMDs and SGs may find their niche role. They can facilitate access to EMRs and AI-predictive data including clinical decision support tools, and promote care team collaboration as well as facilitate ongoing medical training.

How COVID-19 accelerated augmented intelligence in healthcare

During the pandemic, two major New York City hospitals utilized AI-applications to enhance physicians’ decisions on when and how patients should transfer into another phase of care or be sent home. ¹⁴ A premier academic medical center in NYC developed and tested an AI-algorithm to predict the hospitalized patients’ risk of suffering from adverse events. Similarly, Epic, a major vendor of electronic health record software, validated its new AI-model determining which coronavirus patients may become severely ill with data from over 16,000 hospitalized American patients. And outside of hospitals, AI-enabled risk factor models helped to track uninfected patients that might be susceptible to COVID-19 complications if they contracted the virus. ¹⁴Meanwhile, a previously developed AI program at the University of California, designed to help doctors identify pneumonia, was quickly deployed and applied to over 6,000 cases when the pandemic occurred. Doctors reported that the tool that color-dots X-ray images at possible damaged lung areas influenced their decision-making about a third of the time. ¹⁴

Key HMD and SG Players in Healthcare

Both Magic Leap and Microsoft’s HoloLens offer 3D AR technology for medical purposes. A hand-held controller navigates the Magic Leap device and its software. Because it does not provide a HIPAA-compliant software, the wearable spatial computer finds its applicability only in medical simulation and training, as well as 3D imaging and surgical planning.¹⁵ HoloLens’ users can navigate the device and the software through voice commands and hand and eye-tracking and can utilize multiple apps and platforms for medical training or holographic navigation through surgical planning and procedures. Tested for accuracy and feasibility in clinical settings, peer-reviewed data points to the fact that the limited field of vision and the HoloLens navigation accuracy are not yet sufficient for clinical use.^{16, 17}

The vast majority of HMD/SG research has to date focused on Google Glass.^{6, 18} First introduced in 2012, it was discontinued in 2015 because it lacked clarity on who the user was and why this product existed.¹⁹ Its deployment in healthcare was triggered by Augmentix, a software firm that developed an app for Google Glass that provided clinicians the option to live-stream surgery and patient visits, as well as access and record a patient’s EMR in a HIPAA-compliant manner. Multiple research projects on applicability in surgical and nonsurgical medical fields have followed.^{6, 20} Unfortunately, key limitations of Google Glass, like overheating, its relatively short battery life, low quality camera, and connectivity issues have curtailed its deployment in healthcare.⁶ However, an improved version, Google Glass Enterprise Edition 2, has been available since 2019.

Another AR-HMD gaining in popularity is the Vuzix M400. Vuzix partners with VSee, Skype, and Zoom for Healthcare to leverage its collaboration capabilities for training and remote specialty care. It recently reported the first successful knee replacement surgery using its head-mounted tablet in combination with AR technology.¹⁰

More recently, the voice-activated Hippo Virtual Care (HVC) headset and platform has been attracting attention from the medical community. HVC utilizes a military-grade AR head-mounted tablet from RealWear to focus solely on healthcare applications. Hippo offers its HMD in combination with a HIPAA-compliant software platform and cloud service. The hands-free device enables medical staff to communicate in real-time with remote colleagues, pull up medical records, and automatically access files and imaging during patient examinations, procedures, consultations, and rounding.^21-23

With the onset of COVID-19, clinicians around the world are increasingly recognizing the immense potential of HMDs such as Hippo Virtual Care to facilitate remote collaboration in emergencies, train medical staff, and guide treatment procedures. Using the HVC platform has effectively reduced healthcare workers’ exposure to infection and saved the time and cost of wearing personal protection equipment (PPE).^22-24

The Unique Demands on HMDs & SGs in Healthcare

The most important considerations for healthcare providers when it comes to using Head-mounted Displays and Smart Glasses in healthcare are patient data security and clinical usefulness, i.e., the degree to which HMDs or SGs can materially enhance the clinician’s performance.^{26, 27} HMDs and SGs must provide connectivity and interoperability with various hospital systems, software, and communication platforms. Bundled software that enables the user to live-stream audio or video or exchange patient data must be HIPAA compliant to conform to privacy and security rules in healthcare settings.

The most practical feature of HMDs and SGs in both surgical and nonsurgical settings is their ability to operate completely hands-free, using voice commands and speech recognition. This functionality enables the clinician to focus solely on the patient without having to turn away and consult a laptop or tablet, thereby disrupting the medical examination or procedure.^{12, 23, 28}

Active noise cancellation microphones and internal loudspeakers are also critical for wearable technology deployment in healthcare. Being able to hear and communicate in austere situations, such as loud noises from traffic or alarms in first responder emergency scenarios is critical.^{28, 29}

Moreover, a less obtrusive appearance of the clinician wearing the device and the naturalness of communication benefit both the caregiver’s willingness to use the device and the patient’s perception and acceptance of the technology.^{30, 31}

The Benefits of Using HMDs & SGs

Clinicians’ rising workload, staff shortages, and the growing administrative burden in healthcare are all contributing to a reduction in time caring for patients and increasing staff burnout.^{2, 32} Upholding quality of care standards is becoming increasingly challenging and is one of the reasons why 52M American patients per year are affected by medical error.^{3, 33} At the same time, hospitals are continuing to see declining operating margins, hastening the call for improved operational efficiencies.^{4, 5} HMDs and SGs can help clinicians overcome some of these challenges by addressing the root causes of these problems.

Increased collaboration leads to confident decision making

Increasingly, HMDs and SGs are being developed that support HIPAA-compliant collaboration platforms and encrypt patient data to enable broader utilization and applicability in healthcare. Clinicians are recognizing their deployment as a highly useful tool to interact with colleagues located remotely or outside the operating or exam room. ³⁴ For example, a recently published study reported that surgical non-experts receiving AR-telementoring from specialists during leg surgeries performed with fewer errors, better performance scores, and increased confidence compared with the study participants that reviewed the procedure beforehand only. ³⁵And study participants simulating endotracheal intubation using critical information displayed by AR glasses showed improved adherence to evidence-based guidelines compared to those without AR glasses. ¹² HMDs and SGs can not only enable more accurate diagnoses and treatments but can also improve care team coordination. Real-time communication and seamless file exchange between providers along the continuum of care can accelerate patient flow through the hospital or different levels of care and enhance patient satisfaction and provider effectiveness.

Enhanced patient monitoring

HMDs and SGs enable clinicians to monitor their patients through their peripheral vision. ^{36, 37} This is extremely important in anesthesia, ICUs, and emergency medicine. Today’s state-of-the-art monitors lack the ability to provide real-time interpretation of a patient’s vital signs and thus promote a low degree of physician confidence. ^{36, 38} A 2019 multicenter comparative study provides empirical proof that an avatar-based HMD is superior to standard monitoring, especially in combination with auditory signaling. Physicians recognized significantly more changes in patient status and perceived increased certainty in their diagnoses. ³⁶

Remote monitoring efforts are also accelerating outside of the clinical setting. Under FDA guidance, Apple Watch’s electrocardiogram (ECG) app supports clinicians’ diagnosis of certain arrhythmias and related conditions. ³⁹ Patients can easily message a PDF file of their readings and symptoms directly to their physician’s HMD or SG.

Improved care coordination

Timely collaboration of all care team members is vital to manage complex and chronically ill patients, helping to avoid double diagnosis and duplication of treatment, and thus promote the desired health outcomes. HMDs and SGs will enable clinicians to enter and pull from EMR data quickly and to collaborate with other caregivers or relatives on the fly via messaging, or audio or video connection. ²² In emergency situations, ED staff can see “through the eyes” of first responders when they triage patients, plan resources before patients even arrive at the hospital, and thus avoid critical time delays. ^{2. 3}System-integrated medical devices like EKGs or ultrasound devices can send patient data or images directly to physicians’ head-mounted devices for faster patient examination and evaluation. ⁴⁰ These examples serve to illustrate that the deployment of HMDs and SGs can vastly improve patient flow from one care setting to another, thereby increasing clinical efficiency and effectiveness and improving patient safety, satisfaction and health outcomes.

A patient-centered care approach: the emergence of “Polymorphic Medicine”

Most health systems have already introduced monitoring systems to collect, process, and analyze data from portable sensing devices and IoMT (Internet of Medical Things) used by patients with chronic diseases. Data from patient glucometers, oxygen or blood pressure sensors or wearables, deposited in a connected health platform, can help HMD users to assess more dimensions of an individual’s health status beyond standard snapshot examinations. ⁴¹ This applies not only to clinical settings like hospitals, rehabilitation, or home care settings, ⁴² but also for health promotion and disease prevention on an individual level. ⁴³Hippo Technologies, Inc., for example, believes that care delivery of the future will start with the individual in the home, introducing a concept they refer to as “Polymorphic Medicine.” It consists of three pillars: (1) Self-care promotes individual health awareness through education and leveraging patient-derived data to support and motivate healthier lifestyle choices. (2) Virtual care refers to telemedicine and remote care delivery in a place of the patient’s choosing; and (3) Physical care refers to traditional in-person care, benefiting from enhanced information access and clinical collaboration through the HVC platform. ^twenty-one

Improving clinicians’ health and safety

Integrating HMDs or SGs into clinical workflows and care pathways also provides tangible benefits to the user. Access to crucial information, peer support, and decision support tools such as clinical guideline checklists, as well as real-time EMR inputs, have been proven to enhance clinicians’ effectiveness and productivity. This can also lead to reduced stress levels that may prevent clinician burnout. ^{35, 36, 44}

Another advantage of using HMDs or SGs became abundantly clear during the COVID-19 pandemic. In more than 10 hospitals in Wuhan, the HVC platform transmitted live video exams of patients communicating vast amounts of information to clinicians and specialists outside the quarantine zone. Using the HVC headset and platform significantly reduced the number of clinical staff exposed to potential infection, and helped prevent the virus from spreading. ^{23, 25}

Multiple studies have emphasized that information displayed on HMDs and SGs is easy to read, clear, and understandable. They confirmed that even sustained usage of the devices does not lead to eye fatigue ^{12, 45} and that the perceived enjoyment of using new technology actually has a positive impact on users. ⁴⁶ An additional study reports preventive health benefits during microsurgery. Transferring the surgical microscope view to a connected HMD created a 3D image in the surgeons’ displays. Thus, their posture was much more flexible during the procedure, and their discomfort level – especially in the neck and shoulders – was significantly lower than in surgeons using a microscope with eyepieces. ⁴⁷

“After successful test runs in February 2020, the Catalan government and health agency will deploy the HVC platform for its first responders, like policemen and firefighters, to provide real-time medical assistance at emergency sites everywhere in Catalonia by 2025.”

Ideal in extreme situations and environments

HMDs and SGs are particularly useful in emergencies or under extreme conditions. Close collaboration and communication between first responders in the field and clinicians back in the emergency room can be potentially life-saving. In Norway, for example, the HVC platform is being used by paramedics under challenging conditions, including heavy snowfall. The head-mounted device enhances orientation in the field through its geolocation capability and enables paramedics to initiate necessary treatments by synchronizing with corresponding specialists in the hospital. Additionally, it improves operational preparedness in the emergency room before patients even arrive. ^21-23

Benefits during rescue missions are also reported by dog handlers of the German Red Cross Rescue Dog Unit. They visualized their team members’ locations on digital maps on their HMDs and were able to enhance team communication during their missions. ²⁸ Recently, Nature published an article that represents another step towards the adoption of AR-HMDs for providing specialist consultations remotely in austere situations. During simulated scenarios that included smoke and loud noises of gunshots and explosions, participants performed life-saving surgical procedures (cricothyroidotomy) under remote guidance by either audio-only or AR-HMD. Using AR-HMDs resulted in a significantly better overall performance and was associated with a safer and more successful surgery in the field.

“Most beneficial, the surgeon is not forced to look away from the surgical site, in contrast to common imaging techniques, and the hands-free device control mode allows the team to interact with the device even in sterile environments.”

Surgery

Among multiple medical disciplines, HMDs and SGs are most commonly being used in surgery. Increasingly, more clinicians are realizing that the ability to work in symbiosis with a computer can broaden the horizon of what is possible in surgery.

HMDs and SGs can expand the individual surgeon’s knowledge base tremendously. They can receive expert and second opinions or share their expertise themselves through high-quality real-time video and audio consultations. They can display procedure checklists, microscope visuals, training videos, EMR or medical images, and 3D anatomy animations, or overlay visuals for surgical planning and intraoperative navigation. ^{34, 49, 50}
Existing literature points to the fact that the increasing interest and use of AR-enabled HMDs or SGs can lead to improved safety and efficacy of surgical procedures. ⁴⁹

Non-surgical care provision

Other medical fields have also begun to implement AR- or VR-enabled Head-mounted Displays to support both patients and clinicians and deliver improved outcomes and experiences. Some of the more successful patient-centered deployments include assisting patients with motor or visual impairments, cognitive rehabilitation, developmental and psychiatric disorders, and pain and weight management. ^{6, 9, 51} Meanwhile, HMDs and SGs can support clinical users with medical education and proctoring, disaster relief, diagnostics, nursing, autopsies, and wound care, to name just a few potential use cases. ⁶

A growing number of health systems around the world are starting to implement HMDs and SGs in their routine processes across a wide range of use cases. In sub-Saharan Africa, the Hippo Virtual Care platform connects hospitalists to rural and remote locations and helps to train nurses in the field. ²² Meanwhile, in northern Spain, the independent region Catalonia plans to equip its police officers and fire workers with HMDs by 2025 to advance real-time physicians’ assistance during emergency situations. ⁴⁸ And as mentioned above, HMDs are already supporting the treatment and care coordination of COVID-19 patients around the globe. ^{14, 23, 25}

Transforming Medical Education

Major training gaps

The healthcare sector is lagging behind other industries when it comes to training practices. Research indicates that healthcare workers spend about 25 hours, a third less than the average, on training per year. And healthcare organizations’ annual spending for employee training is less than 50% compared to other industries. ⁵² Various hospital systems have adopted AI and video-based training platforms, in an attempt to meet the demand for learning opportunities for redeployment or continuing medical education, especially triggered by the 2020 pandemic. ^{52, 53}Live patient cases are a vital aspect of medical training, but unfortunately these opportunities are not being cultivated enough. Bedside rounding, for example, an essential point of care for hospitalized patients, has decreased more than 75% since the 1960s and now takes place in only about 18% of a caregivers’ time. ⁵⁴ Besides being a critical tool for patient-centered care, it always served as an effective way of teaching hands-on professional skills. The diminishing hands-on training opportunities for medical and nursing students is negatively impacting their ability to gain essential clinical skills, evolve their bedside manner, and build empathy. ⁵⁴Taking the estimated provider shortage, social distancing rules during the pandemic, and the increasing clinician workload into account, healthcare systems must find new, innovative approaches to address the apparent deficits in medical education.

HMDs can provide virtual access to a huge variety of care settings that students wouldn’t necessarily be exposed to, such as busy inner-city hospitals, remote rural community health centers, and field clinics in developing countries, war zones and disaster relief locations.

HMDs and SGs provide the answer

3D-VR technologies have been proven to be effective and offer an exciting learning environment for the study of endoscopic surgery and anatomy. ¹¹ Clinicians and students can inhabit the same virtual space and study mutually interactable objects in front of them, or the shared virtual environment itself. ⁹ In behavioral counseling, for example, 3D-VR training is assumed to be more effective than existing methods. Immersive simulations delivered through an HMD promote interpersonal skill development and empathic understanding regardless of the trainee location. ⁵¹

AR-enabled HMDs and SGs have been proven to be most useful in the delivery of practical skills. ^{11, 12} The overlay technology can be used for surgery guidance, like lumbar punctures or knee replacements, or practice of other complex clinical procedures, thereby improving the accuracy and success rate of the clinician. ^{10, 11, 55}

The real-time collaboration capabilities afforded by HMDs and SGs, however, perhaps provide the most significant potential in the medical education arena. Medical students, trainees or CME mentees “look and learn through the eyes of the physician” who wears the device, effectively replicating a one-on-one training scenario. HMDs can provide virtual access to a huge variety of care settings that students wouldn’t necessarily be exposed to, such as busy inner city hospitals, remote rural community health centers, and field clinics in developing countries, war zones and disaster relief locations.

Additionally, HMDs and SGs can be used to assess students through remote proctoring. This gives students the chance to get a feel for what practicing medicine will be like without someone looking over their shoulder while allowing teachers to assess their skills in real time. The ability to record procedures through “store-and-forward” functionality enables students and faculty to review and discuss clinical practice after the event, and keep an historical record of progress. This tele-mentoring option is proving to be highly effective in surgical and nonsurgical medical settings. ^{29, 35, 56}

Physician Adoption of HMDs & SGs

Physician adoption of HMDs and SGs continues to grow as technology companies like Hippo Virtual Care and Vuzix focus on specific healthcare applications and use cases.

For device adoption to be successful in healthcare, functionality must enhance clinical effectiveness, improve peer-to-peer collaboration, increase medical knowledge and education, and provide easy and timely access to information – all without interrupting their day-to-day clinical routine and interaction with the patient. ²⁶

In addition, they must provide intuitive navigation of the devices that does not interfere with the naturalness of gestures or conversations. Perhaps surprisingly, researchers discovered that it was not the clinician’s age but the technology innovation itself that is one of the most critical drivers for caregivers to adopt HMDs or SGs. ²⁷ When they were asked to evaluate such devices, they mostly pointed to their ease of use and learning, compatibility, and speech recognition capabilities as the most significant factors for adopting the technology in their daily routines. ²⁶However, privacy concerns sometimes prompted a negative response, which is why healthcare-focused solutions have recently accelerated enhancements around HIPAA compliance and data encryption. Hippo Virtual Care, for example, offers a military-grade customizable hardware-software packaged solution, guaranteeing compliance with all patient privacy and data security regulations including HIPAA, HL7, 508, FedRAMP and GDPR. This will eventually enable full interoperability with hospital systems in a safe and secure manner. ²¹

Patients’ Perception of HMDs & SGs

Improved quality of care

Statistics show that patients in the US wait, on average, at least 20 days for a specialty appointment, delaying much-needed therapy or medication, ⁵⁷ and thus possibly exacerbating their illness. Additionally, more than one in 20 patients in the US are harmed by medical errors across all healthcare settings. ³³ These delays in diagnosis and treatment, as well as medical errors, are preventable if clinicians take advantage of current wearable technologies. HMD and SG deployment has been proven to minimize errors and enhance health outcomes across a variety of care settings. ^{12, 26}They can provide faster access to second opinions or specialists, improve emergency management, and provide critical clinical decision support by enabling real-time access to medical information. For example, research studies pointed to significantly better results of inexperienced surgeons when they received remote help from specialists. ²⁹ Others reported 100% adherence to guidelines if clinicians see a checklist in their Head-mounted Display. ¹² And existing machine-learning algorithms, if made compatible with HMDs and SGs, can help clinicians to assess the risk of future events, like drug side effects, or support decision-making for patients in life-threatening situations. ⁵⁸Moreover, two reviews of the literature published between 2013 and 2019 highlighted the feasibility and increased effectiveness if patients actually use AR / VR-enabled HMDs or SGs as a tool for rehabilitation, physical therapy, or treatment of neurological disorders. ^{6, 59}

Saving time and money, and improving healthcare access

Not only do patients have to wait, but they also face high costs for physician appointments, aside from out-of-pocket expenses and insurance premiums. Data from the US Bureau of Labor Statistics showed that Americans spend, on average, 45 minutes on traveling and waiting time to be seen for their doctor’s appointment. Quantifying this time into average hourly wages leads to an astonishing economic impact of $ 89 billion a year. ⁶⁰ HMDs and SGs could significantly reduce this burden and also increase patient satisfaction if broadly implemented across all levels of healthcare settings; now patients do not necessarily have to visit a specialist as head-mounted devices can increase the capacity to provide specialty care in community settings.

Besides the monetary benefit for patients, HMDs and SGs also address shortcomings in access to healthcare. In the US, more than 15% of the population does not receive necessary medical care due to barriers like cost and geographic reach, especially in small metropolitan and rural areas. ^{61, 62} And because of the COVID-19 pandemic, access has diminished even more. The CDC reported 34% of US patients not receiving the care they need during the pandemic. These numbers were a result of canceled appointments, cutbacks in transportation options, and fear of going to the emergency room. ^{62, 63}However, the recent proliferation of telemedicine platforms and remote monitoring platforms during the crisis has effectively opened the way for more widespread use of HMDs and SGs as a means to address barriers to healthcare access in the future, especially for specialty consultations.

Patients’ acceptance of HMDs and SGs

This topic is of special interest to clinicians, some of whom have expressed concern about the impact of wearable technologies on the user’s appearance and any adverse effect on the clinician-patient relationship. Conspicuous devices such as the HoloLens or Magic Leap are more likely to be perceived as “inhuman” as patients find it difficult to see the caregiver’s eyes. This may be a reason why they are predominantly seen in surgery and medical education. Less intrusive head-mounted displays such as Google Glass, Vuzix, or Hippo Virtual Care allow for continued eye contact with the patient and impede to a much lesser extent on the naturalness of the care situation. Surveys and studies have shown that, even if they are unfamiliar with these types of devices, ⁶⁴

Patients are becoming passively trusting of this new technology because they have no active control over the device, but are nevertheless dependent on outcomes influenced by the devices. It is, therefore, no surprise that as wearable technologies become more prevalent in care delivery, patients have mixed feelings about privacy and personal data security. ³¹ Clinicians should proactively address these concerns by educating their patients about the regulatory compliance of HMDs and SGs for use in healthcare and the benefits for the patient. ³⁰

Challenges to Overcome

Interoperability with hospital systems and patient devices

One of the most significant potential benefits of HMDs and SGs lies in their ability to receive or enter, assess, and compute information for enhanced health outcomes and clinical decision support in real time. However, this potential can only be fully realized through HMDs and SGs which are linked to health systems, patient devices, care coordination instruments, and AI-enabled applications. Thus, patient-to-machine interactions would allow for real-time diagnoses, coordinated treatment plans, and follow-up care. ^{65, 66}

Considerations for a successful deployment of HMDs and SGs in healthcare

Inclusive IT-design: ensure that platforms offer flexible, open-source, interoperable IT-solutions that can be integrated seamlessly with existing hospital systems
Human Resource Management: ensure that training programs actively educate and engage medical staff to boost adoption rates
KPI Tracking: define success measures up front and track key metrics to calculate impact
Clinical Workflow: ensure that new technologies are designed to complement existing clinical workflows
Vendor Selection: Choose a reliable, customizable solution Continuing With customer support, regularly update options, and verifiable compliance With Appropriate legal, compliance and ethical frameworks

Regulatory and Reimbursement

The sharp rise in demand for virtual and remote care ⁶⁶ around the world as a result of the global pandemic, has led to a relaxation in federal privacy regulations, and current reimbursement models have been stretched on a “temporary and emergency basis” so that telemedicine services for Medicare beneficiaries, for example, are covered if they are delivered from a physician’s practice or an ED. ⁶⁷Robust demand for HMDs and SGs during the pandemic has triggered in increase in deployment in healthcare, and it is becoming increasingly apparent that this new form of wearable technology brings specific and extended benefits that traditional telehealth platforms cannot offer. Remote and virtual care will no doubt continue to catalyze our care model and can help improve the cost-effectiveness of our respective healthcare systems. ⁶⁵However, increased device adoption is as much dependent on more permanent changes to reimbursement models for virtual care, as on continuous advancements in the digital technology itself. All stakeholders – industry, providers, payers, government, advocacy groups, and patients – should be encouraged to push for the digital transformation of healthcare and the increased deployment and utilization of smart and wearable technologies, as well as reimbursement reforms that endure long after the pandemic is over.

Beware algorithm biases

Sophisticated medical devices are entering the market that use machine learning algorithms or AI-powered platforms that do not require clinicians to recognize patterns and make inferences. Their inclusion in HMD and SG platforms is only a matter of time. Like the IDx-DR that detects retinopathy, existing technologies can already predict outcomes by analyzing massive, constantly changing datasets. ^{13, 58} They have the potential to significantly improve health outcomes and reduce costs if clinicians leverage these emerging technologies as a complement to – as opposed to substitute for – their diagnoses and treatments. Clinicians must not become overly dependent on new tools or algorithms and should continue to interpret the results within the appropriate context and evaluate recommendations for each individual.

Deploying HMDs and SGs in healthcare is a proactive and collaborative approach to addressing burgeoning issues like provider shortages, patient access, the rising administrative burden, gaps in medical education, and the need to coordinate care teams more efficiently.

Conclusion

Within the ongoing digital transformation of healthcare, Head-Mounted Displays (HMDs) and Smart Glasses (SGs) represent a promising blend of virtual and physical care. Their relatively recent introduction into healthcare has already proven their effectiveness in surgical settings and for medical education use cases. They can be used to plan and execute procedures and guide even non-experienced physicians through surgery. And medical students and nurses can see all facets of care delivery through the caregiver’s eyes, from diagnosis to rehabilitation. Operating the devices hands-free in sterile environments, clinicians can display data in their peripheral view or overlay medical images without the need to turn away from the patient, or collaborate via real-time video and audio streaming with peers or specialists. And Wi-Fi, Bluetooth, LTE or 5G can seamlessly connect HMDs and SGs with hospital systems and EMR data, and ultimately patient wearables and sensors, enabling faster triage and smarter clinical decision making.

The true benefit of HMDs and SGs has become clearly apparent during the COVID-19 pandemic. Through enhanced HMD platforms such as Hippo Virtual Care, clinicians are now sharing and receiving real-time information, video and audio in a variety of clinical and non-clinical settings. In particular, the hands-free, rugged, medical grade models have graduated from being a novelty used by early adopters to becoming an effective and essential way of providing specialist care to patients and teaching medical staff at scale while significantly reducing exposure to the virus.

We believe that this trend a precursor to how care delivery will look in the future. Deploying HMDs and SGs in healthcare is a proactive and collaborative approach to addressing burgeoning issues like provider shortages, patient access, the rising administrative burden, gaps in medical education, and the need to coordinate care teams more efficiently. Looking ahead, HMDs and SGs will be increasingly empowered by emerging machine-learning algorithms and AI-technologies that can collect and compute information in real time, allowing for faster diagnoses, more personalized treatment and better quality outcomes. This will effectively enhance clinical decision-making, improve our ability to predict adverse events, and maximize treatment success. It’s a brave new world and we are at the tipping point of a paradigm shift in how healthcare is taught, delivered and experienced.