Morris Panner is President of Intelerad, a global leader in medical image management solutions.
Virtual reality (VR) has been talked about for at least a few decades, but you may have also heard about augmented reality (AR) and perhaps even mixed reality (MR) in recent years. Collectively, these terms are commonly referred to as the building blocks of the “metaverse” and define new ways in which we interact with technology.
While virtual reality, augmented reality and mixed reality are most often associated with immersive entertainment and the gaming industry, they are making innovative inroads in other industries, notably healthcare. For a specialty like radiology, in particular, which is all about imaging, the potential for these technologies to make dramatic breakthroughs is significant.
First, let’s clarify what each of these terms means:
Virtual reality. Using a combination of both specially designed software and hardware (typically a headset device over the eyes and joystick), VR immerses you in a simulated world that you can move within. For example, using VR could put you in a virtual surgery room where you experience or conduct a procedure in detail.
Augmented reality. AR, on the other hand, allows you to see your actual environment in real time, with virtual objects or animated characters superimposed. Pokemon Go was the first AR game to go mainstream with millions of users.
Mixed reality. MR is a hybrid of both virtual reality and augmented reality, which has the real and the simulated worlds interacting in real time.
What’s Possible For Radiology
Surgeons do extensive training, including assisting on many operations with a more experienced surgeon before taking the reins themselves. Because no person or situation is identical, what could be better than being able to practice for a procedure in advance, on the actual patient, virtually? AR, VR and MR can help a surgeon do just that by utilizing the patient’s own medical images.
Radiology has long been a guiding light for surgeons who rely on a patient’s medical imaging to make a diagnosis or map out precisely where to operate. VR and AR are now being used in conjunction with medical imaging to assist with the preparation of procedures. Like virtual reality, radiology is all about imaging. Radiology uses sophisticated imaging techniques (CT scans, x-rays, etc.) to take pictures of the interior of our bodies.
The quality of these pictures gets finer with higher resolution as the imaging technology improves, and now VR and AR technologies are making it possible to view them in a more interactive way. Algorithms can interpret CT data into a 3-D map, allowing doctors and other medical professionals to explore and manipulate images for a growing variety of use cases.
A company called Brainlab, for example, has created a Mixed Reality Viewer that brings patient images off the computer screen and into life, allowing doctors to literally walk around realistic 3-D images of a patient’s anatomy, easily zooming in on areas of interest. It can be a helpful tool for colleagues discussing a procedure or for a doctor consulting with a patient.
Fierce Biotech recently reported on a partnership between VR developer Surgical Theater and the medical technology giant Medtronic in which they will meld AR with its surgical navigation platform for tackling aneurysms, tumors and other conditions. The technology shows more promise than only allowing surgeons to test different approaches pre-operation, but should be able to provide them with a live, 3-D rendering above the surgical site during the actual procedure.
A Safe Way To Train New Doctors
Medical students can also use these technologies in their training to practice procedures more frequently. Vanderbilt University has been using a simulator that provides trainees with the physical sensations of using instruments and feeling body tissue while performing a virtual surgery on a virtual patient. The VR tracks and analyzes their progress in real time.
A UCLA study showed that medical students who have used VR tools to perform a particular procedure demonstrated both greater speed and accuracy.
Also, because VR is virtual, it doesn’t necessarily have to be tied to a physical location. This opens opportunities for the long-distance training of students, including those who may be in under-resourced areas and have limited access to qualified teachers. During the pandemic, students have been able to train socially distanced using VR simulations.
VR and AR can also help radiology students in their training. For students in the process of learning what they need to search for when closely examining medical images, VR headsets can enable them to view and interact with these images in immersive 3-D.
AR and mixed reality are very exciting developments for interventional radiology, the process of using medical imaging to guide real-time procedures such as a biopsy, angioplasty and stenting. Interventional radiology is often considered preferable to surgical alternatives since it can be less costly, less risky and can sometimes be done without a hospital stay. But these procedures can be even more effective with these new technologies.
“With augmented or mixed reality, medical imaging can be more readily accessible or displayed in actual three-dimensional space during procedures to enhance guidance, at times when this information is most needed,” noted the authors of this study on AR and MR in interventional radiology.
As you can imagine, having a detailed, multi-dimensional view of the vascular anatomy can be a huge help when trying to carefully guide a needle or a catheter in a person’s body.
A Multidimensional Future
With time, VR and AR may no longer be thought of as radical technologies as they grow increasingly integrated into the world of radiology, from a doctor giving us an interactive consult on our MRI results to everyday surgical procedures performed alongside real-time 3-D renderings for guidance. For the discipline of radiology, this is yet another thrilling step towards providing both patients and doctors with a better view of the human body.