Mixed Reality enables Google Maps for surgery

By Alan Kwok
Posted on August 23, 2021

University of Oklahoma researcher Mohammad Abdul Mukit and surgeon Dr. Christian El Amm at OU Health are making a Google Maps for surgery. The vision is to utilize mixed reality to guide surgeons and use AI to “visualize superimposed and anatomically aligned 3D CT scan data during surgery”.

They noticed surgeons were missing something that a pilot flying an airplane or a google map user has. A user guided application that would tell surgeons where they are in the surgery and other vital data. The vision for this application was to modernize surgeries, reduce prep time, and equip surgeons with every scenario in a matter of seconds. The design has a similar effect to google maps. Prep time is eliminated or reduced, and the technology corrects users when they take a detour or go the wrong direction.

Complex surgeries require a significant amount of prep time and to memorize unique cases. From the moment a procedure starts, surgeons are on their own with “no targeting or homing devices or head-mounted displays to assist them.” This application will “be used to see the reconstruction steps he had planned during surgery while never losing sight of the patient.”

The use of mixed reality and the HoloLens was a no brainer. The HoloLens unlocked new possibilities to improve efficiency and increase safety. Two big doors unlocked are hands free operations and collaboration.

In surgery we want surgeons to remain focused on the task at hand and not if they have sanitized after touching a computer. The HoloLens provides surgeons with a computer in their field of view, where they no longer bounce back and forth between computer screen and operating table. Virtually touching objects, voice, and gaze commands reduce touch point where hands are contaminated.

Collaboration improves patient healthcare and hospital efficiency. The HoloLens allows surgeons to share their field of view with experts from around the world. Subject matter experts can annotate and point out areas that look concerning or where to operate. As experts work from home or from one centralize location, travel is reduced, and they are able participate in more operations virtually.    

Training is another benefit. Anyone can virtually be a fly in the wall and observes at the right angle while out of sight and mind. The HoloLens can record procedures to be use for training on demand when needed.


Get the latest news on Mixed Reality.