NASA utilizes Mixed Reality Applications for Space Station repairs

By Alan Kwok
Posted on September 27, 2021

Since 2016 and the release of Microsoft HoloLens, Mixed Reality has captured a keen interest from NASA. The HoloLens and Mixed reality might be the perfect solutions to solve many needs. Whether for remote operations, self-inspections, maintenance, and repair. The HoloLens and Mixed Reality is quickly becoming the right tool for the job.

As you can imagine communications is vital for NASA. Out in space astronauts cannot depend on the support from subject matter experts at a moment’s notice. Communication delays can mean the difference between correct repairs or breaking an expensive piece of machinery costing millions of dollars.  Due to uncontrollable communication delays, NASA is developing “tools to increase astronaut autonomy to operate spacecraft or systems without assistance.”

The T2 Augmented Reality (T2AR) project is the “first in-space operational use of the HoloLens in combination with custom-built AR software”. The software is the first of its kind. Astronauts are now able to perform self-inspections, maintenance, and repair equipment that would traditionally require support from subject matter experts. This is a shift in operations where a user (astronauts) can diagnose and repair equipment on their own without previous knowledge needed.

T2AR is like a combination of the HoloLens applications Dynamics 365 Guides and Remote Assist. Astronauts are head up hands free to perform tasks on their own. Simple instruction step card guide users through how to conduct inspections, identify and diagnose issues, and how to repair issues as they arise. In the event of no communication, astronauts are armed with the tools to solve problems in real time on their own. When communication permits astronauts can always call a subject matter expert back on earth for support. Experts can point specific parts out and explain how to fix issues like they are there in person.

Early results of the project are promising. NASA and astronauts on the International Space Station have used the HoloLens and T2AR for maintenance on an exercise equipment called T2 Treadmill. The HoloLens provides holographic pdf documents within the user’s field of view and promotes hands free operations. Along with the T2AR software astronauts were armed with “step-by-step guidance and cues to assist in the work.”

The success has NASA exploring other ways to implement this solution in other areas of operations. Other potential scenarios include use when astronauts are on the moon or Mars and communications delays/issues will reside. This will allow astronauts to be independent and fully operational in these scenarios.

SUBSCRIBE

Get the latest news on Mixed Reality.