A clearer image

Curious about medical technology, friends and classmates Jose Amich and Raahil Sha jumped at the opportunity to sit in on a surgery at Massachusetts General Hospital during an undergraduate course.

Watching the surgeon perform a minimally invasive procedure, they noticed how heavily he relied on a CT scan and MRI, which affected his speed and efficiency.

“Since these medical images are usually displayed on 2D screens, surgeons have to perform a complicated set of mental gymnastics to match what they see on the screen to the patient,” said Sha, A.B. ’18, a computer science concentrator at the Harvard John A. Paulson School of Engineering and Applied Sciences. “We thought that we could build some augmented reality tools that allow them to bring intrinsically three-dimensional data into the 3D world and overlay it on top of the patient.”

They began building the back-end algorithms to power such a tool and soon, the startup Z Imaging was born. The company develops augmented reality (AR) toolkits that can be used to power applications in medicine and industry.

Sha, who was then completing a senior thesis on alignment between different sets of 3D data, drew on optimization and linear algebra lessons from SEAS courses to build a software development kit from scratch. He and Amich had to balance tradeoffs between speed and accuracy while designing tools for high-stakes situations.

“A lot of AR software development kits are designed for gaming, where precision isn’t too important,” said Sha. “If you are playing a game, and an enemy moves 3 centimeters to the left, you are fine. But when trying to perform an operation, that 3 centimeter difference could mean the death of the patient.”

Using computer vision and other machine learning techniques, they created software that can perform highly accurate alignment between the virtual and real worlds with only sub-millimeter errors.

As they refined the software, Sha and Amich also relied on the SEAS course Startup R&D (ES 95r), taught by Paul Bottino, Executive Director of Innovation Education, for help launching a startup. They took to heart Bottino’s advice on the importance of talking to experts and potential customers.

“We both come from a technical background, so before ES 95r, we weren’t too sure how to navigate the startup world or pitch to people,” said Amich, a molecular and cellular biology concentrator. “There are many applications of AR that don’t really improve performance or quality of life in a significant way. What we wanted was to find cases where AR/VR could vastly improve workflows.”

Through those business conversations, they decided to focus on health care and industry. Amich and Sha also relied on the invaluable support, mentorship, and resources they have received through MassChallenge and the Harvard Innovation Labs Venture Incubation Program.

Z Imaging’s software development kit enables developers to build custom AR tools that seamlessly fuse 3D images with objects in the real world. The co-founders are also exploring the development of packaged solutions for specific industries, such as the aerospace sector.

In aircraft quality control, for instance, an AR-based interface could provide a technician with process instructions and important information about different aircraft components. It could also enable a worker to overlay virtual images of undamaged parts to identify defects.

“We can do some very precise localization of the different pieces with sub-millimeter precision, so we can also navigate technicians through a protocol involving assembly of very small components,” said Amich.

As technology continues to evolve, Z Imaging will be able to provide even more precise solutions, Sha said. For instance, advancements in 3D camera and AR headset technology will enable even more detailed overlays.

Sha and Amich are excited to propel Z Imaging into the future. The firm recently secured its first client—a company that is developing an AR-guided surgical system—and they are working with other potential customers, including a major Chinese hospital and an aircraft manufacturer.

While the risks of making their college startup into a full-time job loom large for Sha and Amich, they are inspired by the opportunity to use their skills and passions to make an impact.

“These time savings make a real difference. If a patient is in the operating room for less time, it means they won’t be under anesthesia as long, they lose less blood, and they can recover quicker after surgery. In the technical space, companies do a lot to save even 5 percent of their time, and if they are able to use AR to streamline complicated procedures, they’d be able to save even more,” Sha said. “Our system will be able to power a new set of tools that will apply AR in spaces that really matter and have massive potential for improving quality of life, and we are excited about that.”