Project Overview

Cloud-based and API-driven infrastructure for medical imaging AI.

Motivation and Objective

Radiologists are increasingly adopting AI tools and AI driven workflows. Luxsonic is building a complete immersive medical imaging solution that works on any VR headset. However, the ability of these headsets to process data and perform inferences using machine learning is limited and inconsistent. MLHub is a backend platform built to perform these functions. MLHub plays an integral part of Lusxsonic VR medical imaging platform, as well as a major project with Sunnybrook Hospital, the Memorial University of Newfoundland and Voronoi Health Analytics.

Key Contributions

As part of this project, I:

  • selected the entire technical stack for this infrastructure.
  • determined which AI functionality to implement.
  • implemented and evaluated 6 machine learning algorithms for imaging.
  • evaluated 3 speech-to-text models using a benchmark dataset I created.
  • created a data ingestion pipeline.
  • created a refinement algorithm to detect and correct spurious results while adding very little time (less than 0.05s).
  • rendered all functionality accessbible via RESTful API.
  • implemented model staging and other resource management techniques.
  • maintained development and production servers.

Technologies Used

The technologies that are/were used.

  • Backend: AWS EC2, Ubuntu, Python, Flask, NVidia Clara, MONAI framework (PyTorch-based), Docker, REST, Bash.
  • Tools: Git, Asana/Jira
  • Other: 3D Slicer, Jupyter Notebook.

Challenges and Solutions

  • Challenge The resources need to be front-end agnostic, whether in VR or from any other platform, such as Web.

    • Solution Flask was used as a framework to create RESTful API endpoints and serve all functionality as microservices.

  • Challenge Inference models, particulary at the boundary of industry and academic research in medical imaging, come in various types of packages.

    • Solution The choice of Linux OS, Docker and Python allows the implementation of any inference algorithm. We are able to encapsulate them for the purposes of our data workflow, or inherit the encapsulation provided by models built to be used in the MONAI framework.

  • Challenge The upload of medical images can be slow.

    • Solution Preloading of datasets was implemented, as was as static sessions so that the data and results would remain persistent.

  • Challenge Medical imaging inference can create a great deal of performance overhead due to processing times.

    • Solution Various functions are benchmarked in real time with the data logged so that performance can be profiled to identify any bottlenecks.

Impact and Results

  • NVidia’s DeepGrow3D algorithm was implemented in MLHub with the corresponding UI and UX workflow in SieVRt, Luxsonic’s own VR medical image viewer. This was showcased as RSNA 2021 and received very positive feedback.

  • The establishing of the MLHub platofrm has helped Luxsonic secure a large partnership project and grant funding as part of the INOVAIT innovation fund.

  • Prospective partners show continued interest in making their algorithms available on the MLHub platform.

SieVRt medical imaging viewer