Project Overview
Visual feedback to tell radiologists when and where to look again.
Motivation and Objective
Up to 30% of findings are missed by radiologists when reviewing medical images. A great deal of effort and money has gone towards the development of inference models that could replace radiologists. However, research shows that the majority of these findings are plainly visible in retrospect. The radiologist’s failure to notice the feature is typically not due to an inability to detect it, but rather the result of various human factors associated with the repetitive nature of visual search tasks, such as the satisfaction of search.
SecondRead was a software which used data from screen-mounted eye-tracking hardware to map the visual search patterns of radiologists. The collected data was used to build a biometric profile of their search patterns in the context of the image being searched. The goal was to detect abnormal search patterns and give the radiolgist immediate feedback as to how their visual search differed from their typical one, before they moved on to the next case.
Key Contributions
- Created the concept after extensive research of the literature and conversations with leading researchers in the field of errors in diagnostic radiology.
- Coordinated launch of pilot at breast cancer screening program.
- Designed software architecture and contributed to development and troubleshooting of all its components.
- Obtained funding from various sources.
- Mentored junior developer and ML intern.
- Oversaw the creation of synthetic data.
- Oversaw the creation of inference models.
Technologies Used
- Frontend: C#, WPF (Windows Presentation Framework)
- ML: PyTorch, Jupyter Notebooks, Python.
- Models Dynamic Time Warping, Long Short Term Memory.
- Tools: Git
Challenges and Solutions
Challenge Integration into radiology software is extremely difficult and time consuming with incumbents often needing 6-12 months themselves.
- Solution I worked closely with the imaging center’s IT lead and we created our software as a universal and transparent Windows overlay so that it could work with any existing software. We used information available at the operating system level to map the content of windows to their locations. Notably, we were able to use this information to track patients anonymously and detect the end of the current case. We also provided a minimal UI for radiologists to select their profiles.
Challenge We initially had no data to serve as a baseline for model development.
- Solution I instructed my intern on ways to create synthetic data, initially using geometric shapes before moving on to simulated visual search pattern based on preliminary data.
Challenge There is no way to label the acquired eye-tracking data as being from a good or problematic visual search.
- Solution From the start, we focussed on unsupervised machine learning meethods and due to the time series nature of the data, we focussed on Dynmatic Time Warping and Long Short Term Memory Archtectures.
Challenge The pilot site was 90-minutes away and not always accessible for the retrieval of data.
- Solution We created an easy to use upload portal on our website so that the IT manager could upload the data. We also kept the data in a human-readable and easy to explore format so that it could be inspected by anyone onsite if desired.
Impact and Results
Our novel approach to helping radiologists with AI instead of trying to replace them was very well received. Most of the radiologists on site agreed to participate in the pilot.
The startup I founded driving this project, ClearVoxel, received several accolades, including 2019 Velocity Fund Finals winner, SIIM 2019 Innovation Award semi-finalist and 2019 Waterloo MedTech Top Startup Award.
International Patents were filed to protect aspects of this workflow not yet in the public domain as well as the forward-looking functionality it would enable.
Unfortunately, this project was shut down due to the Covid-19 pandemic and we were unable to relaunch it as uptake for mammography in the following months was very low due to it being an elective procedure.