AI Matches the Accuracy of Trained Radiologists for Identifying Brain Hemorrhage in a Head-to-head Test

Can an AI become as good as a radiologist and aid care? The answer is Yes. In time critical diagnosis like Brain Bleeds, an early detection can significantly increase the chances of survival. There can be cases where a radiologist is not on duty, or a radiologist might miss sometimes. In such cases, AI can be of help. We have trained a deep neural network to be a second pair of eyes, as good as radiologists themselves, which doesn’t take breaks. The research we did is now available as a preprint and will be submitted to peer-reviewed journals soon. We will talk more about our brand new method and dataset, but a bit about who are we before that.

At ParallelDots Inc., we are serious about building a proprietary AI stack, which can solve real-world problems end to end. This means building proprietary datasets, proprietary algorithms and deployment mechanisms. While our Data Tagging engine churns out annotated data from various domains (social media, healthcare, retail), our Data Science team develops novel algorithms which can use this data and deliver AI agents that can be deployed in real world. In a previous article, we told you about our all-new NLP stack which is felicitated by our Data Tagging engine and our Data Science team’s efforts to be in touch with the most cutting-edge tech. With this new series of articles which try to make a “common” sense of our research, we are trying to get you closer to what happens in AI deployment trenches. Hope you enjoy.

RADnet: Matching the accuracy of trained radiologist for identifying brain hemorrhage

ezgif.com-optimize
Meta

Head Injuries or any other condition that can cause a brain hemorrhage or brain bleed are serious and need to be detected as soon as possible. The preferred method of diagnosis is a computed tomography (CT) Scan. CT Scan is detailed enough (not as detailed as an MRI) and it’s faster to image patients (MRI is slower). NCCT Scan (Non Contrast CT) head which is done for detection of brain hemorrage is a 3D map of the brain where brain can be viewed as a sequence of 2D slices. Doctors generally scroll up/down these sequence of slices to locate anomalies.

Dataset

Our medical Data Annotation team tagged a dataset of 2D CT Slice sequences. Being medical professionals, this was the more obvious modality for them to annotate, as they mostly work with it. For each slice, the area with an anomaly (in this case brain hemorrhage) was marked by loose approximate boundaries if they were present. Over 300,000 CT Slices have been tagged by our medical Data Annotation team in this fashion for the presence of multiple pathologies. The subset which the Data Science team first chose to train to make a Neural Network Recognize was Brain Hemorrhage marked slices.

The Architecture
1
RADnet Architecture

We defined the problem statement for the Data Science team as the following: Given a CT Scan, the AI needs to tell whether a hemorrhage is present in a slice and tell what structures in the slice made it think so. As shown in the aforementioned GIF, you can see the approximate area the AI thinks is important in determining hemorrhage (also called attention of the AI). It was almost a no brainer choosing a Deep Neural Network for the task. But what would the architecture look like was a question our Data Science team had to solve.

We decided to model this problem as a sequence modeling problem, where each element of a sequence was a 2D slice and might/might not have an area of interest. A Convolutional Network models each image, which the tagged Region of Interest serving as an attention for classification, and then the representations from such a DenseNet for an entire sequence are passed through a bidirectional LSTM to model context. This combination of Recurrent (LSTM) model with DenseNets with Attention, is what we call RADNet.

Performance Benchmarking and Result

For any automated system to be deployed for a clinical emergency set-up, reliable estimation along with high sensitivity to the level of human specialists is required. This necessitates the need to benchmark against specialists in the field. We compared the performance of our algorithm to the performance of real-world radiologists. The performance of three senior radiologists and RADnet was measured on a dataset of 77 brain CTs. RADnet demonstrated 81.82% hemorrhage prediction accuracy at CT level that is comparable to radiologists. The results are shown in this table:

3
The accuracy benchmarks for RADnet against the three radiologists

RADnet achieves higher recall than two of the three radiologists, which is remarkable.

The Path Forward

The RADnet algorithm emulates radiologists’ method for diagnosis of the brain hemorrhage from CT scans and is on par with the radiologists in detecting the anomaly. Noticeably, very high sensitivity is required to deploy automated emergency diagnostic tools. Also, there still exist so many other equally severe brain conditions that the given algorithm is unaware of.

We envision a future where similar emergency diagnostic tools can detect different anomalies from brain CT scans. We highly regard the fact that the presented solution should not be misinterpreted as a plausible replacement for actual radiologists in the field. RADnet demonstrates potential to be deployed as an emergency diagnosis tool. However, its real-world performance is still subjected to further experimentation.

At ParallalDots, we are working on challenging problems in Healthcare, NLP and Image Classification. Please watch this space for more updates on the research work at ParallelDots.




ParallelDots AI APIs, is a Deep Learning powered web service by ParallelDots Inc, that can comprehend a huge amount of unstructured text and visual content to empower your products. You can check out some of our text analysis APIs and reach out to us by filling this form here or write to us at apis@paralleldots.com.

Leave a Reply