Identifying abnormal cells in blood samples or biopsies can be like finding a needle in a haystack — and most tools take a one-size-fits-all approach. Nuclei.io, an AI-based digital pathology framework developed by James Zou, PhD, and Thomas Montine, MD, PhD, is designed to improve workflow and diagnosis in cancer and other diseases. Instead of trying to replace pathologists, it learns from them, adapting to individual workflows and offering personalized assistance in spotting cells linked to disease. The tool also allows pathologists to share their models with colleagues—almost like a social network—making it easier to compare results, collaborate, and build on one another’s expertise. And at its core, it uses a human-in-the-loop process: pathologists remain the decision-makers, but AI guides them to make diagnoses more efficiently and with greater accuracy.
Nuclei.io was selected as a Stanford Medicine Catalyst-awarded project, underscoring its potential to reshape digital pathology and transform how pathologists work. As a Catalyst-awarded project, it is part of a program designed to help promising ideas move beyond the lab. Catalyst provides guidance, resources, and access to Stanford’s clinical ecosystem so innovations like Nuclei.io can be refined, tested, and brought into practice—making an impact for both clinicians and patients.
Transcript
Thomas Montine:
We are about to enter a period of medicine that will have the greatest demand for care in the history of humankind. The number of patient biopsies that will be coming to pathologists is going to increase exponentially. The number of pathologists is flat, so how are we going to handle this over the next 30 years? The only solution is to innovate. A gentleman named Virchow — he brought together an emerging technology of the day, which was the chemistry to dye clothing with very thinly cut tissue sections. And those two things together allowed him and others to visualize the components of cells that had never been possible before. And with that started what we now call surgical pathology. A 140-year-old technology is basically unchanged. That is now being transformed by high compute and machine learning.
James Zou:
Because these pathology images are so large, they’re gigabytes images, and they’re so information rich. It does take the human pathologist quite a lot of time to interpret what’s going on in each image, and they have to often figure out from this big image, which has tens of thousands of cells, what are some small minutia of cells that look different that could potentially be indicators of cancer, tumor, different other diseases. And this is where we think that AI can be very helpful in assistance.
Montine:
We should create a human in the loop AI that would help our pathologists be better at reaching the diagnosis, not tell them what the diagnosis is, and James took that idea and ran with it.
Zou:
Nuclei.io is an AI platform that we developed that allows pathologists to very easily adapt AI models to their personal needs. Maybe the algorithms say, OK, maybe you might have missed this part of the image. Can you take a second look? Or maybe algorithms say, OK, this part of the image looks like the most malignant cells. They also allow pathologists to share their models with their colleagues, almost like a social network. They can say, OK, that actually gets the five best pathologists in the world and their models. And have each of those five models actually also make a prediction on the same image and then use each other’s models to help them to improve their diagnosis or decisions. What’s quite really exciting to me about Nuclei.io is that we actually have done a lot of user studies with our pathology colleagues here at Stanford. So they’re actually using these models and then seeing how much can the algorithm, the AI improve their performance.
Brandon Say:
They’re usually a little bit more uniform. I can’t tell if this is some air dry artifact.
Christina Su:
Yeah, because they look so small on the cell block.
Brandon Say:
Exactly. And a lot of these are small. Then there’s some larger ones kind of scattered throughout, which of course you can see in higher grade tumors, but it could also be artifact.
Yang:
When we’re doing the reader study and we’re using the AI, it’s fantastic when we can’t use the AI, it’s like we just want to leave the room essentially because it’s so tedious. Within this stromal portion of the biopsy, I’m looking for plasma cells. In order for us to not miss the plasma cell, what we do frequently would be to order a CD138 immunohistochemical stain, and that requires additional effort, additional costs and additional day. Now with Nuclei.io, just from the H and E, so no extra stain is needed. The AI, which has been trained by the pathologist, will help us find the plasma cells, which would just take a few seconds rather than several minutes — five minutes to 10 minutes that it might take me to scan through the entire biopsy specimen.
Brooke Howitt:
I was ready to just get the list of possible plasma cells to speed it up. It’s hard to go back to just searching through the slides unassisted.
Montine:
There are hundreds of thousands of these biopsy samples that come through just our medical center each year. And a key component of this work is what’s called turnaround time. Patients are waiting for what medicines, what treatment protocols they’re going to be put on. Many times, enrollment in clinical trials, especially for cancer, are dependent on the outcome of the surgical pathology results.
Zou:
AI has tremendous potential to change medicine, to change health care. In order for AI to really achieve its full potential, we also need to engender trust. This is where this human in the loop process come in. The human pathologist will still make the final decisions to make the final diagnosis, but they’ll be guided in that diagnosis and decision.
Montine:
This was the future of pathology and that we could lead or follow. And Stanford is committed to leading the solution that Nuclei.io presents — to make that limited number of pathologists faster. What we also discovered is that it made them safer, they had greater confidence, and that’s everybody’s goal from the very beginning. I’m excited to get Nuclei.io deployed in Stanford Health Care, and we have a path to doing that and I’m excited about what’s next.
