From gaming system to medical breakthrough: how Microsoft and Novartis created Assess MS


When Microsoft released the Kinect system for playing Xbox video games about five years ago, it attracted the interest of an unlikely source: the healthcare company Novartis.


For years, Novartis has been trying to find more consistent ways to quantify whether the treatments it is developing for multiple sclerosis are working, but assessing whether a patient’s symptoms are stabilizing or getting worse is complicated.

That’s partly because multiple sclerosis itself is complicated: In some patients, symptoms might progress with heartbreaking speed, while in others they may show up slowly, erratically and over a period of many years.

“One of the most difficult things about MS is the uncertainty of it,” said Cecily Morrison, a researcher in Microsoft’s Cambridge, UK, research lab, who has spent the last couple of years working on the research project that was borne out of Novartis’s idea, dubbed Assess MS.

To try to quantify the progress of multiple sclerosis, doctors have developed a standard set of tests they perform, like asking a patient to touch their nose or sit with their arms outstretched. Doctors watch the patient and then use a rating scale to determine how strong the patient’s symptoms are.

The problem? Doctors are only human, and despite all their best efforts to standardize the MS test, in the end it is subjective. The researchers found that when a group of doctors are shown the same patient doing the same movement, some may interpret it as a “1” on the rating scale, while others will say it’s a “2.” Even when the same doctor is shown the same movement on two different days, that doctor may give that patient a different rating.

“The clinicians that we worked with really care about their patients. They really want what’s best for them, and even the best neurologist will admit that when they use these rating scales, it’s pretty coarse-grained,” said Abigail Sellen, a principal researcher in the Human Experience and Design group at Microsoft’s Cambridge, UK, lab. “They know that there’s a lot of variability, even in their own judgments, over time.”

That’s why the possibility of using computer vision, which is the type of technology found in the Kinect system, was so intriguing. Using a tool like the Kinect, the researchers at Novartis figured they could get a more consistent reading of how a patient performed on each of the tests, bringing a new level of uniformity that would help doctors better assess the progress of the disease. That, in turn, could speed up the process of getting the right treatments to patients.

The goal was not to replace the doctor but rather to augment the doctors’ knowledge of the disease with a more consistent measurement of the symptoms, in the same way that an ophthalmologist can measure a patient’s declining eyesight.

“What you don’t want to do with these systems is replace the expert. You want to bolster the expert,” Sellen said. “What we’re doing is giving them a set of data that they can then weave into their judgment.”

Postdoctoral researcher Peter Kontschieder (Photo by Jonathan Banks)
Postdoctoral researcher Peter Kontschieder (Photo by Jonathan Banks)

For full story click here

To read more information, click here.

At Microsoft Research in Cambridge, we truly aspire to transform the world through deep research. The bold and inquisitive minds of our researchers and engineers have produced, and continue to produce, significant contributions to Microsoft’s most successful products and services, as well as to the broader research community. The interdisciplinary nature of our lab ensures that we push the boundaries of computing in an inclusive way, resulting in robust and trusted technologies that can be deployed at scale. We have only scratched the surface of what technology can do for us, and I am tremendously excited to be working in a team that is so committed to having a significant impact upon our society.

Microsoft Research Ltd