A recently published paper in the prestigious journal Applied Intelligence introduces a novel method by which machine learning can extract valuable information from uncertainty or noise in input data. The approach has been proven on materials science and physics test cases and has potential to provide vital insights across chemistry, formulations development, life sciences - and even beyond industrial R&D.
The work is the result of a collaboration between Intellegens CSO Dr Gareth Conduit and his PhD student at the University of Cambridge, Bahdan Zviazhynski. Experimental data in the physical sciences often contain uncertainty due to natural scatter in the experimental results, limitations in the precision of measurements, errors, or a need to extrapolate from incomplete data. R&D teams are increasingly interested in using experimental results to train machine learning models that can predict and design future performance. But uncertainty in the training data often renders machine learning unviable. The latest work not only overcomes this barrier but, perhaps counter-intuitively, can actually use uncertainty to increase the accuracy of predictions, turning the negative impact of noise into an asset.
There are already well-known examples of noise being used in this way. For example, volatility in financial markets is used to predict derivative prices. A key breakthrough in the latest work is to analyse this problem in a generic way, creating an algorithm that can characterise and exploit the uncertainty across many types of problem. In the published paper, the method was applied to study a dielectric crystal, a material type often used in electronics, and was able to accurately predict physical properties that were missed by standard machine learning approaches. It was also successfully applied to another tricky noisy experimental dataset, single-particle diffraction of droplets. In other work, the approach has been used to design two concrete mixes with improved performance.
The generic nature of the approach allows it to be integrated into machine learning software, such as Alchemite™ from Intellegens. “Alchemite™ already has unique capabilities that enable it to accurately estimate uncertainty and succeed with sparse, noisy experimental data, where other machine learning methods fail,” explains Gareth Conduit.
Gareth goes on to say “Incorporating this new method will enable it to go further in extracting hidden value from key datasets. We look forward to working with our industrial R&D customers to provide even greater insights into their development challenges. We’re also excited by potential applications in areas spanning finance to retail.”
