Outsourcing AI For Drug Discovery: Independent Expertise Is Key To Avoid Overhyped Claims

Investments in artificial intelligence (AI) for drug discovery are surging. Big Pharmas are throwing big bucks at AI. Sanofi signed a 300 Million dollars deal with the Scottish AI startup Exscentia, and GSK did the same for 42 Million dollars. Also, the Silicon Valley VC firm Andreessen Horowitz launched a new 450 Million dollars bio investment fund, with one focus area in applications of AI to drug discovery.

In this craze, lots of pharma and biotech decision-makers wonder whether they should jump on the bandwagon, or wait and see.

In this post, I am presenting results of an independant critical analysis of several research publications in the field of AI, which suggests that AI researchers tend to overhype their achievements.. This practice seems to be widespread, in my opinion, and for illustration purposes, I picked research from one "big pharma" company, AstraZeneca, two academic labs -- at Harvard and Stanford, and one privately held company, Insilico Medicine -- as some of their research results are publicly available and suitable for evaluation.

In an innovative environment like drug discovery, accessing quality innovations is key for success, since first-movers enjoy a huge competitive advantage. However, it is hard to assess the quality in a field as complex as data science. A good compromise is then to employ third-party independent counter-expertise along the way -- to perform technical due-diligence.

One such resource is Startcrowd -- an online network of AI experts and enthusiasts, well-positioned to deliver independent expertise. We tap into an emerging talent pool, educated on online courses. It keeps Startcrowd far from the conflicts of interests peculiar for industrial environment.

This counter-expertise is a way to prevent new disappointments with computer-aided methods. Senior executives probably remember the epic failure of rational drug design in the 1980’s. At that time, big Pharmas like Merck were promising the next industrial revolution. It didn’t happen the way they anticipated.

I am optimistic that things can be different today. Not even because of the novel AI technology innovations, but because outsourcing business models can now be improved: finding and involving stronger subject-matter expert to evaluate research is now easier, in the era of online education and social networks.

Now let’s get into the technical part, with some examples where I provide subject matter critics of AI research, suggesting that it might be overhyped.

AstraZeneca

In this paper, AstraZeneca researchers (joint with others) want to generate novel molecules using recurrent neural networks. This question is important because a creative AI should bring more diversity to the lead generation process, and ultimately, AI could substantially improve the work of human medicinal chemists.

This paper caught my attention because of the large part devoted to the evaluation of the model. It gives the appearance of depth. Various metrics are introduced, based on Tanimoto-similarity and on Levenshtein distance. They provide an impressive number of visualizations, using histograms, Violin plots and t-SNE.

However, all their measures are made between AI-generated molecules on the one hand, and natural molecules on the other hand. They always ‘omitted’ to measure the distance of AI-generated molecules with each other. This omission allows to build an illusion of diversity: a large distance between AI-generated and natural molecules allows to think that the AI got creative, and that it explored new directions in the chemical space. Graphically, it lets think that we got something like this: