Improving In Vitro Fertilisation (IVF) with Machine and Deep Learning

Dr Nicholas Knowlton, Senior Research Fellow, Molecular Medicine and Pathology, Faculty of Medical and Health Sciences;
Dr Nidhi Gowdra, eResearch Solutions Specialist, Centre for eResearch

Home

•

Project

•

Improving In Vitro Fertilisation (IVF) with Machine and Deep Learning

By exploring how to improve the success rate of In Vitro Fertilisation (IVF) implantations, we hope the knowledge will be embedded in a model and made it widerly available locally and overseas where the investment will generate export value for New Zealand and benefit the needed parents by reducing IVF waiting time and increasing the rate of live births.

The current process of selecting embryos for implantation in IVF is based on little knowledge of the relationship between the parameters for embryo selection and the actual success rates post-implantation. Embryos are selected based on the features in a single image taken at a single time. Some researchers and clinicians are starting to apply artificial intelligence (AI) to a selection of embryos, thereby considering multiple factors at once that indicate potential successful implantation and live birth more likely. Unfortunately, most of these schemes try to copy embryologists’ current and limited approach. We have evaluated the existing schemes and realised that we can do much better; particularly given that none of them appear to improve success rates.

Our team of an embryo quality specialist, a machine learning/AI expert, and a clinical embryologist, a key opinion leader in embryology, will use exclusive access to billions of embryo images alongside their clinical information to develop an AI-based approach to embryo selection. We will use information regarding a wide range of aspects of the embryo at different stages in its development, together with information regarding the parents. This knowledge will be embedded in a model, which will be made available widely in New Zealand and overseas by a new company developed for the purpose. This enterprise will create new export returns for NZ through selling access to the model while generating significant social benefits in New Zealand by reducing IVF waiting times and increasing the number of live births from IVF.

The Centre for eResearch is a key partner in this project through their Machine and Deep Learning (MaD) Service. They provide support with data storage, retrieval, organization, integration with third-party tools and direct coding support. By leveraging their expertise with my research group, we can work more efficiently by tapping into a much larger experience base. While the project is still ongoing, we will discuss our progress on automated embryo morphokinetic identification in this project update.

Problem

With billions of unlabeled images, we needed a way to bootstrap the image labeling process. Directly labeling even a fraction of these images would be time-consuming and use expensive clinical expertise. After considering our options, we went forward with an active learning approach, where a small set of data is labeled to build a base model, and new data is added where the model lacks confidence.

Lay method description

A random set of ~5,000 images was labeled to start the model building process. After this, we found that only embryos that progress to blastocyst provide the most cost-effective samples to label, and an additional 10,000 images were labeled using this approach. With model confidence improving, this approach was continued until model accuracy was approaching 90% for all classes. Amazingly this approach required only ~60,000 hand labeled images or 0.0024% of the dataset. With an accurate model, the focus shifted to identifying images the model has low confidence or marginal classification.

Technical method description

Images are preprocessed and split into training/validation and test sets by 72%/8%/20%. During training, image augmentations are applied including height and width shifts, horizontal and vertical flips and random image rotations. The augmented images are used to train an EfficientNetv2M from scratch. The model is openly available with the standard Tensorflow and Keras frameworks in Python. Further HyperParameter optimizations were performed to reduce model over-fitting and increase classification performance.

Results

All model training was performed on 2 NVIDIA V100 GPUs, which were made available to us via the Nectar Research Cloud. The classification model of embryo morphokinetic stages was evaluated on unseen randomized images with a top-1 accuracy of 92.79% and a top-2 accuracy of 98.19% for 12 distinct classes (Figure 1. confusion matrix). However, more data is needed to further mature the models for commercial deployment. This project is on track to realize the aim of revolutionizing embryo selection within In Vitro Fertilization (IVF) in NZ and abroad.