Deep learning is emerging in many industrial sectors in hand with big data analytics to streamline production. In the biomanufacturing sector, big data infrastructure is lagging compared to other industries. A promising approach is to combine deep neural networks (DNN) with prior knowledge in hybrid neural network (HNN) workflows that are less dependent on the quality and quantity of data. This paper reviews published articles over the past 30 years on the topic of HNN applications to bioprocesses. It reveals that HNNs have been applied to various bioprocesses, including microbial cultures, animal cells cultures, mixed microbial cultures, and enzyme biocatalysis. HNNs have been applied for process analysis, process monitoring, development of software sensors, open- and closed-loop control, batch-to-batch control, model predictive control, intensified design of experiments, quality-by-design, and recently for the development of digital twins. Most previous HNN studies have combined shallow feedforward neural networks (FFNNs) with physical laws, such as macroscopic material balance equations, following the semiparametric design principle. Only recently, deep HNNs based on deep FFNNs, convolution neural networks (CNN), long short-term memory (LSTM) networks and physics-informed neural networks (PINNs) have been reported. The biopharma sector is currently a major driver but applications to biologics quality attributes, new modalities, and downstream processing are significant research gaps.

Deep learning is emerging in many industrial sectors in hand with big data analytics to streamline production. In the biomanufacturing sector, big data infrastructure is lagging comparatively to other industries. A promising approach is to combine Deep Neural Networks (DNN) with prior knowledge in Hybrid Neural Network (HNN) workflows that are less dependent on the quality and quantity of data. This paper reviews published articles over the past 30 years on the topic of HNN applications to bioprocesses. It revealed that HNNs were applied to various bioprocesses, including microbial cultures, animal cells cultures, mixed microbial cultures, and enzyme biocatalysis. HNNs were mainly applied for process analysis, process monitoring, development of software sensors, open- and closed-loop control, batch-to-batch control, model predictive control, intensified design of experiments, quality-by-design, and recently for the development of digital twins. Most previous HNN studies combined shallow Feedforward Neural Networks (FFNNs) with physical laws, such as macroscopic material balance equations, following the semiparametric design principle. Only recently, deep HNNs based on deep FFNNs, Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM) networks and Physics Informed Neural Networks (PINNs) have been reported. The biopharma sector is currently a major driver but applications to biologics quality attributes, new modalities, and downstream processing are significant research gaps.

Due to the complexity of biological processes, developing model-based strategies for monitoring, optimization and control is nontrivial. Hybrid neural models, combining mechanistic modeling with artificial neural networks, have been reported as powerful tools for bioprocess applications. In this paper, a systematic literature review is presented focused on the application of hybrid neural models to bioprocesses by Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) over the last 30 years. This analysis showed that hybrid neural modeling has covered a wide range of microbial processes, animal cells, mixed microbial cultures, and enzyme biocatalysis. Hybrid neural models have been mainly applied for predictive modeling/process analysis, process monitoring/software sensors, open- and closed-loop control, batch-to-batch control, model predictive control, intensified design of experiments, process analytical technology, quality-by-design, and more recently, digital twins. Hybrid modeling experienced a decline in the number of publications after a peak in 2004 and is now surging again. A “model scale” research gap was identified, which will likely narrow by a better integration with deep learning and systems biology in the near future. The biopharma sector is currently a major driver but applications to biologics quality attributes (e.g. glycosylation), new modalities and downstream unit operations are significant research gaps.

Due to the complexity of biological transformations, developing model-based strategies to optimize and control bioprocesses is nontrivial. Hybrid models combining a mechanistic description of known influential factors with machine learning to infer the missing influential factors from data have been reported as powerful tools for bioprocesses applications. The artificial neural network is one of the most popular machine learning methods in this case. This paper presents a systematic literature review by computerized search across two databases: Scopus and Web of Science, and backward citation. The PRISMA method was applied to selecting the publications and 159 research articles were categorized as hybrid model applications to bioprocesses problems. It was found that hybrid models were mainly applied in upstream operation steps with a predominance of bioreaction steps. In downstream processing, chromatography appeared as a more recent research topic, with a relatively small number of publications. Furthermore, holistic hybrid modeling applications that integrate data and knowledge from several bioprocess steps will likely emerge in the future, enabling better optimization and control of the bioprocess’s platform. The combination of other machine learning methods with the hybrid neural network model is another opportunity that could improve the output of the model.