Automated tools for analyzing uncertainty in nonlinear dynamic chemical process models

Abstract

Nonlinear dynamic chemical processes are inherently subject to uncertainty, such as fluctuations in market conditions or changes in operating conditions, creating significant challenges for modeling, optimization, and control in applications such as petroleum recovery, chemical supply chains, and gas separation. Such uncertainty can lower production efficiency and therefore degrade economic performance. This thesis presents automated computational tools that were developed to address these challenges in two ways: through an adjoint subgradient evaluation method for global dynamic optimization problems, which quantifies how uncertainty in model parameters propagates to objective values, and through the optimization of operating schedules for a novel heavy oil recovery process under uncertain future conditions.

Nonconvex dynamic optimization problems emerge in several engineering applications, such as global optimal control, parameter estimation, and safety verification. Such problems are often represented as nonlinear optimization problems with embedded parametric ordinary differential equations (ODEs). Typically, deterministic methods for global optimization employ subgradients of convex relaxations, to construct lower bounds that provide crucial global intuition. Recent results show that subgradients in dynamic optimization problems may be obtained by adapting standard forward or adjoint sensitivity approaches for smooth problems; the adjoint approach ought to be computationally favorable except for small problems. However, established adjoint implementations are incompatible with established software libraries for subgradient evaluation. To address this gap, the first-ever automated proof-of-concept implementation of adjoint subgradient evaluation is developed in C++, nontrivially adapting the convexification package MC++, the ODE solver CVODES, and new differentiation and code generation tools. Within this framework, the adjoint sensitivity system can be constructed with either the forward or reverse modes of subgradient automatic differentiation (AD), adapting recent subgradient propagation approaches. The implementation produces numerical subgradients that agree with finite difference estimates in several case studies, and remains reliable in cases where finite difference estimates fail.

Next, Imperial Oil's Cyclic Solvent Process (CSP) is a solvent-based technology developed as a lower-emission alternative to steam-based approaches. It alternates between injecting a liquid solvent into an underground heavy oil reservoir through a horizontal well to mobilize the oil and producing a mixture of heavy oil and solvent. To meet economic expectations, CSP optimization requires a comprehensive representation of operational characteristics and a well-defined economic model. This thesis develops advanced optimization frameworks for commercial-scale CSP deployment, ultimately enabling robust economic decision-making under uncertainty.

From Imperial Oil, we are provided with simulation-based injection and production data for one nominal well, and the problem considered in this thesis is to schedule the centralized operation of many such wells under various constraints on flow and capacity. Because the centralized resources are limited, wells cannot follow their nominal schedules, and cycle start times may be delayed while production stages may be shortened (“short-cycling”) or lengthened (“long-cycling”). Short-cycling is modeled by truncating the nominal simulation data and optionally boosting the productivity of the next cycle, while long-cycling is modeled by extending production beyond the nominal duration using an exponential decline model fitted to the available data.

Firstly, a new comprehensive mixed-integer linear programming (MILP) process model is developed, capturing all key operational features and economic objectives of commercial CSP operation. This optimization framework addresses the modeling complexities associated with nonlinear CSP operating behavior while retaining the MILP structure for scalability. A novel decomposition approximation method and several implementation considerations are employed to further improve computational performance. To enable more flexible decision-making, a two-stage optimization framework is proposed, where a derivative‐free optimization (DFO) algorithm serves as the primary optimizer while the MILP model functions as a subproblem. An end-to-end Python implementation of this framework was developed, and several recent DFO methods were successfully integrated into this implementation. In addition to conventional DFO methods, a reinforcement learning (RL) algorithm is tailored as an unconventional DFO optimizer. Several case studies are presented to demonstrate the effectiveness of the proposed approaches in maximizing economic performance with reasonable computational effort.

Secondly, a two-stage stochastic optimization framework is proposed to address different sources of uncertainty involved in the CSP operation. Given the computational intractability of solving a large-scale stochastic model directly, and the fact that existing decomposition methods are incompatible with the structure of the proposed formulation, a new tailored decomposition strategy is developed that integrates DFO with MILP solvers, where this integration is specifically designed for two-stage stochastic structure to efficiently coordinate first-stage scheduling decisions with second-stage scenario evaluations. Furthermore, an end-to-end Python implementation is constructed, including data preprocessing, modeling, optimization, and numerical and visual result generation. A case study is presented using this automated implementation.

Lastly, advanced machine learning techniques are explored to extend CSP optimization to settings where uncertainty evolves and decisions must adapt as new information arrives. With limited results obtained from a neural network surrogate that learns the input-output mapping of the MILP subproblems in the two-stage stochastic formulation, the focus shifts to reinforcement learning, given its capability to handle sequential decision-making in complex environments. The CSP optimization problem is translated into a semi-Markov Decision Process and multi-agent formulation, which is solved using Multi-Agent Proximal Policy Optimization (MAPPO). To enforce technical requirements during operation, a linear program (LP)-amended reward function is developed. An end-to-end PyTorch implementation is evaluated on two CSP case studies. The results confirm the effectiveness and stable performance of the proposed RL-based CSP optimization framework. Although future work may help to further improve training efficiency, this framework provides a promising and flexible foundation for exploring alternative RL algorithms and advancing data-driven CSP optimization.