Browsing by Author "Bras, Edward Hendrik"

Now showing 1 - 1 of 1
  • Thumbnail Image
    Item
    A feasibility study of elementary reinforcement learning-based process control
    (Stellenbosch : Stellenbosch University, 2022-04) Bras, Edward Hendrik; Louw, Tobias Muller; Bradshaw, Steven Martin; Stellenbosch University. Faculty of Engineering. Dept. of Process Engineering.
    ENGLISH SUMMARY: The classical control paradigm is widely used in industry, has well-understood theoretical guarantees, and forms part of the foundational knowledge of chemical engineers. Challenging non-linear dynamics prevent its successful application in certain cases, while classical controllers cannot automatically accommodate changing closed-loop dynamics. Advances in computational capabilities have led to a significant research interest in the application of Reinforcement Learning (RL) to control processes. In RL, a computational agent interacts with an environment to maximise the cumulative scalar rewards received. It may be viewed as an alternative paradigm for control, as is done in this thesis, or as an approach to potentially enhancing the performance of classical controllers. This simulation-based study’s purpose is to investigate the feasibility of elementary RL techniques to automatically determine the final element adjustments in a single-loop RL-based control scheme. It places into context what the strengths and limitations are of using elementary RL to control processes and highlights nuances of RL-based control without trying to outperform classical control. The control of a self-regulatory water tank model and the Van de Vusse reaction scheme model (used for benchmarking and requires advanced control solutions) were studied by applying two algorithms – Q-learning and SARSA – in a control scheme synthesized purely for theoretical study. Subsequently, these algorithms and the One-Step Actor-Critic algorithm were applied to the control of particle size in a qualitatively accurate grinding circuit model. All simulations leveraged the simplest possible RL design to allow interpretable and clear accounts of how these systems behave. The results show that the use of elementary RL techniques to obtain interpretable RL-based controllers for simulation-based study worked well for the water tank and Van de Vusse reaction scheme models. This was not the case for the grinding circuit case study. Replacing the classical control paradigm is not likely using elementary RL. Significant safety concerns arise since large amounts of operational data may be required and insufficient training in certain regions of the state-action space leads to unpredictable control behaviour. The strengths and weaknesses of the algorithms studied were investigated. It is unlikely that a reduction of control loop specific tuning parameters in comparison to classical control will be realised in practical control problems by applying RL-based control. Where applicable, classical control outperformed the elementary RL-based controllers which stresses that algorithmic adjustments are required, as is recognised in state-of-the-art RL-based control approaches. To conclude, the most practically feasible RL-based control solutions are likely to lie in the enhancement of existing control solutions by incorporating RL principles. The studied elementary RL-based control methods are not feasible for practical robust control. The control engineer must not be removed completely from the loop, and existing domain knowledge must be reconciled with computational thinking instead.