Stochastic optimal control problems are arisen in various areas, including biology, chemistry, medicine, engineering, economics, business and finance. Many computational approaches have been developed to solve these problems, and theoretical works on this topic are also well-established in the literature. In this talk, we would like to discuss the use of Kalman filtering techniques for solving the discrete-time nonlinear stochastic optimal problem. First, the formulation of the problem is presented. In the presence of random disturbance, the dynamic system becomes random, uncertain and fluctuated. In this situation, optimizing and controlling the dynamic system become more challenging. Thus, Kalman filtering is applied to estimate the state dynamics. For this aim, a loss function, which minimizes differences between actual output and estimated output, is introduced, and the Kalman filter equations are derived after satisfying the necessary conditions. Here, the extended Kalman filter and the unscented Kalman filter are highlighted. Then, by considering the state estimate and the Hamiltonian function, the feedback control law, which satisfies the stationary condition and minimizes the cost function, is designed. The performance of state estimation is measured by mean squared errors and the performance of control law is given by cost function. For illustration, some examples from engineering and finance are studied, and their simulation results are demonstrated. In conclusion, the Kalman filtering techniques are an efficient computational approach for solving stochastic optimal control problems.   

Keywords: stochastic optimal control, Kalman filtering, feedback control law, mean squared errors, cost function

Sie Long Kek, PhD, CQRM, is currently working as a Senior Lecturer in the Department of Mathematics and Statistics, Faculty of Applied Sciences and Technology, Universiti Tun Hussein Onn Malaysia. He received his M.Sc. and Ph.D. in mathematics from Universiti Teknologi Malaysia, Johor, Malaysia in 2002 and 2011, respectively. He was a research associate at the Curtin University of Technology in 2009 during his Ph.D. study. His research interests include optimization and control, operational research and management science, modelling and simulation, parameter estimation, Kalman filtering, and computational mathematics. He has published more than 40 papers in refereed journals and six (6) book chapters. He is a reviewer for the peer reviewed research journals, including Automatica, Optimal Control, Applications and Methods, International Journal of Control, Heliyon, Journal of Industrial and Management Optimization, Measurement and Control, Hindawi Journal of Mathematics and MDPI Journal of Risk and Financial Management. He has hosted two (2) research projects supported by the Ministry of Education Malaysia. He has supervised three (3) master and two (2) Ph.D. students. From 2021-2023, he has been appointed as the head of the research focus group, which is known as the Numerical Simulation and Applications (NSA).