具有外部擾動和測量噪聲的非線性隨機系統強健基於 H∞ 觀測器的輸出回授參考軌跡追蹤控制設計一直是領域中個非常複雜和困難的問題。它需要在設計過程中為非線性觀測器和控制器求解一個非常困難的控制觀測器耦合的 Hamilton Jacobi Isaacs 方程 ( 。目前，還沒有解決這種控制觀察器耦合的HJIE的解析和數值方法。在本文中，提出了一種新的HJIE嵌入深度學習方法作為深度學習演算法和基於 H∞ 觀測器的 軌跡追蹤 控制方案的複合式設計，以直接解決基於 H∞ 觀測器的非線性偏微分控制觀測器耦合HJIE非線性隨機系統的追踪控制設計問題。在離線訓練階段，狀態估測誤差和追蹤誤差輸入到HJIE嵌入深度神經網絡（DNN），輸出HJIE的解。如果不是，則將 HJIE的學習誤差 回授給Adam學習演算法訓練DNN，直到求解HJIE並生成 H∞ 追蹤控制定律、觀測器增益以及最壞情況的外部干擾和測量噪聲，將其發送到非線性隨機系統模型取代外部擾動和測量噪聲，生成輸出測量信號和估測誤差信號，用於下一步訓練。我們可以證明，當 Adam 學習演算法收斂時，所提出的嵌入DNN的 H∞ 基於觀測器的參考軌跡追蹤方案可以實現非線性隨機系統的理論基於 H∞ 觀測器的參考軌跡追蹤控制。如果非線性隨機系統沒有外部干擾和測量噪聲，所提出的嵌入DNN的 H∞ 基於觀測器的參考軌跡追蹤控制方案可以同時逼近隨機漸近狀態估測和參考軌跡追蹤。此外，還可以通過對所提出的方案進行一些修改來解決非線性隨機系統的基於DNN的最優H2觀測器參考軌跡追蹤控制設計問題。最後，提供了具有外部干擾和輸出測量噪聲的四旋翼無人機系統基於 H∞ 觀測器的參考軌跡追蹤控制的設計實例，以說明設計過程並同時驗證所提出的HJIE嵌入 H∞ 基於DNN的狀態估測和參考軌跡追蹤性能在具有外部干擾和測量噪聲的非線性隨機系統的基於觀測器的參考軌跡追蹤控制方案。

The robust H∞ observer-based output feedback reference tracking control design of nonlinear stochastic systems with external disturbance and measurement noise is always a very complicated and difficult problem in the control field. It needs to solve a very difficult control-observer-coupled Hamilton Jacobi Isaacs equation (HJIE) for nonlinear observer and controller in the design procedure. At present, there exists no analytic and numerical way for solving this control-observer-coupled HJIE. In this paper, a novel HJIE-embedded deep learning approach is proposed as a co-design of deep learning algorithm and H∞ observer-based tracking control scheme to directly solve the nonlinear partial differential control-observer-coupled HJIE of H∞ observer-based reference tracking control design problem of nonlinear stochastic systems. In the off-line training phase, state estimation error and tracking error are inputed to HJIE-embedded deep neural network (DNN) to output the solution of HJIE. If not, the learning error of HJIE is fedback to train DNN by Adam learning algorithm until to solve HJIE and to generate the H∞ tracking control law, observer gain as well as the worst-case external disturbance and measurement noise, which are sent to nonlinear stochastic system model to replace the external disturbance and measurement noise to generate output measurement signal and estimation error signal for next step training. We could show that the proposed DNN-embedded H∞ observer-based reference tracking scheme can achieve the theoretical H∞ observer-based reference tracking control of nonlinear stochastic system as the Adam learning algorithm converges. If the nonlinear stochastic system is free of external disturbance and measurement noise, the proposed DNN-based H∞ observer-based reference tracking control scheme can approach to the stochastically asymptotical state estimation and reference tracking simultaneously. Further, the DNN-based optimal H2 observer-based reference tracking control design problem of nonlinear stochastic systems can be also solved with some modifications of the proposed scheme. Finally, a design example of H∞ observer-based reference tracking control for quadrotor UAV system with external disturbance and output measurement noise is provided to illustrate the design procedure and to validate the state estimation and reference tracking performance simultaneously of the proposed HJIE-embedded H∞ DNN-based observer-based reference tracking control scheme of nonlinear stochastic systems with external disturbance and measurement noise.
Index terms– Nonlinear stochastic system, nonlinear H∞ observer-based output feedback reference tracking control, Hamilton-Jacobi Isaacs equation (HJIE), HJIE-embedded DNN-based observer-based control scheme, Adam learning algorithm, quadrotor UAV.

[1] I. Goodfellow, Y. Bengio and A. Courville, Deep learning, Cambridge, U.K. MIT Press, 2016.
[2] G. E. Dahl, D. Yu, L. Deng and A. Acero, ”Context-dependent pretained deep neural networks for large-vocabulary speech recognition,”
IEEE Trans. Audio Speech Lang. process, vol. 20, no. 1, pp. 30-42, Jan. 2012.
[3] A. L. Maas, P. Qi, Z. Xie and A. Y. Hannun, ”Building DNN acoustic models for large vocabulary speech recognition,” Computer
Speech and Language, vol. 41, pp. 195-213, Jan. 2017.
[4] D. Ciregan, U. Meier and J. Schmidhuber, ”Multi-column deep neural networks for image classification,” IEEE Conf. Comput. Vis. and
Pattern Recognit. Providence, pp. 3642-3649, 2012.
[5] H. Shin etal, ”Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer
learning,” IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1285-1298, May 2016.
[6] S. P. Singh, A. Kumar, H. Darbari, L. Singh, A. Rastogi and S. Jain, ”Machine translation using deep learning : An overview,” 2017,
Int. Conf. Comput. Commun. Electron, Jaipur, pp. 162-167, 2017.
[7] C. Deng, S. Liao, Y. Xie, K. K. Parhi, X. Qian and B. Yuan, ”PermDNN: Efficient compressed DNN architecture with permuted
diagonal matrices,” MICRO 2018, Fukuok, pp. 189-202, 2018.
[8] A. E. Eshratifar, A. Esmaili and M. Pedram, ”BottleNet: A deep learning architecture for intelligent mobile cloud computing services,”
ISLPED 2019, Lausanne, Switzerlan, pp. 1-6, 2019.
[9] J. Yin, S. Su, J. Xun, T. Tang and R. Liu, ”Data-driven approaches for modeling train control model: Comparison and case studies,”
ISA Trans, vol. 98, pp. 349-363, Mar. 2020.
[10] N. Berman and U. Shaked, ”H∞-like control for nonlinear stochastic control,” Systems and Control Letters, vol. 55, pp. 247-257, 2006.
[11] T. Basar and P. Bernhard, ”H∞ optimal control and related minmax design problems: A dynamic game approach,” Birkh¨auser Boston,
Boston, MA, 1995.
[12] J. W. Helton and M. R. James, ”Extending H∞ control to nonlinear systems,” SIAM: Philadelphia, 1999.
[13] A. Isidori and A. Astolfi, ”Disturbance attenuation and H∞ control via measurement feedback in nonlinear systems,” IEEE Trans.
Automat. Contr., vol. 37, pp. 1283-1293, 1992.
[14] W. Zhang and B. S. Chen, ”State feedback H∞ control a class of nonlinear stochastic systems,” SIAM J. Contr. Optim., vol. 44, pp.
1973-1991, 2006.
[15] B. S. Chen and W. Zhang, ”Stochastic H2/H∞ control with state-dependent noise,” IEEE Trans. Automat. Contr., vol.49, no. 1, pp.
45-57, 2004.
[16] J. Yong and X. Y. Zhou, ”Stochastic control: Hamiltonian systems and HJB equation,” New York: Springer, 1999.
[17] W. Zhang, L. Xie and B. S. Chen, ”Stochasitc H2/H∞ control: A nash game approach,” CRC Press, 2017.
[18] B. S. Chen and C. H. Wu, ”Robust optimal reference-tracking design method for stochastic synthetic biology systems: T-S fuzzy
approach,” IEEE Trans. Fuzzy Systems, vol. 18, pp. 1144-1158, 2010.
[19] B. S. Chen, C. S. Wu and H. J. Uang, ”A minmax tracking design for wheeled vehicles with trailed based adaptive fuzzy elimination
scheme,” IEEE Trans. Control Syst. Technol., vol. 8, no. 3, pp. 418-434, May 2000.
[20] C. S. Tseng, B. S. Chen and H. J. Uang, ”Fuzzy tracking control design for nonlinear dynamic systems via T-S fuzzy model,” IEEE
Trans. Fuzzy Systems, vol. 9, no. 3, pp. 381-392, 2001.
[21] B. S. Chen, C. P. Wang and M. Y. Lee, ”Stochastic robust team tracking control of multi-UAV networked system under Wiener and
Poisson random fluctuations,” IEEE Trans. Cyber, pp. 1-14, Jan. 2020.
[22] B. S. Chen, W. H. Chen and H. L. Wu, ”Robust H2/H∞ global linearization filter design for nonlinear stochastic systems,” IEEE
Trans. Circuits Syst. I, Reg. Papers, vol. 56, no. 7, pp. 1441–1454, Jul. 2009.
[23] K. A. Hoo and J. C. Kantor, ”Global linearization and control of a mixed-culture boireator with computation and external disturbance,”
Mathematical Bioscience, vol. 82, no. 1, pp. 43-62, Nov. 1986.
[24] S. Boyd, L. EI Ghaoui, E. Feron and V. Balakrishan, ”Linear matrix inequalities in system and control theory,” Philadelphia, PA:
SIAM, 1994.
[25] R. A. Nichols, R. T. Reichert and W. J. Rugh, ”Gain scheduling for H∞ controllers: A flight control example,” IEEE Trans. Control
System Technology, vol. 1, no. 2, pp. 69-79, Jun. 1993.
[26] B. S. Chen and C. F. Wu, ”Robust scheduling filter design for a calss of nonlinear stochastic signal systems,” IEEE Trans. Signal
Processing, vol. 63, no. 23, pp. 6245–6257, Dec. 2015.
[27] D. P. Kingma and J. Ba, ”Adam: A method for stochastic optimization,” ICLR, San Diego, pp. 1-15, arXiv preprint arXir: 1412.6980,
Dec. 2014.
[28] Y. N. Dauphin, H. D. Vries, J. Chung and Y. Bengio, ”RMSProp and equilibrated adaptive learning rates for non-convex optimization,”
Machine Learning, arXiv preprint arXiv:1502.04390, Feb. 2015.
[29] B. Zhao, B. Xian, Y. Zhang and X. Zhang, ”Nonlinear robust adaptive tracking control of a quadrotor UAV via immersion and invariance
methodology,” IEEE Trans. Ind. Electron., vol. 62, no. 5, pp. 2891-2902, May 2015.
[30] M. Y. Lee, B. S. Chen, C. Y. Tsai and C. L. Hwang, ”Stochastic H∞ robust decentralized tracking control of large-scale team
formation UAV network system with time-varying delay and packet dropout under interconnected couplings and Wiener fluctuations,”
IEEE Access, vol. 9, pp. 41976-41997, Mar. 2021.
[31] B. S. Chen, M. Y. Lee and T. H. Lin, ”DNN-based H∞ control scheme of nonlinear time-varying dynamic systems with external
disturbance and its application to UAV tracking design,” IEEE Access, vol. 9, pp. 69635-69653, May 2021.
[32] H. B. Abebe, C. L. Hwang, B. S. Chen, F. Wu and C. Jan, ”Recurrent neural network with fractional learning-based fixed-time formation
tracking constrained control for a group of quadrotors,” IEEE Access, vol. 9, pp. 81399-81411, May 2021.
[33] B. S. Chen, C. P. Wang and M. Y. Lee, ”Stochastic robust team tracking control of multi-UAV networked system under Wiener and
Poisson random fluctuations,” IEEE Trans. Cybernetics, pp. 1-14, Jan. 2020.
[34] K. Hornik, ”Multilayer feedforward networks are universal approximators,” Neural Network, vol. 2, pp. 359-366, 1989.
[35] W. Zhang, L. Xie and B. S. Chen, Stochastic H2/H∞ Control: A Nash Game Approach. Boca Raton, FL, USA: CRC Press, 2017.