強化學習為機器學習中的一個分支，它不僅可以學習資料間的關聯，
還可以解決最優控制的問題，因而被視為是達到通用人工智慧的方法之
一。
然而這個研究領域大多專注在如何有效率的計算價值函數上，較少有
人討論最優策略的存在性，確定最優策略的存在可以減少因為最優策略不
存在而造成的演算法發散問題。更進一步若能知道最優策略的簡單形式能
大幅減少資料運算量而使計算加快。
本篇論文討論強化學習在離散時間、隨機獎勵、歷史性策略下最優策
略的存在性證明。我們證明了只要每個時刻動作空間是有限的，最優策略
即存在，並且在計算最優策略時只需考慮馬可夫最優平穩策略即可，這可
以大幅減少需要考慮的策略集合。

This thesis studies Markov decision processes with probabilistic rewards, expected discounted return, discrete-time, finite action space, stationary transition probability, and probabilistic nonstationary policy. In this thesis, we focus on the existence of optimal policies under this framework. We prove that an optimal policy exists if the set of actions that can be selected at each moment is finite. And when computing the optimal policy, we can restrict our attention to stationary deterministic Markovian policies that only consider the current state. This can greatly reduce the computational complexity.

[1] A. Rao, and T. Jelvis, “Foundations of reinforcement learning with applications
in finance,”[Unpublished manuscript] Stanford University, 2020.
[2] D. Blackwell, “Discounted dynamic programming,” The Annals of Mathematical
Statistics 36.1: 226-235. 1965
[3] A. Gosavi, Simulation-based Optimization, Berlin: Springer, 2015.
[4] M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic
Programming, New York: John Wiley Sons, 2014.
[5] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd
ed. Cambridge, MA: MIT Press, 2018.