簡易檢索 / 詳目顯示

研究生: 王靜媛
Wang,Ching-Yuan
論文名稱: 潛在狄利克雷分佈下證據下界的研究
The Study of Evidence Lower Bound under Latent Dirichlet Allocation
指導教授: 張延彰
Chang,Yen-Chang
口試委員:
學位類別: 碩士
Master
系所名稱: 南大校區系所調整院務中心 - 應用數學系所
應用數學系所(English)
論文出版年: 2016
畢業學年度: 104
語文別: 中文
論文頁數: 15
中文關鍵詞: 潛在狄利克雷分佈證據下界變分推論Kullback信息量Chernoff信息量
外文關鍵詞: Latent Dirichlet Allocation, Evidence Lower Bound, Variational Inference, Kullback information, Chernoff information
相關次數: 點閱:3下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 潛在狄利克雷分佈(Latent Dirichlet Allocation,LDA)在文件採礦(Text mining)中是被廣泛採用的模型,它擁有很好的判斷及訓練模式,然而參數估計卻相對複雜,於是採用了變分貝氏(Variational Bayes)估計的方法,此方法可使其得到一個不錯的估計結果。在變分貝氏估計中,需先找到一個概似函數的證據下界(Evidence Lower Bound,ELBO),本文中,我們利用Chernoff 信息量推導得一個信新的證據下界,並證明此下界較原先的為大。


    Latent Dirichlet Allocation is a model which is widely used in text mining. It also has good judgment and training mode. Howere, the estimation of its parameter is relatively
    complex. So we use the method, Variational Bayes, to get a good result of estimation. In Variational Bayes, we need to find an Evidence Lower Bound of likelihood function. In this paper we derive a new Evidence Lower Bound from Chernoff information and prove that it is bigger than the original one.

    1 緒論 1 2 文獻回顧 2 2.1 文件分類 2 2.2 LDA 3 3 變分推論 5 3.1 變分貝氏估計方法 5 3.2 變分貝氏估計公式推導 7 4 證據下界ELBO 9 4.1 證據下界的介紹 9 4.2 新證據下界之推導 11 5 結論 14 5.1 結論 14 5.2 限制與未來建議14

    [1] Wikipedia, Data mining, https : //en:wikipedia:org/wiki/Data_ mining.
    [2] Darroch, J.N. and Ratcliff, D. (1972) Generalized iterative scaling for log-linear models.The Annals of Mathematical Statistics, 43:1470–1480.
    [3] Adam Berger. (1997) The Improved Iterative Scaling Algorithm: A Gentle Introduction,http://www.cs.cmu.edu/ aberger/pdf/scaling.pdf.
    [4] Kilian Q. Weinberger, Matt J. Kusner, Nicholas I. Kolkin, Yu Sun. (2015) From Word Embeddings To Document Distances. The International Conference on Machine Learning(ICML), Washington University in St. Louis , 1 Brookings Dr., St . Louis, MO 63130.
    [5] Andrew Y. Ng,David M. Blei ,Michael I. Jordan and John Lafferty. (2003) Latent Dirichlet Allocation. Journal of Machine Learning Research, 3:993–1022.
    [6] David M. Blei, Francis Bach, Matthew D. Hoffman.(2010) Online Learning for Latent Dirichlet Allocation. Advances in Neural Information Processing Systems(NIPS),856-864.
    [7] David M. Blei. (2011) Variational Inference. Lecture from Princeton, variational inference, https://www.cs.princeton.edu/courses/archive/fall11/cos597C/lectures/variationalinference-i.pdf
    [8] Chernoff, Herman. (1952) A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations. Annals of Mathematical Statistics, 25, 573-578.

    無法下載圖示 全文公開日期 本全文未授權公開 (校內網路)
    全文公開日期 本全文未授權公開 (校外網路)

    QR CODE