研究生: |
王靜媛 Wang,Ching-Yuan |
---|---|
論文名稱: |
潛在狄利克雷分佈下證據下界的研究 The Study of Evidence Lower Bound under Latent Dirichlet Allocation |
指導教授: |
張延彰
Chang,Yen-Chang |
口試委員: | |
學位類別: |
碩士 Master |
系所名稱: |
南大校區系所調整院務中心 - 應用數學系所 應用數學系所(English) |
論文出版年: | 2016 |
畢業學年度: | 104 |
語文別: | 中文 |
論文頁數: | 15 |
中文關鍵詞: | 潛在狄利克雷分佈 、證據下界 、變分推論 、Kullback信息量 、Chernoff信息量 |
外文關鍵詞: | Latent Dirichlet Allocation, Evidence Lower Bound, Variational Inference, Kullback information, Chernoff information |
相關次數: | 點閱:3 下載:0 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
潛在狄利克雷分佈(Latent Dirichlet Allocation,LDA)在文件採礦(Text mining)中是被廣泛採用的模型,它擁有很好的判斷及訓練模式,然而參數估計卻相對複雜,於是採用了變分貝氏(Variational Bayes)估計的方法,此方法可使其得到一個不錯的估計結果。在變分貝氏估計中,需先找到一個概似函數的證據下界(Evidence Lower Bound,ELBO),本文中,我們利用Chernoff 信息量推導得一個信新的證據下界,並證明此下界較原先的為大。
Latent Dirichlet Allocation is a model which is widely used in text mining. It also has good judgment and training mode. Howere, the estimation of its parameter is relatively
complex. So we use the method, Variational Bayes, to get a good result of estimation. In Variational Bayes, we need to find an Evidence Lower Bound of likelihood function. In this paper we derive a new Evidence Lower Bound from Chernoff information and prove that it is bigger than the original one.
[1] Wikipedia, Data mining, https : //en:wikipedia:org/wiki/Data_ mining.
[2] Darroch, J.N. and Ratcliff, D. (1972) Generalized iterative scaling for log-linear models.The Annals of Mathematical Statistics, 43:1470–1480.
[3] Adam Berger. (1997) The Improved Iterative Scaling Algorithm: A Gentle Introduction,http://www.cs.cmu.edu/ aberger/pdf/scaling.pdf.
[4] Kilian Q. Weinberger, Matt J. Kusner, Nicholas I. Kolkin, Yu Sun. (2015) From Word Embeddings To Document Distances. The International Conference on Machine Learning(ICML), Washington University in St. Louis , 1 Brookings Dr., St . Louis, MO 63130.
[5] Andrew Y. Ng,David M. Blei ,Michael I. Jordan and John Lafferty. (2003) Latent Dirichlet Allocation. Journal of Machine Learning Research, 3:993–1022.
[6] David M. Blei, Francis Bach, Matthew D. Hoffman.(2010) Online Learning for Latent Dirichlet Allocation. Advances in Neural Information Processing Systems(NIPS),856-864.
[7] David M. Blei. (2011) Variational Inference. Lecture from Princeton, variational inference, https://www.cs.princeton.edu/courses/archive/fall11/cos597C/lectures/variationalinference-i.pdf
[8] Chernoff, Herman. (1952) A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations. Annals of Mathematical Statistics, 25, 573-578.