3D手指動作捕捉是近來人機介面系統中最為直覺的方法，基於即時性以及精確度的需求，本論文提出了一個以手部剪影上深度分布為估測方式的模型估測法(Depth distribute Model-based Estimation)來估測出重建出手勢所需的運動關節參數。我們利用手部剪影上的深度分布當成我們模型中的範例。也就是說，以此手部遮罩以及深度分布為依據，並且使用差異函數(difference function)計算與模型中各範例的difference value當成相似程度，藉此篩選出模型中與目前輸入畫面手勢最近似的手部關節角度參數。
此估測系統的基本原則為：若是當前輸入畫面的手勢深度分布以及遮罩大小與手部模型中的某個手勢非常相似的話，我們即認為目前輸入畫面的手勢參數與此模型的手勢參數相同。因為手部模型相當龐大，為了降低運算量以達到即時性並且同時兼顧精確度，我們提出了PSO(Particle Swarm Optimization)演算法，這個演算法模仿於鳥群尋找食物行為，其優點是(1)分散式搜尋可避免陷入了找到local optimal solution的泥淖中。(2)各個Particle會分享記憶給其他的Particle，然後一起合作找到最佳參數。(3)可以藉由粒子(Particle)數量以及最大疊代次數來控制搜尋時的最大計算量。其演算法適合在連續性的範圍內搜尋，為了讓此演算法可以在離散性的範圍內運作外，也要讓整套系統不會隨著手部模型越來越大而使運算量線性增加，我們使用LHS(Locality-Sensitive Hashing)來對手部模型內做排序(Sorting)的動作，如此一來當PSO疊代過程中估測出新的參數時並在模型中找最相似的樣本時，搜尋動作就會被簡化為簡單快速的查表動作。他可以有效的降低在模型中搜尋最近樣本的計算時間，並且不會隨著模型大小的增加而線性增加計算時間。最後再配合PSO演算法快速估測手勢的運動關節參數。

Recently, hand poses estimation is a very popular method in human-computer interface, virtual reality. In this thesis, we introduce a model-based method: to search appropriate parameter in 3-D hand model by Particle Swarm Optimization. To improve this algorithm, we combine difference function as distance and apply LHS(Locality-Sensitive Hashing) as that PSO can run in a discrete hand model successfully. The difference function, which is used to evaluate the similarity of the samples in hand model and input image’s depth distribution. Using PSO for the discrete hand model, we compute the smallest distance between hand model and the new estimate parameter in each iteration. To reduce this compute, we use LHS, searching in the look-up table. Combining PSO, difference function and LHS, our system and achieve in real-time successfully.
First, we obtain the hand silhouette using depth map. Because of depth map, it is insensitive in light change. Then we have depth distribution from kinect sensor and 16-D hand parameter by using 5DT data glove as a sample. With many samples in a 3-D hand model, we may search the model by using LHS. Based on the 3-D hand model construction, we estimate the hand pose in input image. In testing process, with the depth distribute of the hand, we use difference function to evaluate distance between model and input image, and PSO should estimate parameter by many particles and iterations. By control the number of the particle and iteration, we can achieve the estimate system run in real-time.

[1] S. Belongie, J. Malik, and J. Puzicha. Shape Matching and Object Recognition Using Shape Contexts. PAMI, 24(4):509–522, 2002.
[2] Iason Oikonomidis, Nikolaos Kyriazis and Antonis A. Argyros, Efficient Model-based 3D Tracking of Hand Articulations using Kinect. BMVC, 2011.
[3] M.W. Lee and I. Cohen. A Model-Based Approach for Estimating Human 3D Poses in Static Images. PAMI, 28(6):905-916, 2006.
[4] Xinghua Wu. A Density Adjustment Based Particle Swarm Optimization Learning Algorithm For Neural Network Design. ICECE, 2011.
[5] G. Mori and J. Malik. Recovering 3D Human Body Configurations Using Shape Contexts. PAMI,28(7):1052-1062, 2006.
[6] A. Gionis, P. Indyk, and R. Motwani. Similarity Search in High Dimensions via Hashing. VLDB, 1999.
[7] G. Shakhnarovich, P. Viola, and T. Darrell. Fast Pose Estimation with Parameter Sensitive Hashing. ICCV, 2003.
[8] J. Rehg and T. Kanade, “Model-Based Tracking of Self-Occluding Articulated Objects,” Proc. IEEE Int’l Conf. Computer Vision, pp. 612-617, 1995.
[9] L. Y. Chang, N. S. Pollard, T. M. Mitchell, and E. P. Xing, "Feature selection for grasp recognition from optical markers ," in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007.
[10] V. Athitsos and S. Sclaroff, "Estimating 3D hand pose from a cluttered image ," in IEEE Conference on Computer Vision and Pattern Recognition, 2003, pp. 432-439.
[11] M. T. Ciocarlie, S. T. Clanton, M. C. Spalding, and P. K. Allen, "Biomimetic grasp planning for cortical control of a robotic hand," in IEEE/RSJ International Conference on Intelligent Robots and Systems , 2008 , pp. 2271-2276.
[12] R. Benetis, C. S. Jensen, G. Karciauskas, and S. Saltenis "Nearest and reverse nearest neighbor quer ies for moving objects ," The VLDB Journal, Vol. 15, no. 3, pp. 229-250, 2006.
[13] S. Malassiotis , N. Aifanti and M. G. Strintzis "Personal authentication using 3-D finger geometry", IEEE Trans. Inf. Forensics Security, vol. 1, no. 1, pp.12 -21 2006
[14] G. Zheng , C. J. Wang and T. E. Boult "Application of projective invariants in hand geometry biometrics", IEEE Trans. Inf. Forensics Security, vol. 2, no. 4, pp.758 -768 2007
[15] Microsoft Corp. Remond WA, Kinect for Xbox 360.
[16] A. Morales , M. Ferrer , F. Daz , J. Alonso and C. Travieso "Contact-free hand biometric system for real environments", Proc. 16th Eur. Signal Process. Conf., 2008
[17] J. Deutscher, A. Blake, and I. Reid” Articulated Body Motion Capture by Annealed Particle Filtering” IEEE CVPR 2000.
[18] P. Peursum. “On the Behaviour of Body Tracking with the Annealed Particle Filter in Realistic Conditions” Technical Report of the Dept. of Computing, Curtin Univ. of Technology, 2006.
[19] S. Malassiotis, N. Aifanti, and M. G. Strintzis, "Personal Authentication using 3-D finger geometry", IEEE Trans. Info. Forensics & Security, vol.1, no. 1, pp. 12-21, Mar. 2006.
[20] A. Kumar and Ch. Ravikanth, "Personal authentication using finger knuckle surface", IEEE Trans. Info. Forensics & Security, vol. 4, no. 1, pp. 98-110, Mar. 2009.
[21] Sajjawiso, T. “3D Hand pose modeling from uncalibrate monocular images ”, JCSSE 2011.
[22] X. K. Wei and J. Chai, "Modeling 3d human poses from uncalibrated monocular images", in Computer Vision, 2009 IEEE 12th International Conference on, 29 2009-oct.2 2009, pp. 1873-1880.
[23] Hamer, H. , Gall, J. , Weise, T. , Van Gool, L. “An object-dependent hand pose prior from sparse training data ”, CVPR 2010.
[24] H. Kjellström, J. Romero, D. M. Mercado, D. Kragic. Simultaneous visual recognition of manipulation actions and manipulated objects. ECCV, 2008.
[25] B. Rosenhahn, C. Schmaltz, T. Brox, J. Weickert, D. Cremers, H.-P. Seidel. Markerless motion capture of manmachine interaction. CVPR, 2008.
[26] Martin de La Gorce, Nikos Paragios, David J. Fleet. Model-Based Hand Tracking with Texture, Shading and Self-occlusions. CVPR, 2008.
[27] V. Athitsos, J. Alon, S. Sclaroff, and G. Kollios. Monocular real-time 3D articulated hand pose estimation. IEEE-RAS, 2009.
[28] Matthieu Bray, Esther Koller-Meier, Luc Van Gool, Smart Particle Filtering for 3D Hand Tracking, FGR, 2004