在本篇論文之中，我們提出一個視訊資料模型來描述視訊資料的內涵資訊。對於視訊場景中所有出現的視訊物件，他們的感知以及語意資訊將會被紀錄下來，並且用來代表視訊場景的內涵資訊。對於不同的應用環境而言，我們將會利用預先建好的階層式知識模型來定義用來描述視訊物件的語意、事件與關聯的意涵。在我們所提出的視訊資料模型之中，除了一些基本的感知特性之外，視訊物件的移動軌跡也會被紀錄下來，並且用來推導一些物件本身或是物件之間的移動事件，像是「快速移動」或是「距離越來越遠」等高階的移動描述。此外，一些語意上的特性像是物件產生的行為或是物件之間的關聯性也都會紀錄下來。在這樣的視訊模型基礎之下，我們也提出了一個視訊資料的查詢語言，可以提供使用者利用視訊物件、物件產生的行為以及物件間的關係來描述心目中理想的視訊內涵作為查詢之用。
在索引與查詢處理上，我們提出了概念階層來提供使用者概念式的查詢。我們分別對於視訊物件的感知資訊，語意資訊以及移動特性配合概念階層來設計對應的索引結構藉以提供有效率且具彈性的查詢處理。此外，配合視訊資料模型中定義的階層式知識模型，我們也可以找到相似於使用者查詢的視訊資料作為查詢結果。
由於視訊資料可以表示成紀錄多種特徵值連續變化的多重屬性字串，因此，如果使用者查詢當中包含了q種特徵資料，視訊資料查詢的問題將可以被轉換成q重屬性字串比對的問題。傳統的字串比對方法並無法直接應用在q重字串比對的問題上。在本篇論文之中，我們針對精確比對以及近似比對分別提出了可以應用在不同q值上的索引結構以及相對應的查詢處理機制。經由實驗的驗證，我們所提出的方法的確可以快速而有效率的處理q重屬性字串比對的問題。

In this dissertation, a video data model is proposed to represent the content of video data. The perceptual and semantic properties of the video objects appearing in the video scene are recorded to represent the content of the video scene. For different applications, a predefined hierarchical knowledge model is used to express the semantic meanings contained in the videos such as the event happening in the video or the relationship between two objects in the video. In the proposed model, the trajectory and other properties of objects are recorded. From the trajectory, the motion events such as “high speed” of an object and “increasing distance” between objects can be automatically derived. Moreover, the semantic properties such as the actions performed by the object and the relationship between video objects are also recorded in the proposed model. A query language named V-SQL based on the video data model is also proposed for the users to describe the content of the desired video clips. The objects, the actions performed by the objects and the relations among objects are used to specify rich and complex semantic meanings in a query.
The concept hierarchy is used for user to specify concept query. The index structure, including the buckets for the attribute, suffix tree for motion sequence of video objects and the metadata table of the semantic properties of the video scene, is constructed to retrieve the matched video object based on the selected element in the concept hierarchy. The user specified concept query as well as the motion query will be processed through the index structure to find the corresponding video objects. Based on the proposed video model, the approximation query is also considered to find the similar retrieval result. Based on the proposed knowledge model, the retrieval system is able to make inferences on the predicates in a query, and determine whether a video semantically matches the query conditions. The semantic similarity measurement is also proposed for processing approximate queries.
Video data can be represented as a multiple-attribute string of feature values corresponding to multiple features of the data. Therefore, the retrieval problem can be transformed into the q-attribute string matching problem if q features are considered in a query. Traditional string matching algorithms cannot be efficiently applied to the q-attribute string matching problem. In this dissertation, two index structures and the corresponding matching algorithms, which can be applied on different values of q, are proposed for exact and approximate q-attribute string matching problem. The performance analysis and the experiment results are presented to show the efficiency of the proposed algorithm.

[1]. A. Aghbari, K. Kaneko and A. Makinoughi, “Modeling and Querying Videos by Content Trajectories,” in Proc. IEEE International Conference on Multimedia & Expo, 2000.
[2]. H.W. Agiuo and M.C. Angelides, “Modeling Content for Semantic-Level Querying of Multimedia,” in Multimedia Tools and Applications, Vol.15, No.1, 2001.
[3]. E. Ardizzone, M.L. Cascia and D. Molinelli, “Motion and Color-Based Video Indexing and Retrieval,” in Proceedings of Pattern Recognition, 1996
[4]. E. Ardizzone and M.S. Hacid, “A Semantic Modeling Approach for Video Retrieval by Content,” in Proc. IEEE International Conference on Multimedia of Computing Systems, 1999
[5]. E. Ardizzone and M.S. Hacid, “A Knowledge Representation and Reasoning Support for Modeling and Querying Video Data,” in Proc. IEEE International Conference: Tools with Artificial Intelligence, 1999
[6]. R.B. Yates and G.H. Gonnet, “A New Approach to Text Searching,” in Communications of the ACM, Vol.35, No.10, pp. 74-82, Oct. 1992
[7]. N. Beckmann, H.P. Kriegel, R. Schneider and B. Seeger, “The R*-Tree: An Eficient and Robust Access Method for Points and Rectangles,” in Proc. IEEE International Conference on Data Engineering, pp. 516-523, February, 1990.
[8]. J.L. Bentley, “Multidimensional Binary Search Trees Used for Associative Searching,” in Communications of ACM, Vol.18, No.9, pp.505-517, 1975.
[9]. A.D. Bimbo, “Visual Information Retrieval,” Morgan Kaufmann Publishers, 1999.
[10]. R.S. Boyer and J.S. Moore, “A Fast String Searching Algorithm,” in Communications of the ACM, Vol.20, Oct. 1977.
[11]. A.L.P. Chen, M. Chang, J. Chen, J.L. Hsu, C.H. Hsu and S.Y.S. Hua, “Query by Music Segments: An Efficient Approach for Song Retrieval,” in Proc. IEEE International Conference on Multimedia and Expo, 2000.
[12]. T. Chua and L. Ruan, “A Video Retrieval and Sequencing System,” in ACM Transactions on Information Systems, 1995.
[13]. T.H. Corman, C.E. Leiseson and R.L. Rivest, “Introduction to Algorithms,” The MIT Press: McGraw-Hill, 1993.
[14]. D. Berndt and J. Clifford, “Using dynamic time warping to find patterns in time series,” in AAAaI-94 workshop on knowledge discovery in databases, pp. 229-248.
[15]. S. Dagtas, W.A. Khatib, A. Ghafoor and A. Khokhar, “Trail-Based Approach for Video Data Indexing and Retrieval,” in Proc. IEEE International Conference on Multimedia Computing and Systems, pp. 235-239, June 1999.
[16]. S. Dagtas, W.A. Khatib, A. Ghafoor and R.L. Kashyap, “Models for Motion-Based Video Indexing and Retrieval,” in IEEE Transactions on Image Processing, Vol. 9, No. 1, pp.88-101, January 2000.
[17]. Y.F. Day, S. Dagtas, M. Iino, A. Khokhar , and A. Ghafoor, “Object-Oriented Conceptual Modeling of Video Data,” in Proc. IEEE International Conference on Data Engineering, pp401-408, 1995.
[18]. Y.F. Day, S. Dagtas, M. Iino, A. Khokhar, and A. Ghafoor, “Spatio-temporal modeling of video data for on-line object-oriented query processing,” in Proc. IEEE International Conference on Multimedia Computing and Systems, pp. 98-105, May, 1995.
[19]. Y.F. Day, A. Khokhar, S. Dagtas, A. Ghafoor, “A multi-level abstraction and modeling in video databases,” in ACM Multimedia Systems, Vol. 7, No. 5, pp. 400-423, 1999.
[20]. C. Decleir and M.S. Hacid, “Modeling and Querying Video Data: A Hybrid Approach,” in Proc. IEEE Workshop: Content-Based Access of Image and Video Libraries, 1998.
[21]. C. Decleir and M.S. Hacid, “Modeling and Querying Video Databases,” in Proc. Euromicro Conference, 1998.
[22]. C. Decleir and M.S. Hacid, “A Database Approach for Modeling and Querying Video Data,” in Proc. IEEE International conference on Data Engineering, 1999.
[23]. A.E. Lilac, A. Safadi and J.R. Getta, “Semantic Modeling for Video Content-Based Retrieval Systems,” in Proc. Computer Science Conference, ACSC 2000.
[24]. C. Faloutsos, “Searching Multimedia Databases by Content,” Kluwer Academic Publishers, 1996.
[25]. A. Guttmann, “R-Trees: A Synamic Index Structure for Spatial Searching,” in Proc. ACM SIGMOD International Conference on the Management of Data, pp. 47-57, June, 1984.
[26]. M.S Hacid, C. Decleir, and J. Kouloumdjian, “A database approach for modeling and querying video data,” in IEEE Transactions on Knowledge and Data Engineering, Vol.12, No.5, pp.729-750, Sept.-Oct. 2000.
[27]. M. Hee, Y.Y. Ik and K.C. Kim, “Intelligent Hybrid Video Retrieval System supporting Spatio-temporal correlation, Similarity retrieval,” in Systems, Man, and Cybernetics, 1999.
[28]. R. Hielsvold and R. Midtstraum, “ Modeling and Querying Video Data,” in Proc. International Conference on Very Large Databases, 1994
[29]. H.V. Jagadish, N. Koudas, and D. Srivastava. “On effective multi-dimensional indexing for strings,” in Proc. ACM SIGMOD International Conference on the Management of Data, pp.403-414, 2000.
[30]. H. Jiang, D. Montesi and A.K. Elmagarmid, “VideoText Database Systems,” in Proc. IEEE International Conference on Multimedia Computing and Systems, 1997
[31]. H. Jiang, A. K. Elmagarmid, “Spatial and Temporal Content-Based Access to Hypervideo Database,” in The VLDB Journal, Vol. 7, No. 4, pp. 226-238, 1998
[32]. T. Kahveci and A. Singh, “Efficient index structures for string databases,” in Proc. International Conference on Very Large Databases, 2001.
[33]. T. Kahveci, A. Singh and A. Gurel, “Similarity Searching for Multi-attribute Sequences,” in Proc. International Conference on Scientific and Statistical Database Management, pp. 175-184, 2002.
[34]. W.A. Khatib and A. Ghafoor, “An Approach for Video Meta-Data Modeling and Query Processing,” in Proc. ACM International Conference on Multimedia, pp. 215-224, Orlando, 1999.
[35]. D.E. Knuth, J.H. Morris, V.R. Pratt, “Fast Pattern Matching in Strings,” in SIAM J. Computing, pp.323-350, 1977.
[36]. J.L. Koh, C.S. Lee and A.L.P. Chen, “Semantic Video Model for Content-Based Retrieval,” in Proc. IEEE International Conference on Multimedia Computing and Systems, 1999
[37]. T.C.T. Kuo and A.L.P. Chen, “A Content-based Query Language for Video Databases,” in Proc. of IEEE International Conference on Multimedia Computing and Systems, June 1996.
[38]. T.C.T. Kuo and A.L.P. Chen, “Indexing Query Interface and Query Processing for Venus: A Video Database System,” in Proc. Cooperative Databases for Advance Applications, 1996.
[39]. T.C.T. Kuo and A.L.P. Chen, “Content-Based Processing for Video Databases,” in IEEE Transactions on Multimedia, Vol. 2, No. 1, pp. 1-13, 2000.
[40]. S.L. Lee, S.J. Chun, D.H. Kim, J.H. Lee and C.W. Chung, “Similarity Search for Multidimensional Data Sequences,” in Proc. International Conference on Data Engineering, pp.599-608, 2000
[41]. W. Lee and A.L.P. Chen, “Efficient Multi-Feature Index Structures for Music Data Retrieval,” in Storage and Retrieval for Media Databases, Proceedings of SPIE Vol. 3972, pp.177-188, 2000.
[42]. V. Levenshtein. “Binary codes capable of correcting spurious insertions and deletions of ones,” in Problems of Information Transmission, 1:8-17, 1995
[43]. J.Z. Li, M.T. Ozsu, and D. Szafron, “Modeling of moving objects in a video databases,” in Proc. IEEE International Conference on Multimedia computing and Systems, pp. 336-343, 1997.
[44]. C.H. Lin and A.L. P. Chen, “Motion Event Derivation and Query Language for Video Databases,” in Storage and Retrieval for Media Databases, Proceedings of SPIE Vol. 4315, 2001.
[45]. C.H. Lin and A.L.P. Chen, “Indexing and Query Processing for Video Databases,” in Proc. IEEE International Conference on Image Processing, 2001.
[46]. C.H. Lin, A.H.C. Lee and A.L.P. Chen, “A Semantic Model for Video Description and Retrieval,” in Proc. IEEE Pacific-Rim Conference on Multimedia, 2002.
[47]. C.H. Lin and A.L.P. Chen, “Indexing and Matching Multiple-Attribute Strings for Efficient Multimedia Query Processing,” in IEEE Transactions on Multimedia (to appear).
[48]. C.H. Lin and A.L.P. Chen, “Approximate Video Search Based on Spatio-Temporal Information of Video Objects,” submitted for publication, 2005.
[49]. C.C. Liu and A.L.P. Chen, “Vega: A Multimedia Database System Supporting Content-Based Retrieval,” in Journal of Information Science and Engineering, Vol. 13, No. 3, pp. 369-398, 1997.
[50]. C.C. Liu, J.L. Hsu and A.L.P. Chen, “An Approximate String Matching Algorithm for Content-Based Music Data Retrieval,” in Proc. IEEE International Conference on Multimedia Computing and Systems, pp.105-112, 1999.
[51]. C.C. Liu and A.L.P. Chen, “3D-List: A Data Structure for Efficient Video Query Processing,” in IEEE Transastions on Knowledge and Data Engineering, Vol. 14, No. 1, pp.106-122, 2002.
[52]. E. McCreight, “A Space-Economical Suffix Tree Construction Algorithm,” in Journal of Association for Computing Machinery, pp. 262-272, 1976.
[53]. G. Navarro, “A Guided Tour to Approximate String Matching,” in ACM Computing Surveys, Vol. 33, No. 1, pp. 31–88, March 2001.
[54]. J. Nievergelt, H. Hinterberger and C. Sevcik, “The Grid File: An Adaptable, Symmetric, multikey file structure,” in ACM TODS, Vol. 9, March, pp. 38-71, 1984.
[55]. E. Oomoto and K. Tanaka, “OVID: Design and Implementation of a Video-Object Database System,” in IEEE Transactions on Knowledge and Data Engineering, Vol. 5, No. 4, pp. 629-643, 1993.
[56]. S. Pradhan, K. Tajima, K. Tanaka, “Querying Video Databases based on Description Substantiality and Approximations,” in Proc. IPSJ International Symposium on Information Systems and Technologies for Network Society, September 1997
[57]. N. Roussopulos, S. Kelley and F. Vincent, “Nearest Neighbor Queries,” in Proc. ACM SIGMOD International Conference on the Management of Data, pp. 71-79, June, 1995
[58]. H. Samet, “Hierarchical Representations of Collections of Small Rectangles,” in ACM Computing Surveys, Vol.20, No.4, pp.271-309, 1988.
[59]. H. Samet, “The Design and Analysis of Spatial Data Structures,” Addison Wesley, 1989
[60]. B. Seeger and H.-P. Kriegel, “The Buddy-Tree: An Efficient and Robust Access Method for Spatial Database Systems,” in Proc. of VLDB, pp.590-601, Augus 1990.
[61]. T.G.A. Smith and G. Davenport, “The Stratification System: A Design Environment for Random Access Video,” in Proc. Workshop on Networking and Operating System Support for Digital Audio and Video, 1992.
[62]. S.W. Smoliar and H.J. Zhang, “Content Based Video Indexing and Retrieval,” in IEEE Multimedia, 1994.
[63]. U. Srinivasan and G. Riessen, “A Video Data Model for Content-Based Search,” in Database and Expert System Applications, 1997.
[64]. K. Uehara, M. Oe and K. Maehara, “Knowledge Representation, Concept Acquisition and Retrieval of Video Data,” in Proc. Cooperative Database And Applications, 1996
[65]. T.T.Y. Wai and AL.P. Chen, “Retrieving Video Data via Motion Tracks of Content Symbols,” in Proc. International Conference on Information and Knowledge Management, pp. 105-112, Nov. 1997.
[66]. M. Washisaka, T. Takada, S. Anyagi and R. Onai, “Video/Text Linkage System Assisted by a Concept Dictionary and Image Recognition,” in Proc. IEEE International Conference on Multimedia Computing and Systems, 1996
[67]. P. Weiner, “Linear pattern matching algorithms,” in Proc. IEEE 14th Annual Symposium on Switching and Automata Theory, pp.1-11, 1973.
[68]. I.H. Witten, A. Moffat and Y.C. Bell, “Managing Gigabytes,” Van Nostrand Reinhold, 1994.
[69]. S. Wu and U. Manber, “Fast Text Searching Allowing Errors,” in Communication of the ACM, Vol.35, No.10, pp.83-91, Oct.1992.
[70]. D. Zhong and S.F. Chang, “Video Object Model and Segmentation for Content-Based Video Indexing,” in IEEE International Conference on Circuits and Systems, June, 1997, Hong Kong