研究生: |
吳文哲 Wen-Jer Wu |
---|---|
論文名稱: |
高效能測試個案設計之研究 On the Study of Designing Efficient Test Cases |
指導教授: |
唐傳義
Chuan Yi Tang |
口試委員: | |
學位類別: |
博士 Doctor |
系所名稱: |
電機資訊學院 - 資訊工程學系 Computer Science |
論文出版年: | 2000 |
畢業學年度: | 88 |
語文別: | 英文 |
論文頁數: | 84 |
中文關鍵詞: | 記憶體的測試 、通訊協定的測試 |
外文關鍵詞: | memory testing, protocol testing, test cost reduction, test timing assignment, synchronizable test sequences generation, rural Chinese postman problem, weighted set-covering problem, linear programming |
相關次數: | 點閱:2 下載:0 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
測試的目的在檢驗實作是否符合其規格;測試的流程,無論在測試個案的開發階段或是量產產品的測試階段均是非常耗時耗資的。如何有效的設計測試個案以達到減少測試所需時間及花費是本論文所欲探討的主題。在此論文中,我們將要針對幾個高效能測試個案設計的主題進行研究及討論,包括:測試時序(測試時序的自動安排方法)、縮減測試時間(移除多餘的測試個案的方法及串聯測試個案的方法)及簡化測試系統(可同步測試個案的產生方法);一些由測試記憶體及通訊協定所衍生出來的問題。
測試記憶體的目的是要檢驗記憶體晶片的時序及功能是否正常。我們將從解決自動測試時序安排的問題開始著手,本文所提出的方法可以同時把測試的目標、測試的規格及測試機的限制列入考慮。然後把指定測試時序的問題轉換成線性規劃(linear programming)的模型,由現存的線性規劃解決器自動解決。
多餘的測試個案(Redundant test cases)是可以移除而不會減低錯誤的包含率(fault coverage)。藉由移除多餘的測試個案來達成縮減測試時間,可同時運用於測試個案的開發階段(有明確的錯誤模型)及量產產品測試的階段(錯誤模型不明確,但可以蒐集到測試的結果)。在測試個案的開發階段,可以藉由分析每個測試個案在錯誤模型中可以偵測到的錯誤。選擇一組具有完整的錯誤包含率以及測試時間最少的測試個案當成測試程式,將使得量產產品測試的實施時間最短。然而在進入量產產品的測試後,也可能會有多餘的測試個案發生,其原因是來自於理論上的錯誤模型與實際上發生的結果有偏差所致。本文的方法是去蒐集及分析未通過測試的晶片其未通過的項目,並由這些未通過的項目來進一步移除多餘的測試個案。本文的研究結果顯示,這兩個移除多餘測試個案的問題是一樣的,都可以轉換成一個具權值的集合包含問題(weighted set-covering problem)。在實際應用上,我們可以將其轉換成整數型線性規劃(integer linear programming)的模型,由現存的線性規劃解決器自動解決。
在量產產品測試的過程中,我們可以利用串聯測試個案的方式,再進一步的縮減測試的時間。其主要的觀念是要充分的利用前一組測試個案所遺留下來的記憶體狀態,使得下一組測試個案可以省去設定初值的動作;而下一組測試個案的起始驗證動作也可以省去前一組測試個案的後段驗證動作。此外,測試機的信號設定時間也將因串聯的結果而減為最少。在本論文中,我們把串聯測試個案的問題轉換成一個鄉村中國郵差問題(rural Chinese postman problem),並加以解決。但是當我們把鄉村中國郵差問題轉換成一個整數型線性規劃的模型時,卻發現限制(constrains)的數目可能達到指數級(exponential),而無法直接列在一個模型上來解決。在本論文中,我們將提出限制漸增法來解決這個問題;而其方法主要是迴避直接列在一個模型所衍生的問題,轉而去解決一連串限制較少的整數型線性規劃模型,並分析解決鄉村中國郵差問題所需要的模型及限制數目。
測試的同步問題常發生在需要兩個以上的測試機來同步完成測試的測試系統中,因此常見於多方通訊協定(multi-party communication protocol)的測試。在本論文中,我們提出一個多重圖(multiplex digraph)模型來作為產生可同步測試個案的參考;多重圖模型可以適用於各種形式的測試系統架構,並可同時將測試個案所選用的輸出/輸入指令及同步指令的花費列入考慮;並將利用多重圖模型來產生花費最少的可同步的轉移序列、可同步的 T 方法序列、以及可同步的重疊 U 方法序列,並分析其複雜度。
The objective of testing is to see if an implementation conforms to its specification. Testing is costly and time-consuming on both development and test application phases. How to efficiently design test cases for minimizing test time and cost is considered. In this dissertation, we study several efficient test case problems, including timing issue (test timing assignment), test time reduction issues (redundant test cases removal and test cases interconnection), and test architecture simplification issue (synchronizable test case generation), which are arisen from memory testing and communication protocol testing.
The purpose of memory testing is to ensure that memory chip can work correctly in timing as well as functionality. We begin with solving timing assignment problem automatically. The proposed method is comprehensive enough to take the test objective and constraints from specification and tester into consideration. The test timing assignment problem is then transformed into a linear programming model, which can be automatically solved.
Redundant test cases can be removed without reducing fault coverage. Test time reduction by removing redundant test cases for both test development phase, which has a well-defined fault model, and production test phase, which has no fault model, is discussed. During the test development phase, the process is to analyze fault detectability with respect to a given fault model. An optimal set of test cases is selected to compose the final test program, which results in minimum test application time. After proceeding to the production test phase, redundancy can also exhibit due to dependency of faults or discord between theoretical and practical fault models. The process is to analyze fail labels to remove redundant test cases optimally. Our result shows that these two methods are just dual to each other, and moreover, their underlying problem is an instance of weighted set-covering problem. In practical application, these two problems can be transformed into integer linear programming models, which can be automatically solved.
As the production test process goes, a further test time reduction technique can be applied. The idea is to interconnect test cases to reuse memory states left from the previous test case for saving initialization and verification sequences as well as signal settling time of tester. The interconnection problem is transformed into an instance of Rural Chinese Postman (RCP) problem. However, the number of constraints could be exponential in the worst case if transforming RCP problem into an integer linear programming model. In this dissertation, a novel incremental constraining method is proposed, and our approach is to solve a number of successive integer linear programming models with smaller number of constraints. The total numbers of iterations and constraints applied to solve RCP problem are analyzed and compared.
The test synchronization problem happens in a special test architecture, which requires at least two remote testers to test the implementation coordinately. This problem is usually encountered in testing multi-party communication protocol implementations. Testing system can be significantly simplified if synchronizable test cases are applied. In this dissertation, we propose a multiplex digraph model for synchronizable test case generation, which will consider both the costs of input/output operations and external synchronization operations. Applications for generating minimum-cost synchronizable transfer sequence, T-method test sequence, and Overlapping U-method test sequence are discussed and analyzed.
[Abramovici90] M. Abramovici, M. A. Breuer, and A. D. Friedman, Digital Systems Testing and Testable Design, Computer Science Press, 1990.
[ADVANTEST96] ADVANTEST Co., T55xx Series Memory Test System Programming Reference Manual, 1996.
[Aho91] A. V. Aho, A. T. Dahubra, D. Lee and M. U. Uyar, "An optimization technique for protocol conformance test generation based on UIO sequences and rural Chinese postman tours," IEEE Transaction on Communications, Vol. 39, No. 11, pp. 1604-1615, 1991.
[Berg96] J. P. Van den Berg, "Multiple order pick sequencing in a carousel system: a solvable case of the rural postman problem," Journal of the Operational Research Society, Vol. 47, pp. 1504-1515, 1996.
[Berkhout90] S. R. Berkhout and M. F. Witteman, "Application of multi-party conformance test methods to the GSM mobile network system," Proceedings of IFIP 3rd International Workshop on Protocol Test Systems, McLean, Virginia, U.S.A., October 1990.
[Bochmann78] G. v. Bochmann, "Finite state description of communication protocols," Computer Networks, Vol.2, pp. 362-372, 1978.
[Bochmann80] G. v. Bochmann, "A general transition model for protocols and communication services," IEEE Transaction on Communications, Vol. Com-28, pp. 643-650, April 1980.
[Bochmann83] G. von Bochmann and C. A. Sunshine "A survey of formal methods," Computer Networks and Protocols, Plenum Press, New York, pp 561-578, 1983.
[Boyd91] S. C. Boyd and H. Ural, "The synchronization problem in protocol conformance testing," Information Processing Letters, Vol. 40, pp 131-136, November 1991.
[Chan89] W. Y. L. Chan, S. T. Vuong, and M. R. Ito, "An improved protocol test generation procedure based on UIOs", Proceedings of ACM SIGCOMM, pp 283-294, 1989.
[Chanson90] S. T. Chanson, S. T. Vuong, and H. Dany, "The ferry clip approach to multi-party testing," Proceeding of 3rd IFIP International Workshop on Protocol Test Systems, McLean, VA, October 1990.
[Chen90] W.-H. Chen, C. S. Lu, L. Chen, and J. T. Tang, "Synchronizable protocol test generation via the duplex technique," Proceedings of IEEE INFOCOM'90, pp. 561-563, 1990.
[Chen90] W.-H. Chen, C. S. Lu, E. R. Brozovsky, and J. T. Wang, "An optimization technique for protocol conformance testing using multiple UIO sequences," Information Processing Letters, Vol. 36, No. 1, pp. 7-11, 1990.
[Chen94] W.-H. Chen, The Study on Test Sequence Generation for Protocol Conformance Testing, Ph.D. Dissertation, Department of Computer Science, National Tsing-Hua University, Taiwan, January 1994.
[Chen95] W.-H. Chen and H. Ural, "Synchronizable test sequences based on multiple UIO sequences," IEEE/ACM Transaction Network, Vol.3, No. 2, pp.152-157, April 1995.
[Chow78] T. S. Chow, "Testing software design modeled by finite-state machines," IEEE Transaction on Software Engineering, Vol. SE-4, No. 3, pp. 178-187, 1978.
[Christofides86] N. Christofides, V. Campos, A. Corberan and E. Mota, "An algorithm for the rural postman problem on a directed graph," Mathematical Programming Study, Vol. 26, pp. 155-166, 1986.
[Chvatal79] V. Chvatal, "A greedy heuristic for the set-covering problem," Mathematics of Operation Research, Vol. 4, No. 3, pp. 233-235, 1979.
[Cockburn94] B. F. Cockburn, "Tutorial on semiconductor memory testing," Journal of Electronic Testing: Theory and Applications, Vol. 5, pp.321-335, 1994.
[Coremen90] T. H. Cormen, C. E. Leiserson, and R. L. Rivest, Introduction to Algorithms, MIT press, 1990.
[Dekker90] R. Dekker, et al., "A Realistic fault model and test algorithms for static random access memories," IEEE Transaction on Computers, Vol. C-9, No. 6, pp. 567-572, 1990.
[Dror97] M. Dror, A. Langeoin, "Generalized travelling salesman problem approach to the directed clustered rural postman problem," Transportation Science, Vol. 31, pp. 187-192, 1997.
[Edmonds73] J. Edmonds and E. L. Johnson, "Matching, Euler tours and the Chinese postman," Mathematical Programming, Vol. 5, pp. 88-124, 1973.
[Eswardan76] K. E. Eswardan and R. E. Tarjan, "Augmentation problems," SIAM Journal on Computing, Vol. 5, No. 4, pp. 653-665, 1976.
[Fujiwara91] S. Fujiwara, G. von Bochmann, F. Khendek, M. Amalou, and A. Ghedamsi, "Test selection based on finite state models," IEEE Transaction on Software Engineering, Vol 17, No 6, pp. 591-603, June 1991.
[Garey79] M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP- Completeness, Freeman, San Francisco, CA, 1979.
[Gibbons85] A. Gibbons, Algorithmic Graph Theory, Cambridge Univ. Press, Cambridge, 1985.
[Goenenc70] G. Goenenc, "A method for the design of fault detection experiments," IEEE Transaction on Computer, Vol. C-19, pp. 551-558, 1970.
[Golumbic80] M. C. Golumbic, Algorithmic Graph Theory and Perfect Graphs, Academic Press, 1980.
[Hochbaum96] D. S. Hochbaum, "An optimal test compression procedure for combinational circuits," IEEE Transaction on Computer-Aided Design of Integrated Circuits and Systems, Vol. 15, No. 10, Oct. 1996.
[Goor91] A. J. van de Goor, Testing Semiconductor Memories: Theory and Practice, John Wiley & Sons, 1991.
[Guyot95] S. Guyot and H. Ural, "Synchronizable checking sequences based on UIO sequences," Proceedings of IFIP International Workshop on Protocol Test Systems, pp 385-397, 1995.
[HP97] Hewlett-Packard Co., Testing Memories with the HP83000, July 1997.
[Ignizio94] J. P. Ignizio and T. M. Cavalier, Linear Programming, Prentice Hall, 1994.
[ISO87] ISO TC97/SC21 "OSI conformance testing methodology and framework - part 1-5," ISO 2nd DP9646-1 revised text, Vancouver, December 1987.
[Kohavi78] Z. Kohavi, Switching and Finite Automata Theory, New York, McGraw-Hill, 1978
[Lin98] R. S. Lin and M. C. Yuang, "Synchronization Paradigm for Protocol Testing under Multi-Party Configuration," to appear on Computers & Mathematics with Applications.
[Lin98a] R. S. Lin, Conformance Testing and Fault Diagnosis for Communication Protocols, Ph.D. Dissertation, Institute of Computer Science and Information Engineering, National Chiao Tung University, 1998.
[Luo93] G. Luo, R. Dssouli, G. von Bochmann, P. Venkataram, and A. Ghedamsi, "Generating synchronizable test sequences based on finite state machine with distributed ports," Proceeding of 6th International IFIP Workshop on Protocol Test Systems, 1993.
[Luo94] G. Luo, G. von Bochmann, and A. Petrenko, "Test selection based on communicating nondeterministic finite-state machines using a generalized wp- method," IEEE Transaction on Software Engineering, Vol 20, No 2, pp 149-162, 1994.
[Miller91] R. E. Miller and S. Paul "Generating minimal length test sequences for conformance testing of communication protocols," Proceedings of IEEE INFOCOM, pp. 970-979, April 1991.
[Naito81] S. Naito and M. Tsunoyama "Fault detection for sequential machines by transition tours," Proceedings of 11th IEEE Fault Tolerant Computing Symposium, 1981.
[Nobert96] N. Nobert and J. C. Picard, "An optimal algorithm for the mixed Chinese postman problem," Networks, Vol. 27, pp. 95-108, 1996.
[Pearn95] W. L. Pearn and T. C. Wu, "Algorithms for the rural postman problem," Computers & Operations Research, Vol. 22, No. 8, pp. 819-828, 1995.
[Prince91] B. Prince, Semiconductor Memories, 2nd edition, John Wiley & Sons, 1991.
[Rayner87] D. Rayner, "OSI conformance testing," Computer Networks and ISDN Systems, Vol 14, pp 79-98, 1987.
[Sabnani88] K. K. Sabnani and A. T. Dahbura, "A protocol test generation procedure," Computer Networks and ISDN Systems, Vol. 15, No. 4, pp. 285-297, 1988.
[Sachdev95] M. Sachdev, "Reducing the cmos ram test complexity with Iddq and voltage testing," Journal of Electronic Testing: Theory and Applications, Vol. 6, pp. 191-202, 1995.
[Sarikaya84] B. Sarikaya and G. v. Bochmann, " Synchronizable and specification issues in protocol testing" IEEE Transaction on Communications, Vol. 32, No. 4, pp. 389-395, Apr. 1984.
[Schrijver86] A. Schrijver, Theory of Linear and Integer Programming, Wiley, 1986.
[Stevens86] A. K. Stevens, Introduction to Component Testing, Addison-Wesley, 1986.
[Tai95] K. C. Tai and Y. C. Young, "Port-synchronizable test sequences for communication protocols," Proceedings of IFIP International Workshop on Protocol Test Systems, pp 369-384, 1995.
[Tai98] K. C. Tai, and Y. C. Young, "Synchronizable Test Sequences of Finite State Machines," Computer Networks and ISDN Systems, Vol. 30, No. 12, 1998
[Tanenbaum88] A. S. Tanenbaum, Computer Networks (2nd Edition), Prentice-Hall, Englewood Cliffs, N. J., 1988.
[Teradyne96] Teradyne, Inc., Marlin-J996 Memory Test System Reference Manual, 1996.
[Ural92] H. Ural, "Formal methods for test sequence generation," Computer Communications, Vol 15, No 5, pp 311-325, 1992.
[Ural93] H. Ural and Z. Wang, "Synchronizable test sequence generation using UIO sequences," Computer Communications, Vol. 16, No.10, pp. 653-661, Oct. 1993
[Uyar87] M. U. Uyar and A. T. Dahbura, "Optimal test sequence generation for protocols: the Chinese Postman algorithm applied to Q.931," Proceeding of IEEE GLOBALCOM , 1987.
[Vuong89] S. T. Vuong, W. Y. L. Chan and M. R. Ito, "The UIOv-method for protocol test sequence generation," Proceeding of 2nd IFIP International Workshop on Protocol Test Systems, 1989.
[Wu96] W.-J. Wu, C. Y. Tang and M. Y. Lin, "Methods for memory test time reduction," Proceeding of IEEE International Workshop on Memory Technology, Design and Test, 1996.
[Wu98] W.-J. Wu, W.-H. Chen, C. Y. Tang, "Synchronizable test sequence for multi-party protocol conformance testing," Computer communications, Vol. 21, pp. 1177-1183, 1998.