Mutation Analysis (MA) is a fault-based simulation technique that is used to measure
the quality of testbenches in error (mutant) detection. Although MA effectively
reports the living mutants to designers, it suffers from the high simulation
and analysis cost. This paper presents a probabilistic MA preprocessing technique,
Error Propagation Analysis (EPA), to speed up the MA process. EPA can
statically estimate the probability of the error propagation with respect to each mutant
for guiding the observation-point insertion. The inserted observation-points
will reveal a mutant’s status earlier during the simulation such that some useless
testcases can be discarded later. Additionally, EPA can generate a set of
unsensitized-targets for each living mutant to help designers improve their testbench
quality. We use the mutant model from an industrial EDA tool, Certitude,
to conduct our experiments on the OpenCores’ RT-level designs. The experimental
results show that the EPA approach can save about 14% CPU time while
obtaining the same mutant status report as the traditional MA approach.

在功能驗證各種方法中，突變式分析法 (Mutation Analysis) 屬於
一種基於錯誤模擬法，其利用錯誤偵測的方式衡量測試平台的品質。
雖然突變式分析法可以指出電路中的潛在錯誤 (Living Mutant)，但其
所消耗的模擬和分析時間非常大。這篇論文提出了一種機率式的突變
式分析法前置作業，可用來加快突變式分析法的整體執行時間，此方
法稱作「錯誤傳遞分析法」(Error Propagation Analysis)。錯誤傳遞分
析法使用靜態的機率運算，估計每一個在電路中的潛在錯誤傳遞的機
率，再利用其結果作為加入觀察點位置的參考。在電路中的觀察點可
以在電路模擬中提早判斷出錯誤的狀態，若有測試資料在電路模擬時
無法在觀察點中產生錯誤的資訊，這些測試資料即可在執行突變式分
析法前先移除。除此之外，錯誤傳遞分析法可以針對每一個潛在錯誤，
自動產生對應的「未觸發目標」作為潛在錯誤可能發生的來源，以幫
助電路設計者改善測試平台的品質。我們這裡使用產業界的電腦輔助
設計工具─Certitude，以及OpenCore 網站上所公布的測試電路來輔
助我們的實驗。實驗結果顯示出在相同的錯誤模型下，使用錯誤傳遞
分析法作為突變式分析法的前置作業，能有效地減少約14%的執行時
間。

[1] Icarus Verilog. [Online]. Available: http://iverilog.icarus.com/
[2] OpenCores. [Online]. Available: http://opencores.org/
[3] SpringSoft. [Online]. Available: http://www.springsoft.com/
[4] S. Anellal and B. Kaminska, “Scheduling of a control and data flow graph,” in Proc. IEEE
Int. Symp. on Circuits and Systems, May 1993, pp. 1666 – 1669.
[5] J. Arlat, A. Costes, Y. Crouzet, J. Laprie, and D. Powell, “Fault injection and dependability
evaluation of fault-tolerant systems,” IEEE Trans. Computers, vol. 42, no. 8, pp. 913 – 923,
Aug. 1993.
[6] A. Benso, A. Bosio, S. Di Carlo, and R. Mariani, “A functional verification based fault
injection environment,” in Proc. Defect and Fault-Tolerance in VLSI Systems, 2007, pp. 114
– 122.
[7] A. Benso, M. Rebaudengo, M. Reorda, and P. Civera, “An integrated HW and SW fault
injection environment for real-time systems,” in Proc. Defect and Fault Tolerance in VLSI
Systems, 1998, pp. 117 – 122.
[8] R. DeMillo, R. Lipton, and F. Sayward, “Hints on test data selection: Help for the practicing
programmer,” IEEE Trans. Computers, vol. 11, no. 4, pp. 34 – 41, Apr. 1978.
[9] L. Drucker, “Functional coverage metrics–the next frontier,” in EETimes, Aug. 2002.
[10] F. Ferrandi, F. Fummi, and D. Sciuto, “Implicit test generation for behavioral VHDL models,”
in Proc. Int. Test Conference, 1998, pp. 587 – 596.
[11] A. Fin and F. Fummi, “A VHDL error simulator for functional test generation,” in Proc.
Design, Automation and Test in Europe, 2000, pp. 390 – 395.
[12] A. Gluska, “Coverage-oriented verification of banias,” in Proc. Design Automation Conference,
2003, pp. 280 – 285.
[13] R. Hamlet, “Testing programs with the aid of a compiler,” IEEE Trans. Software Engineering,
vol. SE-3, no. 4, pp. 279 – 290, July 1977.
[14] M. Hampton and S. Petithomme, “Leveraging a commercial mutation analysis tool for research,”
in Proc. Testing: Academic and Industrial Conference Practice and Research Techniques
- Mutation, 2007, pp. 203 – 209.
[15] W. Howden, “Weak mutation testing and completeness of test sets,” IEEE Trans. Software
Engineering, vol. SE-8, no. 4, pp. 371 – 379, July 1982.
[16] Y. Jia and M. Harman, “An analysis and survey of the development of mutation testing,”
IEEE Transactions on Software Engineering, vol. 35, no. 6, pp. 1 – 32, 2010.
[17] T. Lv, H. wei Li, and X. wei Li, “Automatic selection of internal observation signals for
design verification,” in Proc. VLSI Test Symposium, 2009, pp. 203 –208.
[18] J. C. Miller and C. J. Maloney, “Systematic mistake analysis of digital computer programs,”
Commun. ACM, vol. 6, pp. 58 – 63, Feb. 1963.
[19] R. Namballa, N. Ranganathan, and A. Ejnioui, “Control and data flow graph extraction for
high-level synthesis,” in Proc. IEEE Annual Symposium on VLSI, Feb. 2004, pp. 187 – 192.
[20] A. Offutt, “The coupling effect: fact or fiction,” in Proc. Symp. Software Testing, Analysis,
and Verification, 1989, pp. 131 – 140.
[21] A. J. Offutt and R. H. Untch, “Mutation 2000: Uniting the orthogonal,” in Proc. Mutation,
2000, pp. 45 – 55.
[22] A. J. Offutt, “Investigations of the software testing coupling effect,” ACM Trans. Software
Engineering and Methodology, vol. 1, pp. 5 – 20, Jan. 1992.
[23] S. Tasiran and K. Keutzer, “Coverage metrics for functional validation of hardware designs,”
IEEE Design and Test of Computers, vol. 18, no. 4, pp. 36 – 45, Jul./Aug. 2001.
[24] M. Woodward and K. Halewood, “From weak to strong, dead or alive? an analysis of some
mutation testing issues,” in Proc. Workshop Software Testing, Verification, and Analysis,
1988, pp. 152 –158.
[25] H. H. Yeh and C. Y. Huang, “Automatic constraint generation for guided random simulation,”
in Proc. Asia and South Pacific Design Automation Conference, 2010, pp. 613 – 618.