本研究以 OWASP Benchmark 比較 Web 應用程式漏洞掃描器 OWASP
ZAP 的兩個最新版本,評估其在 OWASP Benchmark 測試下發現漏洞的效率。
研究方法使用 OWASP ZAP v2.12.0 版和 v2.13.0 版對 OWASP Bench-
mark 進行系統掃描。 OWASP Benchmark 是一個具備標準化架構並提供安全
漏洞測驗資料的網站,可評估漏洞掃描器在識別安全缺陷、不安全的 Cookie、
目錄遍歷攻擊、 SQL 注入攻擊等方面的安全防範能力。從 OWASP Bench-
mark比較中獲得的結果能為我們揭示 OWASP ZAP 各個版本各自的優點和缺點。
研究成果展示漏洞掃描程式發現安全漏洞的機制,能幫助應用程式安全測
試系統的發展。本研究結果可幫助開發人員和IT安全團隊做出補強系統安全漏
洞的決策,使其Web應用程式安全能力提升。
總結而言,本研究使用 OWASP Benchmark v1.2 全面分析、測試 ZAP
偵測安全漏洞的能力。提供安全與開發人員對漏洞測試系統的更多了解,並為
未來 web 應用程式的安全漏洞測試相關研究的進一步探討和發展奠定基礎。

The Huge growth in the usage of web applications has raised concerns regarding their security vulnerabilities, which in turn pushes toward robust security testing tools. This study compares OWASP ZAP, the leading open-source web application vulnerability scanner, across its two most recent iterations. While comparing their performance to the OWASP Benchmark, the study evaluates their efficiency in spotting vulnerabilities in the purposefully vulnerable application,
OWASP Benchmark project.

The research methodology involves conducting systematic scans of OWASP Benchmark using both v2.12.0 and v2.13.0 of OWASP ZAP. The OWASP Benchmark provides a standardized framework to evaluate the scanner’s abilities in identifying security flaws, Insecure Cookies, Path traversal, SQL injection, and more. Results obtained from this benchmark comparison offer valuable insights into the strengths and weaknesses of each version of the tool. This study aids in web application security testing by shedding light on how well-known scanners work at spotting vulnerabilities. The knowledge gained from this study can assist security professionals and developers in making informed decisions to support their web application security status. In conclusion, this study comprehensively analyzes ZAP’s capabilities in detecting security flaws using OWASP Benchmark v1.2. The findings add to the continuing debates about online application security tools and
establish the framework for future studies and developments in the research field of web application security testing.

[1] OWASP Benchmark owasp benchmark project. https://owasp.org/
www-project-benchmark/. Accessed: 2023-09-27.
[2] Himli S Abdullah. Evaluation of open source web application vulnerability
scanners. Academic Journal of Nawroz University, 9(1):47–52, 2020.
[3] Azaz Ahamed, Nafiz Sadman, Touseef Aziz Khan, Mahfuz Ibne Hannan,
Farzana Sadia, and Mahady Hasan. Automated testing: Testing top 10 owasp
vulnerabilities of government web applications in bangladesh. ICSEA 2022,
page 56, 2022.

[4] Azwar Al Anhar and Yohan Suryanto. Evaluation of web application vulner-
ability scanner for modern web application. In 2021 International Conference

on Artificial Intelligence and Computer Science Technology (ICAICST), pages
200–204. IEEE, 2021.
[5] Alde Alanda, Deni Satria, M Isthofa Ardhana, Andi Ahmad Dahlan, and
Hanriyawan Adnan Mooduto. Web application penetration testing using sql
injection attack. JOIV: International Journal on Informatics Visualization,
5(3):320–326, 2021.

[6] Marwan Albahar, Dhoha Alansari, and Anca Jurcut. An empirical compar-
ison of pen-testing tools for detecting web app vulnerabilities. Electronics,

11(19):2991, 2022.

56

[7] Abdulwahed Awad Almutairi, Shailendra Mishra, and Mohammed AlShehri.
Web security: Emerging threats and defense. Computer Systems Science &
Engineering, 40(3), 2022.

[8] Wael Alsabbagh, Samuel Amogbonjaye, Diego Urrego, and Peter Lan-
gend ̈orfer. A stealthy false command injection attack on modbus based scada

systems. In 2023 IEEE 20th Consumer Communications & Networking Con-
ference (CCNC), pages 1–9. IEEE, 2023.

[9] Richard Amankwah, Jinfu Chen, Patrick Kwaku Kudjo, and Dave Towey.
An empirical comparison of commercial and open-source web vulnerability
scanners. Software: Practice and Experience, 50(9):1842–1857, 2020.
[10] Chris Anley. Advanced sql injection in sql server applications. 2002.
[11] Yuanyuan Bai and Zhi Chen. Analysis and exploit of directory traversal

vulnerability on vmware. In Applications and Techniques in Information Se-
curity: 6th International Conference, ATIS 2015, Beijing, China, November

4-6, 2015, Proceedings 6, pages 238–244. Springer, 2015.
[12] Neelima Bayyapu. Sql injection attacks and mitigation strategies: The latest
comprehension. In Advances in Cybersecurity Management, pages 199–220.
Springer, 2021.
[13] Geogiana Buja, Kamarularifin Bin Abd Jalil, Fakariah Bt Hj Mohd Ali, and
Teh Faradilla Abdul Rahman. Detection model for sql injection attack: An
approach for preventing a web application from the sql injection attack. In
2014 IEEE Symposium on Computer Applications and Industrial Electronics
(ISCAIE), pages 60–64. IEEE, 2014.

57

[14] Ming-Syan Chen, Jong Soo Park, and Philip S. Yu. Efficient data mining for

path traversal patterns. IEEE Transactions on knowledge and data engineer-
ing, 10(2):209–221, 1998.

[15] Aryan Chouhan, Aayush Halgekar, Ashish Rao, Dhruvi Khankhoje, and

Meera Narvekar. Sentiment analysis of twitch. tv livestream messages us-
ing machine learning methods. In 2021 fourth international conference on

electrical, computer and communication technologies (ICECCT), pages 1–5.
IEEE, 2021.
[16] Justin Clarke-Salt. SQL injection attacks and defense. Elsevier, 2009.

[17] Ivan Cviti ́c, Dragan Perakovi ́c, Marko Periˇsa, and Dominik Sever. Defin-
ing cross-site scripting attack resilience guidelines based on beef framework

simulation. Mobile Networks and Applications, pages 1–13, 2022.
[18] Irfan Darmawan, Aditya Pratama Abdul Karim, Alam Rahmatulloh, Rohmat
Gunawan, and Dita Pramesti. Json web token penetration testing on cookie
storage with csrf techniques. In 2021 International Conference Advancement
in Data Science, E-learning and Information Systems (ICADEIS), pages 1–5.
IEEE, 2021.
[19] KL Dasun. A Study on effectiveness of software vulnerability assessment for
component-based software development. PhD thesis, 2016.
[20] Nor Izyani Daud, Khairul Azmi Abu Bakar, and Mohd Shafeq Md Hasan. A
case study on web application vulnerability scanning tools. In 2014 Science
and Information Conference, pages 595–600. IEEE, 2014.
[21] Lyubka Dencheva. Comparative analysis of Static application security testing

(SAST) and Dynamic application security testing (DAST) by using open-
58

source web application penetration testing tools. PhD thesis, Dublin, National
College of Ireland, 2022.

[22] Giuseppe A Di Lucca, Anna Rita Fasolino, M Mastoianni, and Porfirio Tra-
montana. Identifying cross site scripting vulnerabilities in web applications.

In Proceedings. Sixth IEEE International Workshop on Web Site Evolution,
pages 71–80. IEEE, 2004.
[23] Vincenzo Di Stasio. Evaluation of Static Security Analysis Tools on Open
Source Distributed Applications. PhD thesis, Politecnico di Torino, 2022.

[24] Manuel Egele, Peter Wurzinger, Christopher Kruegel, and Engin Kirda. De-
fending browsers against drive-by downloads: Mitigating heap-spraying code

injection attacks. In Detection of Intrusions and Malware, and Vulnerability
Assessment: 6th International Conference, DIMVA 2009, Como, Italy, July
9-10, 2009. Proceedings 6, pages 88–106. Springer, 2009.
[25] Malaka El, Emma McMahon, Sagar Samtani, Mark Patton, and Hsinchun

Chen. Benchmarking vulnerability scanners: An experiment on scada de-
vices and scientific instruments. In 2017 IEEE International Conference on

Intelligence and Security Informatics (ISI), pages 83–88. IEEE, 2017.
[26] Aur ́elien Francillon and Claude Castelluccia. Code injection attacks on
harvard-architecture devices. In Proceedings of the 15th ACM conference on
Computer and communications security, pages 15–26, 2008.
[27] Jeremiah Grossman. XSS attacks: cross site scripting exploits and defense.
Syngress, 2007.
[28] Shashank Gupta and Brij Bhooshan Gupta. Cross-site scripting (xss) attacks
and defense mechanisms: classification and state-of-the-art. International
Journal of System Assurance Engineering and Management, 8:512–530, 2017.

59

[29] William G Halfond, Jeremy Viegas, Alessandro Orso, et al. A classification
of sql-injection attacks and countermeasures. In Proceedings of the IEEE
international symposium on secure software engineering, volume 1, pages 13–
15. IEEE, 2006.
[30] Juan R Bermejo Higuera, Javier Bermejo Higuera, Juan A Sicilia Montalvo,
Javier Cubo Villalba, and Juan Jos ́e Nombela P ́erez. Benchmarking approach
to compare web applications static analysis tools detecting owasp top ten
security vulnerabilities. Computers, Materials & Continua, 64(3), 2020.
[31] Wei Hu, Jason Hiser, Dan Williams, Adrian Filipi, Jack W Davidson, David
Evans, John C Knight, Anh Nguyen-Tuong, and Jonathan Rowanhill. Secure
and practical defense against code-injection attacks using software dynamic
translation. In Proceedings of the 2nd international conference on Virtual
execution environments, pages 2–12, 2006.

[32] Isatou Hydara, Abu Bakar Md Sultan, Hazura Zulzalil, and Novia Admod-
isastro. Current state of research on cross-site scripting (xss)–a systematic

literature review. Information and Software Technology, 58:170–186, 2015.

[33] Xing Jin, Xunchao Hu, Kailiang Ying, Wenliang Du, Heng Yin, and Gau-
tam Nagesh Peri. Code injection attacks on html5-based mobile apps: Charac-
terization, detection and mitigation. In Proceedings of the 2014 ACM SIGSAC
Conference on Computer and Communications Security, pages 66–77, 2014.

[34] GV Jordan. Command injections. School of Information Tech. and Engineer-
ing University of Ottawa, Ottawa, 2009.

[35] Christopher Kalaani. Owasp zap vs snort for sqli vulnerability scanning. 2023.
[36] Hyunsoo Kwon, Hyunjae Nam, Sangtae Lee, Changhee Hahn, and Junbeom
Hur. (in-) security of cookies in https: Cookie theft by removing cookie

60

flags. IEEE Transactions on Information Forensics and Security, 15:1204–
1215, 2019.

[37] Emma Lavens, Pieter Philippaerts, and Wouter Joosen. A quantitative as-
sessment of the detection performance of web vulnerability scanners. In Pro-
ceedings of the 17th International Conference on Availability, Reliability and

Security, pages 1–10, 2022.
[38] Yuma Makino and Vitaly Klyuev. Evaluation of web vulnerability scanners.
In 2015 IEEE 8th International Conference on Intelligent Data Acquisition
and Advanced Computing Systems: Technology and Applications (IDAACS),
volume 1, pages 399–402. IEEE, 2015.
[39] Balume Mburano and Weisheng Si. Evaluation of web vulnerability scanners
based on owasp benchmark. In 2018 26th International Conference on Systems
Engineering (ICSEng), pages 1–6. IEEE, 2018.

[40] Reza M Parizi, Kai Qian, Hossain Shahriar, Fan Wu, and Lixin Tao. Bench-
mark requirements for assessing software security vulnerability testing tools.

In 2018 IEEE 42nd Annual Computer Software and Applications Conference
(COMPSAC), volume 1, pages 825–826. IEEE, 2018.
[41] Joon S Park and Ravi Sandhu. Secure cookies on the web. IEEE internet
computing, 4(4):36–44, 2000.

[42] Germ ́an E Rodr ́ıguez, Jenny G Torres, Pamela Flores, and Diego E Bena-
vides. Cross-site scripting (xss) attacks and mitigation: A survey. Computer

Networks, 166:106960, 2020.

[43] Marcus D Ruopp, Neil J Perkins, Brian W Whitcomb, and Enrique F Schis-
terman. Youden index and optimal cut-point estimated from observations

61

affected by a lower limit of detection. Biometrical Journal: Journal of Math-
ematical Methods in Biosciences, 50(3):419–430, 2008.

[44] Suphannee Sivakorn, Iasonas Polakis, and Angelos D Keromytis. The cracked
cookie jar: Http cookie hijacking and the exposure of private information. In
2016 IEEE Symposium on Security and Privacy (SP), pages 724–742. IEEE,
2016.

[45] Kevin Z Snow, Fabian Monrose, Lucas Davi, Alexandra Dmitrienko, Christo-
pher Liebchen, and Ahmad-Reza Sadeghi. Just-in-time code reuse: On the

effectiveness of fine-grained address space layout randomization. In 2013 IEEE
symposium on security and privacy, pages 574–588. IEEE, 2013.
[46] Marina Sokolova, Nathalie Japkowicz, and Stan Szpakowicz. Beyond accuracy,
f-score and roc: a family of discriminant measures for performance evaluation.
In Australasian joint conference on artificial intelligence, pages 1015–1021.
Springer, 2006.
[47] Marco Squarcina, Pedro Ad ̃ao, Lorenzo Veronese, and Matteo Maffei. Cookie
crumbles: Breaking and fixing web session integrity. In 32nd USENIX Security
Symposium (USENIX Security 23), pages 5539–5556, 2023.
[48] Anastasios Stasinopoulos, Christoforos Ntantogian, and Christos Xenakis.

Commix: automating evaluation and exploitation of command injection vul-
nerabilities in web applications. International Journal of Information Secu-
rity, 18:49–72, 2019.

[49] Ankit Thakkar and Ritika Lohiya. A survey on intrusion detection sys-
tem: feature selection, model, performance measures, application perspec-
tive, challenges, and future research directions. Artificial Intelligence Review,

55(1):453–563, 2022.

62

[50] Solomon Ogbomon Uwagbole, William J Buchanan, and Lu Fan. Applied
machine learning predictive analytics to sql injection attack detection and

prevention. In 2017 IFIP/IEEE Symposium on Integrated Network and Ser-
vice Management (IM), pages 1087–1090. IEEE, 2017.

[51] Alice Van Rensburg. Vulnerability testing in the web application development
cycle. University of Johannesburg (South Africa), 2017.
[52] Z Vujovi ́c et al. Classification model evaluation metrics. ˇ International Journal
of Advanced Computer Science and Applications, 12(6):599–606, 2021.
[53] Xiaofeng Zheng, Jian Jiang, Jinjin Liang, Haixin Duan, Shuo Chen, Tao Wan,
and Nicholas Weaver. Cookies lack integrity:{Real-World} implications. In
24th USENIX Security Symposium (USENIX Security 15), pages 707–721,
2015.

[54] Lin Zhou, Ying Liu, Jing Wang, and Yong Shi. Utility-based web path traver-
sal pattern mining. In Seventh IEEE International Conference on Data Min-
ing Workshops (ICDMW 2007), pages 373–380. IEEE, 2007.