The Application Of Machine Learning In Test Case Prioritization - A Review
##plugins.themes.bootstrap3.article.main##
Regression testing is very important but also a very costly and time-consuming activity that ensures the developers that changes in the application will not bring new errors. Retest all, selection of test cases and prioritization of test cases (TCP) approaches are used to enhance the efficiency and effectiveness in regression testing. While test case selection techniques decrease testing time and cost, it can exclude some critical test cases that can detect the faults. On the other hand, test case prioritization considers all test cases and execute them until resources are exhausted or all test cases are executed, while always focusing on the most important ones. Over the years, machine learning has found wide usage in solving different problems in software engineering. Software development and maintenance problems can be defined as learning problems and machine learning techniques have shown to be very effective in solving these problems. In the range of application of machine learning, machine learning techniques have also found usage in solving the test case prioritization problem. In this paper, we investigate the application of machine learning techniques in test case prioritization. We survey some of the most recent studies made in this field and provide information like techniques of machine learning used in TCP process, metrics used to measure the effectiveness of the proposed methods, data used to define the priority of test cases and some advantages or limitations of application of machine learning in TCP.
Downloads
References
-
D. Suleiman, M.Alian, A. Hudaib, ?A survey on prioritization regression testing test case?, presented at the 8th International Conference on Information Technology (ICIT), Amman, Jordan, May 17-18, 2017.
Google Scholar
1
-
P. K. Chittimalli, M. J. Harrold, ?Recomputing coverage information to assist regression testing?, IEEE Trans. Software Eng, vol. 35, pp. 452?469, Aug. 2009.
Google Scholar
2
-
E. Khanna, (November 2016). Regression testing based on genetic algorithm, International Journal of Computer Applications ? IJCA [Online]. 154(8). pp. 43-46. Available: https://www.ijcaonline.org/archives/volume154/number8/26515-2016912201
Google Scholar
3
-
M. Al-Refai, S.Ghosh, W. Cazzola, ?Model-based regression test selection for validating runtime adaptation of software systems?, presented at the 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST), Chicago, IL, USA, April 10-15, 2016.
Google Scholar
4
-
J. S. Rajal, S. Sharma, A review on various techniques for regression testing and test case prioritization, International Journal of Computer Applications (0975 ? 8887) [Online]. 116 (16). Available: https://pdfs.semanticscholar.org/1601/3e34d8b1ff09c2271195c14fdf31557cb2ec.pdf
Google Scholar
5
-
T. L. Graves, M. J. Harrold, J. Kim, A. Porter, G. Rothermel, ?An empirical study of regression test selection techniques?, ACM Transactions on Software Engineering and Methodology (TOSEM), vol. 10, pp. 184?208, April 2001.
Google Scholar
6
-
P. K., A. A. Rao, K. V. G. Rao, S. S. S. Reddy, M. Gopichand, ?Cost effective model based regression testing?, in Proceedings of the World Congress on Engineering 2017, 2017, vol. 1.
Google Scholar
7
-
P. K.Chittimalli and M. J. Harrold, ?Regression test selection on system requirements?, in ISEC ?08 Proceedings of the 1st India software engineering conference, pp. 87?96, India, 2008.
Google Scholar
8
-
G. Duggal and Mrs. B. Suri, ?Understanding regression testing techniques?, presented at the COIT, India, 2008.
Google Scholar
9
-
Varun Kumar, Sujata, and Mohit Kumar, Test case prioritization using fault severity, International Journal of Computer Science and Technology (IJCST) [Online]. 1(1). pp. 67?71.
Google Scholar
10
-
S. Yoo and M. Harman, Regression testing minimisation, selection and prioritisation: a survey, Journal of Software Testing Verification and Reliability [Online]. 22(2). pp. 67?120. Available: https://onlinelibrary.wiley.com/doi/10.1002/stvr.430
Google Scholar
11
-
C. Catal and D. Mishra, Test case prioritization: a systematic mapping study, Software Quality Journal [Online]. 21( 3). pp. 445?478. Available: http://www.ijirst.org/articles/IJIRSTV1I10014.pdf
Google Scholar
12
-
H. Do, S. Mirarab, L. Tahvildari, G. Rothermel, ?The effects of time constraints on test case prioritization: A series of controlled experiments?, IEEE Trans. Softw. Eng., vol. 36, pp. 593?617, Oct. 2010.
Google Scholar
13
-
G. Rothermel, R.H. Untch, C. Chu, M.J. Harrold, ?Prioritizing test cases for regression testing?, IEEE Trans. Software Eng, vol. 27, pp. 929?948, Oct. 2001.
Google Scholar
14
-
N. G?k?e, F. Belli, M. Eminli, B. T. Din?er, Model-based test case prioritization using cluster analysis-a aoft-computing approach, Turkish Journal of Electrical Engineering Computer Sciences [Online]. 23 (3), pp. 623 ? 640, Available: http://journals.tubitak.gov.tr/elektrik/issues/elk-15-23-3/elk-23-3-1-1209-109.pdf
Google Scholar
15
-
M. Khatibsyarbini, M. A. Isa, D. N.A. Jawawi, R. Tumeng, Test case prioritization approaches in regression testing: A systematic literature review, Journal of Information and Software. Technology [Online]. 93 (1), pp. 74?93, Available: https://www.sciencedirect.com/science/article/abs/pii/S0950584916304888
Google Scholar
16
-
B. Kitchenham,? Procedures for performing systematic reviews?, TR/SE-0401, 2005.
Google Scholar
17
-
B. Kitchenham, O. P. Brereton, D. Budgen, M. Turner, J. Bailey, S. Linkman, Systematic literature reviews in software engineering - A systematic literature review, Journal of Information and Software. Technology [Online]. 51(1), pp. 7?15, Available: https://www.sciencedirect.com/science/article/abs/pii/S0950584908001390
Google Scholar
18
-
N. G?k?e, M. Eminov, F. Belli, ?Coverage-based, prioritized testing using neural network clustering?, presented at the Computer and Information Sciences ? ISCIS, Istanbul, Turkey, November 1-3, 2006.
Google Scholar
19
-
B. Korel, G. Koutsogiannakis, Experimental comparison of code-based and model-based test prioritization, in ICSTW ?09 Proceedings of the IEEE International Conference on Software Testing, Verification, and Validation Workshops, pp. 77?84, Washington, DC, USA, 2009.
Google Scholar
20
-
F. Belli, ?Finite-state testing and analysis of graphical user interfaces", in Proceedings 12th International Symposium on Software Reliability Engineering, Hong Kong, China, 2001.
Google Scholar
21
-
N. G?k?e, M. Eminli, Model-based test case prioritization using neural network classification, Computer Science & Engineering: An International Journal (CSEIJ) [Online]. 4( 1), pp. 15?25. Available: https://www.academia.edu/12477424/MODEL-BASED_TEST_CASE_PRIORITIZATION_USING_NEURAL_NETWORK_CLASSIFICATION
Google Scholar
22
-
H. Spieker, A. Gotlieb, D. Marijan, D. Marijan, ?Reinforcement learning for automatic test case prioritization and selection in continuous integration?, in ISSTA 2017 Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 12?22, Santa Barbara, CA, USA, 2017,
Google Scholar
23
-
S. Abele, P. G?hner, ?Improving proceeding test case prioritization with learning software agents?, in ICAART 2014 - Proceedings of the 6th International Conference on Agents and Artificial Intelligence, pp. 293?298, vol. 2, France, 2014,
Google Scholar
24
-
P. Konsaard, L. Ramingwong, ?Total coverage based regression test case prioritization using genetic algorithm?, presented in 2015 12th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), Hua Hin, Thailand, June 24-27, 2015.
Google Scholar
25
-
A. A. Ahmed, M. Shaheen, E. Kosba, ?Software testing suite prioritization using multicriteria fitness function?, in 2012 22nd International Conference on Computer Theory and Applications (ICCTA), Alexandria, Egypt, October 13-15, 2012.
Google Scholar
26
-
S. Mirarab, L. Tahvildari,? A prioritization approach for software test cases based on Bayesian networks?, in FASE?07 Proceedings of the 10th international conference on Fundamental approaches to software engineering, pp. 276?290, Braga, Portugal, 2007.
Google Scholar
27
-
S. Mirarab and L. Tahvildari, An empirical study on bayesian network-based approach for test case prioritization, presented in 2008 1st International Conference on Software Testing, Verification, and Validation, Lillehammer, Norway, April 9-11, May 2008.
Google Scholar
28
-
X. Zhao, Z. Wang, X. Fan, Z. Wang, ?A clustering ? bayesian network-based approach for test case prioritization?, presented in 2015 IEEE 39th Annual Computer Software and Applications Conference,Taichung, Taiwan, July 1-5, 2015.
Google Scholar
29
-
P. Tonella, P. Avesani, A. Susi, ?Using the case-cased ranking methodology for test case prioritization?, presented at the 22nd IEEE International Conference on Software Maintenance, Philadelphia, Pennsylvania, 2006.
Google Scholar
30
-
R. Lachmann, S. Schulze, M. Nieke, C. Seidl, I.Schaefer, ?System level test case prioritization using machine learning?, presented in 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), California, USA, Dec. 18-20, 2016.
Google Scholar
31
-
T. Joachims, Optimizing search engines using clickthrough data, in KDD ?02, 2002, pp. 133?142.
Google Scholar
32
-
R. Lachmann, ?Machine learning-driven test case prioritization approaches for black-box software testing?, presented at the The European Test and Telemetry Conference, Nuremberg, Germany, June 26-28, 2018.
Google Scholar
33
-
B.Busjaeger, T. Xie,? Learning for test prioritization: an industrial case study?, in FSE 2016 Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 975?980, Seattle, WA, USA, 2016.
Google Scholar
34
-
J. Chen et al., Test case prioritization for object-oriented software: An adaptive random sequence approach based on clustering, Journal of Systems and Software, vol. 135, 107?125, Jan. 2018.
Google Scholar
35
-
R. Mukherjee, K. Sridhar Patnaik, A survey on different approaches for software test case prioritization, Journal of King Saud University - Computer and Information Sciences [Online], Available: https://www.sciencedirect.com/science/article/pii/S1319157818303616.
Google Scholar
36