COMPARATIVE BENCHMARKING OF CONSTRAINTS T-WAY TEST GENERATION STRATEGY BASED ON LATE ACCEPTANCE HILL CLIMBING ALGORITHM

Kamal Z. Zamli, Abdul Rahman Alsewari, Basem Al-Kazemi

Abstract


This paper describes  the new t-way strategy based the Late Acceptance based Hill Climbing algorithm,  called  LAHC,  for  constraints  t-way  test generation.  Unlike  earlier  competing work, LAHC does not require significant tuning in order to have it working. In fact, LAHC merely requires minor adjustment of the common controlling parameters involving iteration and population size depending on the given system configuration.  Our benchmarking results have been promising as LAHC gives competitive results in most constraints configurations considered.

Keywords:  Optimization Algorithms, Software Testing, Artificial Intelligent


Full Text:

[PDF]

References


Afzal, W., R. Torkar, et al. (2009). "A Systematic Review of Search-Based Testing for Non-Functional System Properties." Information and Software Technology 51 (6): 957-976.

Ahmed, B. S. and K. Z. Zamli (2010). PSTG: A T-Way Strategy Adopting Particle Swarm Optimization. Proceedings of the 4th Asia International Conference on Mathematical /Analytical Modelling and Computer Simulation, IEEE Computer Society.

Ahmed, B. S. and K. Z. Zamli (2010). T-Way Test Data Generation Strategy Based on Particle Swarm Optimization. Proceedings of the 2nd International Conference on Computer Research and Development, IEEE Computer Society.

Ahmed, B. S. and K. Z. Zamli (2011). "The Development of a Particle Swarm Based Optimization Strategy for Pairwise Testing." Journal of Artificial Intelligence 4 (2): 156-165.

Ahmed, B. S., K. Z. Zamli, et al. (2012). "Constructing a T-Way Interaction Test Suite Using the Particle Swarm Optimization Approach." International Journal of Innovative Computing, Information and Control 8 (1): 431-452.

Alsewari, A. R. A. and K. Z. Zamli (2012). "Design and Implementation of a Harmony-Search-based Variable-Strength t-way Testing Strategy with Constraints Support." Information and Software Technology 54 : 553-568.

Arshem, J. (2010). "TVG." Retrieved 16 June, 2010, from http://sourceforge.net/projects/tvg.

Bryce, R. and C. Colbourn (2007). One-Test-at-a-Time Heuristic Search for Interaction Test Suites. Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, London, England.

Chen, X., Q. Gu, et al. (2009). Variable Strength Interaction Testing with an Ant Colony System Approach. Proceedings of the 16th Asia-Pacific Software Engineering Conference, IEEE Computer Society.

Cohen, D. M., S. R. Dalal, et al. (1997). "The AETG System: An Approach to Testing Based on Combinatorial Design." IEEE Transactions on Software Engineering 23 (7): 437-444.

Cohen, D. M., S. R. Dalal, et al. (1996). "The Combinatorial Design Approach to Automatic Test Generation." IEEE software 13 (5): 83-88.

Cohen, M. B. (2004). Designing Test Suites For Software Interaction Testing. PhD, University of Auckland.

Cohen, M. B. (2004). Designing Test Suites for Software Interaction Testing. Ph.D, University of Auckland.

Cohen, M. B., C. J. Colbourn, et al. (2008). "Constructing Strength Three Covering Arrays with Augmented Annealing." Discrete Mathematics 308 (13): 2709-2722.

Cohen, M. B., M. B. Dwyer, et al. (2007). Interaction Testing of Highly-Configurable Systems in the Presence of Constraints. Proceeding of International Symposium on Software Testing and Analysis, London, UK, ACM.

Daich, G. T. (2003). Testing Combinations of Parameters Made Easy. Proceedings of the IEEE Systems Readiness Technology Conference (AUTOTESTCON 2003), CA, USA, IEEE Computer Society.

Garvin, B. J., M. B. Cohen, et al. (2011). "Evaluating Improvements to a Meta-Heuristic Search for Constrained Interaction Testing." Empirical Software Engineering(16): 61-102.

Grindal, M., J. Offutt, et al. (2005). "Combination Testing Strategies: A Survey." Journal of Software Testing, Verification and Reliability 15 (3): 167-199.

Harman, M. and B. F. Jones (2001). "Search-Based Software Engineering." Information and Software Technology 43v(14): 833-839.

Hartman, A. and L. Raskin (2004). "Problems and Algorithms for Covering Arrays." Discrete Mathematics 284 (1-3): 149-156.

Hedayat, A. S., N. J. A. Sloane, et al. (1999). Orthogonal Arrays: Theory and Applications. New York, Springer Verlag.

Klaib, M. F. J. (2009). Development Of An Automated Test Data Generation And Execution Strategy Using Combinatorial Approach. PhD, Universiti Sains Malaysia.

Lei, Y., R. Kacker, et al. (2007). IPOG: A General Strategy for T-Way Software Testing. Proceedings of the 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems, Tucson, AZ U.S.A, IEEE Computer Society.

Mandl, R. (1985). "Orthogonal Latin Squares: An Application of Experiment Design to Compiler Testing." Communications of the ACM 28 (10): 1054-1058.

McCaffrey, J. (2010). An Empirical Study of Pairwise Test Set Generation Using a Genetic Algorithm. Proceedings of the 7th International Conference on Information Technology, IEEE Computer Society.

Nie, C. and H. Leung (2011). "A Survey of Combinatorial Testing." ACM Computing Surveys 43 (2).

Othman, R. R. and K. Z. Zamli (2011). "ITTDG: Integrated T-way Test Data Generation Strategy for Interaction Testing." Scientific Research and Essays 6 (17): 3638-3648.

Pallas, D. (2003). "Jenny." Retrieved 16 June 2010, 2010, from http://www.burtleburtle.net/bob/math.

R.R. Othman, N. Khamis, et al. "Variable Strength T-Way Test Suite Generator with Constraints Support." Malaysia Journal of Computer Science 27 (3): 204-217.

Sherwood., G. (2006). "TestCover." Retrieved 15 June, 2011, from http://testcover.com/pub/constex.php.

Shiba, T., T. Tsuchiya, et al. (2004). Using Artificial Life Techniques to Generate Test Cases for Combinatorial Testing. Proceedings of the 28th Annual International Computer Software and Applications Conference, IEEE Computer Society.

Stardom, J. (2001). Metaheuristics and The Search for Covering and Packing Array Master of Scienc Master thesis, Simon Fraser University.

Sthamer, H. (1995). The Automatic Generation of Software Test Data Using Genetic Algorithms. PhD thesis, Universityof Glamorgan,Pontyprid, Wales.

Tung, Y. W. and W. S. Aldiwan (2000). Automatic Test Case Generation For The New Generation Mission Software System. Proceedings of IEEE Aerospace Conference, Big Sky, MT, USA.

Wang, Z. Y., B. W. Xu, et al. (2008). Greedy Heuristic Algorithms to Generate Variable Strength Combinatorial Test Suite. Proceedings of the 8th International Conference on Quality Software, IEEE Computer Society.

Williams, A. W. (2000). Determination of Test Configurations for Pair-wise Interaction Coverage. Proceedings of the 13th International Conference on Testing of Communicating System, Ottawa, Canada.

Williams, A. W. (2002). "TConfigure" Retrieved 16 June 2010, 2010, from http://www.site.uottawa.ca/~awilliam.

Williams, A. W. (2010). "TConfigure" Retrieved February, 2012, from http://www.site.uottawa.ca/~awilliam/.

Yan, J. and J. Zhang (2006). Backtracking Algorithms And Search Heuristics To Generate Test Suites For Combinatorial Testing. Proceeding of the 30th Annual International Computer Software and Applications Conference, IEEE Computer Society.

Younis, M. I. (2010). MIPOG: A Parallel T-Way Minimization Strategy For Combinatorial Testing. PhD, Universiti Sains Malaysia.

Younis, M. I. and K. Z. Zamli (2010). "MC-MIPOG: A Parallel T-Way Test Generation Strategy for Multicore Systems." ETRI Journal 32 (1): 73-83.

Younis, M. I., K. Z. Zamli, et al. (2008). IRPS --- An Efficient Test Data Generation Strategy for Pairwise Testing. Proceedings of the 12th international conference on Knowledge-Based Intelligent Information and Engineering Systems, Part I, Zagreb, Croatia, Springer-Verlag.

Younis, M. I., K. Z. Zamli, et al. (2008). MIPOG - Modification Of The IPOG Strategy For T-Way Software Testing. Proceeding of The Distributed Frameworks and Applications (DFmA), Penang, Malaysia.

Younis, M. I., K. Z. Zamli, et al. (2008). A Strategy For Grid Based T-Way Test Data Generation. Proceedings the 1st IEEE International Conference on Distributed Frameworks and Application (DFmA '08), Penang, Malaysia.

Yu-Wen, T. and W. S. Aldiwan (2000). Automating Test Case Generation for the New Generation Mission Software System. Proceedings of the IEEE Aerospace Conference, Big Sky, MT, USA, IEEE Computer Society.

Zamli, K. Z., M. F. J. Klaib, et al. (2011). "Design And Implementation Of A T-Way Test Data Generation Strategy With Automated Execution Tool Support." Information Sciences 181(9): 1741-1758


Refbacks

  • There are currently no refbacks.