Why is the testing stage of systems development so important? November 2012 Introduction • Systems Development Lifecycle • Types of SDLC • Stages of SDLC • • • • • • • • • • Waterfall method Testing Programme Testing Acceptance Testing Limitations of Testing Case Study 1: University of Salford Case Study 2: Knight Capital Group Summary Conclusion References Systems Development Lifecycle • Systems Development Life Cycle (SDLC) is the process of developing information systems through various stages from identifying initial requirements through analysis, design, testing, mplementation and maintenance. • Many different models but each generally consists of a series of defined steps or stages. Laudon & Laudon (2006) Types of SDLC • Waterfall • Fountain • Spiral • Rapid prototyping • Incremental Bocij et al (2006) Waterfall • One of the oldest, best known • Linear sequential set of phases • Review at end of each phase before moving on to the next • Logical chain of events, e. g. programming cannot begin until specification completed • Many different versions • Many critics – does not work? Laplante & Neill (2004) Dennis et al (2008) Stages of SDLC
Vary according to the methodology used but usually some variation of the waterfall method which has Galin (2004) served as a framework for others. • Systems analysis – identify problems & establish requirements • Systems Design – create design specifications • Programming – translate design specification into code • Testing • Conversion – plan conversion, train users • Production & Maintenance – operate, evaluate and modify the system Laudon & Laudon (2006) Testing Why test? System should be tested to ensure that it meets the requirements laid out at the start of the process. Develop a test plan • Involve users as well as developers • Simulate data and test system processes e. g. entering a new record, inserting incorrect data Laudon & Laudon (2006) Test Plan Test Number Purpose of Test Test Data Expected Result Actual Result 1 Test the validation rule on the “Date of Birth” field to ensure that valid dates of birth can be entered. 15/02/1988 Accepted Accepted 2 Test the validation rule on the Date of Birth field to ensure that a date in the future cannot be entered. 14/10/2013 Rejected Accepted Adapted from: Laudon & Laudon (2006) Testing 2 basic phases of testing: programme testing – ensures that the programmes work as expected in controlled environment • acceptance testing – ensures users are able to use the system and that it operates as expected when in use. Laudon & Laudon (2006) Programme Testing 3 phases • unit testing – individual modules of the system are tested for any potential errors in the code. • integration testing – to check the modules work together as expected • complete system testing – see how all the components will work under various conditions Bocij et al (2006) Acceptance Testing User Acceptance Testing (UAT) by actual end users of the ystem to determine how well it meets their expectations and requirements. • alpha testing – using simulated data • beta testing – using actual data – final stage before implementation Bocij et al (2006) Acceptance Testing • Security testing – whether security policies have been implemented as intended • Recovery testing – how the system reacts when it has been forced to fail • Performance testing – in different environments • Stress testing – attempts to “break” the system by not following guidelines. Looks at how the system performs under heavy demand. Bocij et al (2006) Limitations of Testing Impossible to guarantee that once a system is tested it will work exactly as expected • Too many factors to carry out exhaustive testing • It is often given insufficient importance and other stages overrun at the expense of testing • Cannot rule out unforeseen problems further down the line BUT: Gives a good idea, identifies major errors Morris (2005), Bocij et al (2006) Case Study 1 – Careers Service • New system to manage Careers Service appointments, workshops and events • Alpha testing revealed few errors which were resolved • Beta testing revealed significant issues re: student status, ducation details, appointment booking abilities • Issues were resolved and pilot system launched • Phased implementation. Testing continues… Case Study 2 – Knight Capital Group • Trading US equities electronically (high frequency trading) • Dormant software from a legacy system inadvertently reactivated • Multiplied stock trades by 1000 • 45 minute delay in shutting down the system • Proper testing and “appropriate market simulations” • Estimated $440M loss Detterman (2012), Ruhle, Harper and Mehta (2012) Summary • Defined SDLC • Types & Stages of SDLC • Types of Testing • Limitations of Testing Case studies demonstrate importance of thorough testing Conclusion • Testing reveals any errors and incompatibilities • Essential to ensure they can be corrected before implementation • Testing is a continual process. There needs to be a cut off point before implementation but it can (and should) continue • Testing is critical because it is the only way to determine whether the system operates to requirements and actually works! Laudon & Laudon (2006), Cervone (2007) References Cervone, H. F. (2007),”The system development life cycle and digital library development”, O CLC Systems & Services, Vol. 3 Iss: 4 pp. 348–352 [online]. Available from: [Accessed 12 November 2012] Dennis, A. , Wixom, B. H. & Roth, R. M. (2008) Systems Analysis and Design, 4th Edn, Oxford: Wiley Detterman, E. (2012) “Software Development Life Cycle (SDLC) Case Study – Result = $440m Loss. ” Solid Logic Technology. 12 August 2012 [online]. Available from: [Accessed 11 November 2012] Galin, D. (2004) Software Quality Assurance: from theory to implementation. Harlow: Pearson Education Laplante, P. A. & Neill, C. J. (2004), “The Demise of the Waterfall Model Is Imminent” and Other Urban Myths”, ACM Queue, Vol 1, No. 10 [onliine].
Available from: [Accessed 12 November 2012] Laudon, K. C. and Laudon, J. P. (2006) Essentials of Management Information Systems: Organization and Technology in the Networked Enterprise, ninth ed. , New Jersey: Prentice-Hall Morris, S. , (2005) “Software Development Cycle”, Tessella Support Services plc, available at < http://www. tessella. com/wp-content/uploads/2008/05/softwaredevelopmentcycle. p df> [accessed 12 November 2012] Ruhle, S. Harper, C. & Mehta, N. (2012) Knight Trading Loss Said to Be Linked to Dormant Software. Bloomberg. 14 August 2012. [online] Available from: {Accessed 01-11-2012]