Review System Requirements
Prior to testing a website or system, the product may have gone through a number of phases. One of these phases is a requirements analysis where information is obtained from the client about what the system should achieve, what functions it should / should not perform and what business requirements should be addressed. The documentation would also include technical specifications, initial designs and project plans. The output from this phase are the 'Requirements Reports' and 'Project Plans'.
The Requirements Report outlines the purpose for which a system is required. It may specify mandatory and optional requirements. These are sometimes in order of priority.
In addition there will be industry-based specifications and standards.
You may also consider a review of legal requirements. These may include:
* Privacy. When personal information may be transmitted, published, solicited and collected via the internet the relevant privacy implications must be considered.
* Restricted content. Content with a sensitive or offensive nature may not be published without restrictions.
* Copyright. The Copyright Act includes legislation regarding the use of copyright material on-line.
* Encryption and security. Encryption and security may be required when information is transmitted over the Internet. Encryption and security may be associated with financial transactions, confidential documents, personal information and military information.
* Accessibility. Disability Discrimination applies to WWW in Australia - read World Wide Web Access: Disability Discrimination Act Advisory Notes.
For a website, there will be many requirements that need to be met. The client may request testing of each and every requirement to make sure it has been achieved. Alternatively the client may take a risk-based approach, in which case only those areas of high risk which have a high likelihood of being incomplete or unsatisfactory and also have a high impact or high severity associated with the client's business or product quality may be assessed.
This process will enable the identification and selection of mandatory and optional criteria for the system and enable the development of acceptance criteria.
Develop the Test Plan
After the acceptance criteria have been determined and agreed on, the next step is to incorporate these into a formal test plan. The test plan provides the agreed framework against which the test will be conducted. In any tightly-focused test, the plan should detail the items listed in the example below.
Resources for the test would include:
People (Developers, Users, Subject Matter Experts), equipment and documents that provide measurement criteria (Functional Requirements, Technical Specifications, Initial Designs).
People: Who does the testing?
Some people know how software works (developers) and others just use it (users). Accordingly, any testing by users or other non-developers is sometimes called "black box" testing. Developer testing is called "white box" testing. The distinction here is based on what the person knows or can understand.
Definitions:
* Black Box Testing
Also known as functional testing. A software testing technique whereby the internal workings of the item being tested are not known by the tester. For example, in a black box test on software design, the tester only knows the inputs and what the expected outcomes should be and not how the program arrives at those outputs. The tester does not ever examine the programming code and does not need any further knowledge of the program other than its specifications.
* White Box Testing
Also known as glass box, structural, clear box and open box testing. A software testing technique whereby explicit knowledge of the internal workings of the item being tested are used to select the test data. Unlike black box testing, white box testing uses specific knowledge of programming code to examine outputs. The test is accurate only if the tester knows what the program is supposed to do. He or she can then see if the program diverges from its intended goal. White box testing does not account for errors caused by omission and all visible code must also be readable.
Review the Test Plan
Following development of the plan, a quality review and validation is required.
The objective of this review and validation is to confirm that the test plan, when implemented, will achieve the intended outcome. That is - 'Provide the client with confidence that the product will perform as expected'.
This review is preferably performed by someone independent from the test plan author.
The review should address the following criteria:
* The plan execution is feasible in terms of resource requirements, timeframe, and practical implementation.
* All plan activities are - Relevant, Specific and Measurable.
* All plan activities conform to the client business standards, policies and procedures.
* All plan activities focus on the objective of proving that the system or component satisfies the relevant acceptance criteria.
* The plan execution will produce timely, meaningful and relevant information.
* The plan output information will be in a format and content suitable to the user's level of understanding and knowledge.
Notification of Acceptance Test
There should be no surprises for the client who should be closely involved in the development of the acceptance criteria, test plan and relevant resource needs. The client needs to fully understand the processes involved in the plan execution and the required level of involvement. The client should be made aware that this testing provides the opportunity for them to expose the system to all reasonably expected results, prior to going 'live'. The client will be expected to accept ('sign off') at the end of the testing process. This 'sign off' may range from a decision not to implement, through acceptance with conditions, to full implementation.
Copies of any developed user manuals or instruction documentation should be made available in advance of the test to enable prior client training. Examples of system outputs - reports, error messages, on-line user instructions - are also valuable pre-test information for the client.
The communication method will depend on arrangements made with the client; but should allow the opportunity for feedback and clarification if necessary. It should, however, be a formal documented process.
The client should be requested to provide:
* Necessary infrastructure to simulate a 'live' test environment.
* Test data to populate input areas.
* User / customer input forms or data structures.
* People resources to test the system - these can range from Subject Experts to untrained users.
The timetable and location for initial and re-testing (if required) should be agreed with the client prior to the test execution. An indicative timeframe for test result feedback should also be established.
The above arrangements should be made with the authorised business representative for the client. This representative is often established earlier in the original client / developer discussion. It may be appropriate to re-confirm the authorised contact.
Activity 1: Research standards organisations and write brief report
A: The W3C is a company that provides a technical report development process which is the set of steps and requirements followed by W3C Working Groups to standardize Web technology. Through this process, W3C seeks to maximize consensus about the content of a technical report, to ensure high technical and editorial quality, to promote consistency among specifications, and to earn endorsement by W3C and the broader community. Learn more in the introduction to the W3C Process.Besides, W3C develops technical specifications and guidelines through a process designed to maximize consensus about the content of a technical report, to ensure high technical and editorial quality, and to earn endorsement by W3C and the broader community.
W3C provides a number of views of its specifications, including:
- by technology topic (such as “all the HTML-related specifications”)
- by status (this is an approximation of the "classic" TR view, with recent publications at the top and then all specifications grouped by status, from Recommendations to drafts)
- by date (most recent at the top)
- by group (who is working on the specification)
Perform Functional Testing
Planning for the test was the first phase of Client Acceptance Testing. The planning process involved scoping of acceptance criteria based on the system requirements and formulating and documenting a test plan for communication to the client. It also completed the preparation work by scheduling the acceptance test in liaison with the client's authorised representative.The next phase and the topic of this resource, moves to performing the functional and business process testing in a controlled environment.
This resource focuses on:
- Setting up the client test environment
- Performing the test in compliance with acceptance criteria scoping and associated documentation
- Documenting the results of each test iteration.
During the system's Develop and Document stage in the System Development Life Cycle, bench-testing (often referred to as 'Unit testing') of individual components, processes and other functional aspects (code, connectivity, etc) would have occurred. This testing is generally performed by the development team to ensure that what they have created is stable and functional. They might test that:
- They cover all the lines and logic paths through their code
- All the screens flow backwards and forwards in the correct order
- The software meets the functions specified (e.g. calculations are correct, reports have correct columns, screens have correct validation)
This testing might not be done through the application itself (often because it has not been completely built while they are testing), so they might only add a few records, maybe by editing the file/table and adding the records, rather than using a 'Record Entry Screen'. Also, as the developers are testing their own work, there is a tendency to skip areas they 'know there will not be a problem'.
This development testing may also have involved system prototyping which has enabled client review during development.
In the system's Implementation stage, we first move to a controlled test in a 'live' environment, where the system can be subjected to final independent client rigour. A well-designed client test gives a complete 'business coverage' (as opposed to code or logic path coverage). It tests all the transactions that the client or end-users run and will highlight any potential areas of adverse impact on the business process.
Following a satisfactory acceptance test, the system can safely progress to final real-time implementation and subsequent Maintenance stage.
