Some of the generic challenges in software development that may be applicable to different teams either as it is or with some minor modifications have been identified as below:
1. Basically, very important documents in the development process, like SRS, HLD, LLD’s, Conceptual design docs, and UI information are not available on time. However, non-functional specifications, like reliability, performance and other aspects of the product, are not available on time and in detail.
2. There is no interlock between the design and the development team during the design discussions and the implementation discussions.
3. Clear-cut scope line between the functional/module testing as well as System testing activities is rarely seen.
4. There is less interaction between the product management and the requirements gathering team.
5. There is another possibility for ambiguity to understand the boundaries between the test cases or script for the functional/module testing and the system testing.
6. There is very limited testing time available due to squeezed timeframes for system testing.
7. In addition, the very important challenge is limited and incomplete customer or end-user interaction with the project team.
Finally, the focus of this unit is to establish a good testing process to overcome most of the above-stated limitations or challenges and set up an effective system testing team.
Jump to Section
Identify The Process Elements
It is essential to define an end-to-end procedure for the system-testing phase, including all the exception and deviations that might occur during the testing cycles. The Key to define the end-to-end procedure is to break up the entire procedure into smaller process elements. While each processing element signifies a particular stage of the testing cycle and has well-defined entry and exit points.
This helps the team lead or manager to perform the following activities better:
- Design test strategy while design phase.
- Test plan and test estimation while planning.
- Manage test team while running tests.
- Defect reporting and tracking while System test.
Basically, these activities are the backbone of good testing. Getting familiar with the project domain and requirements is important for a tester. This requires groundwork, as a result for the tester has to approve the entire system and not a part application. Also being present at those critical meetings, and help the test team to be aware of decision points for project execution. Also, the parameters help test manager/test lead to plan and design test activities better. Test team can also look into the requirements from a different perspective by giving a thought to the organization policies and organization audit criterions that are critical to the overall project acceptance by the majority of users.
The System testing phase of software testing can be generally broken up into the following process elements (some of the process elements can be added to or removed from the following list, based on the system testing requirements and the organizational standards):
Identifying the components and areas to be tested for test strategy
>Design master test plan
>Prepare the test schedule and estimates
>Prepare detailed test plan(Components level or for the non-functional attribute testing.)
>Identify the test configuration matrix or the topology plan
>Define the review mechanism for the test plans while running test
Develop and review the test cases
Test Execution and reporting
>Identify the test execution cycles
>Define the testbed setup process while testing phase
>Difine the defect tracking system while system testing
>Quality certification(QCERT) review process
>Include the field reported defects in the test cases for the next release.
This process elements focus on identifying all the areas/components/aspects/facts of the software that need to be tested during the system testing phase. The Test manager/Test lead, responsible to manage a testing team, creates the test strategy initially in the testing process. This testing strategy will define the required testing approach that is to be followed by the testing team corresponding software testing.
For example, consider a web-based application built around the entire architecture. The following areas can be good candidates to be considered for functional and system testing:
- Component functionality testing
- Integration of components as well as intersystem testing.
- Browser compatibility and internationalization testing.
- Proxy and firewall testing
- Authorization and accessibility testing.
- load balancing as well as availability testing.
- Reliability testing and stress testing.
- Package and release documentation testing
- Serviceability testing also
Master Test Plan
These elements focus on defining the master or the overall test plan. In addition, the master test plan should undergo a well-defined review process. The review teams, usually constituted by the Senior Management Executive, Department team members, and as well as managers. Senior members of the testing teams include the test manager. In case of software products planned to have multiple releases, It is recommended to have multiple master test plans associated with each release. Hence, the master test plan from the previous release would define the baseline for planning the next release. A master test plan ideally contains the following sections:
The test plan objective
System test Organization
>Roles and responsibilities statement.
>Componant ownership matrix.
>Skill set definition for the testing team.
System Test Planning
>System Test Strategy
>Template definition for the detailed test plans.
>Configuration matrix or the topology plan
>The asset management repository
>system test entry and exit criteria definition
>reporting mechanism(including the QCERT report)
>In scope and out scope statements for System Testing.
> Automation test execution tools.
> test asset management tool
>project management tools for estimation and scheduling
>team collaboration tools.
Risk and mitigations
list of risk and mitigations with respect to available resources for system testing
list of documents to be referred by the testing team
list of review techniques to estimate the completeness and correctness of system testing process
Also the list of approvers to approve system testing process
Prepare Test Schedule and estimate
A test schedule is an important element in the testing process. It defines the start and the end dates of the various activities, which are to be performed during the testing phase. It also specifies the sequence of events that will be followed during the testing phase. The test schedule also signifies the time requirements for each of the activity identified and hence project the overall time required for the testing phase.
Prepare Detailed Test Plans
Each of the areas identified in the figure must have a detailed test plan addressing the system testing aspects in detail. We also can define a template for a detailed test plan which is as follows:
- Purpose as well as an objective statement
- Scope Definition (In Scope and Out of scope)
- Test Methodology(test scenarios definition)
- Estimated time to test(in man hours)
- Dependencies as well as other components tests
- Required reviewers and approvers
Identify Test Configuration Matrix
The definition of a well-defined configuration matrix or a topology plan can contribute a great deal in completing a major part of the system testing process. Basically, a configuration matrix provides the definition of all the supported permutations and combinations of all the system parameters, like required hardware, software, and documents. Each definition results in a configuration, which should be tested for the software product. The configuration matrix can be in the form of a simple spreadsheet generated by the test manager based on his experience, product domain knowledge, and also a good understanding of the target customers in the market.
Define Review Mechanism for the test plans
Component level system test plans need to be prepared by following a well-defined procedure. A typical process definition can be like this:
- Create test plan and set the status to ‘Draft’
- Create test plan content and set status as ‘Ready for Review’.
- Notify the reviewers to review the test plan.
- Reviewers provide feedback.
- Notify the approvers to approve.
- Once all approver has approved, change status to ‘Accepted’.
We discussed some of the generic challenges in software development. Also, these challenges may be applicable to different teams. In this blog, we pointed out some generic challenges that can affect the whole process of software development. We also discussed some solutions to resolve these generic challenges. Finally, this blog helps to avoid some challenges in software development.
- COVID-19: How We Are Dealing With It as a Company - March 23, 2020
- Agile Testing – The Only Way to Develop Quality Software - February 8, 2019
- How to Perform System Testing Using Various Types Techniques - May 16, 2018