IS IT A GOOD IDEA SOLVING TEST ENVIRONMENT PROBLEMS “LATER”?

In recent years, many companies focus on improving their test process. The aim is that the improvements will lead to measurable improvements in quality, faster time-to-market, cost reduction and more efficient IT operational processes. Still, they think it is best to first establish a good test process and then consider if they need to improve their test environment.

World Quality Report 2013-2014 found that many companies have challenges with the test environment and test data. It is a new area of concern.

The problems are often put forward as “statements” from the test managers and testers. They struggle with downtime and interference with each other during the tests. Too much is often going on in the same test environments simultaneously since projects often have the same delivery deadlines. At the same time, they  do not get much sympathy from management. The project teams are seldom responsible for delivering the test environments – it is delivered by an infrastructure partner. Reports “document” that the test environment is ‘up’ and available according to SLA requirements or other agreements with infrastructure partner. However, I have been test manager in several projects where the test environment was not available half the of the test period.

Examples of problems:

• “Someone” has added new code or changes in the test environment leading to serious errors.
• “Someone” has given the test team the wrong version of the code because of lack of configuration management.
• “Someone” is running a job that leads to poor performance.
• The data is too old and outdated.
• There are not enough devices available for testing apps. This leads to manual tests with the most common devices and poor test coverage.

Assessments we have done for clients show us that these challenges are seldom documented by the test team, and the management does not take decisions based on emotions. They need facts. Problems during the test periods must be documented so that the management can find the right balance between risk, design, management and support, and identifying the optimal level of investment in test environments. 40% of the total test costs are related to the testing hardware and infrastructure. Professional Test Environment Management (TEM) services enable the organizations to speed up their software release schedules by up to 25%, cut infrastructure costs by 5-10% and increase item team productivity by up to 30%.

In other words,  it will be profitable to improve the test environments. Test environments are complex and the challenges will increase with the need of apps testing on devices in addition to traditional tests. It will require professional test environment management and test data management, with SLA agreements that secure stable and available test environments.

Test Environment Management services provides a holistic approach to the area of technical testing services, from Service Management, Data, Build, Architecture, Infrastructure, Applications and Virtualization.

1. Test Environment Management (TEM) Services.
2. Test Data Management (TDM) Services.
3. Development Operations (DevOps) Services.
4. Infrastructure, Virtualization & Cloud (IS) Services.
5. Wide-angle Application Rationalization Program (WARP) Services.
6. Service Virtualization (SV) Services.

To secure expected effect of the improvement of the test process it is necessary to also improve test environment and test data management. It must be seen as key areas in the improvement program. A good advice to all testers struggling: document and report the challenges you have with the test environment.

More Information

Gro Rognstad has a passion for Testing and Quality. She is Chief Technology Officer in Sogeti Norway and responsible for bringing our testing services and innovations to customers and employees. She is a senior advisor with a broad background and 27 year in IT – the last 17 years focusing on Testing and Quality. She advises organizations on innovation and Quality Transformation projects. Gro is active as a course trainer and speaker at seminars. In 2007 Gro was in the management team starting up Sogeti as a company in Norway. Gro is leader of Testing Services Community in Global Service Line Testing at Sogeti Group. She is also member of Global Testing Leadership Community in Cap Gemini group. In addition to that she is engaged as a board member of the Norwegian Computer Society, Community for Software Testing.