MOUNTAIN VIEW, Calif., July 23, 2018 (GLOBE NEWSWIRE) -- Results from new Forrester Consulting research reveal the 20 most important Agile and DevOps quality metrics that separate DevOps/Agile experts from their less advanced peers. The study, which publishes today, was commissioned by Continuous Testing leader Tricentis and was led by Forrester Vice President and Principal Analyst Diego Lo Giudice.
The research examined 94 quality metrics that were classified into four categories. As a high-level summary, the research uncovered the following:
- Understanding of business risk is the most important determining factor of DevOps and Agile maturity.
- Experts focus primarily on contextual metrics (e.g., requirements coverage) while others focus on “counting” metrics (e.g., number of tests).
- Experts are more likely to measure the user experience across an end-to-end transaction while others rely on application-specific or team-specific metrics.
Detailed results and analysis are available in the complete report: The Definitive Software Quality Metrics for Agile+DevOps.
Agile/DevOps Leaders vs Laggards
The study also found that companies with the most successful Agile and DevOps initiatives do a number of things differently:
- They are transforming software testing into Continuous Testing by adopting five core practices, such as automating end-to-end functional testing and integrating testers into cross-functional teams.
- They are almost twice as likely to consider automating the software quality process to be a “critical business differentiator” (53% vs. 27%).
- They are significantly more likely (by 23 percentage points) to have high levels of automation for key testing and QA processes (test case design, functional test automation, test data management, etc.).
“Organizations adopt DevOps to accelerate the delivery of innovative software that creates positive user experiences,” explained Clark Golestani, Managing Director of C Sensei Group and former President, Emerging Businesses and Global CIO of Merck. “If you start accelerating delivery without also rethinking your approach to quality, you will end up rapidly delivering updates that drive your customers to competitors. Automated, continuous testing is the gate that allows you to protect your organization from significant risk as you innovate at the speed modern business demands.”
The Risk “Blind Spot”
Risk is a prevalent thread throughout the various parts of the report. Most firms (80%) believe they deliver within acceptable business risk, but fewer than a quarter state that their QA and testing processes completely cover business risk. Moreover, just 15% of respondents say that their test suites reliably provide a good indication of business risk.
Interestingly, although risk-related metrics do not rank high in overall popularity, Agile and DevOps experts measure them significantly more frequently than the non-experts do. In fact, Agile and DevOps experts consistently rank risk-related metrics among their top 3 most valuable metrics in different phases and categories.
Additional Findings
Other notable findings include:
- Only 26% of firms with active Agile/DevOps adoptions are currently practicing the five core Continuous Testing practices shown to separate Agile/DevOps leaders from laggards.
- Many CXOs and high-level decision makers are vastly overestimating the maturity of their firms’ Agile and DevOps practices.
- Firms that rely on manual testing are more than twice as likely to report that testing is a bottleneck (59%), compared to those with more automated quality processes (26%).
The complete Definitive Software Quality Metrics for Agile+DevOps report is available for complimentary download on the Tricentis web site.
About Tricentis
With the industry’s No. 1 Continuous Testing platform, Tricentis is recognized for reinventing software testing for DevOps. Through risk-based testing, scriptless end-to-end test automation, and the industry’s most extensive technology support, Tricentis breaks through the barriers experienced with conventional software testing methods. Our innovative technologies simplify testing for even the most complex enterprise applications—transforming testing from a roadblock to a catalyst for innovation. The result is accelerated software delivery speed, improved cost efficiency, and reduced business risk.
Tricentis is the only vendor to achieve “leader” status in all three top analyst reports (i.e., the “Triple Crown”). This honor is based on our technical leadership, innovation, and a Global 2000 customer base of 800+ companies, including global enterprises such as Allianz, ANZ Bank, Experian, First Data, HSBC, Lexmark, Merck, Sentry Insurance, Starbucks, Telstra, UBS, Vantiv (now WorldPay), Vodafone, and Whole Foods. These customers rely on Tricentis to achieve and sustain test automation rates of over 90 percent—increasing risk coverage while accelerating testing to keep pace with Agile and DevOps.
Tricentis has a global presence in Austria, Australia, Belgium, Denmark, Germany, India, Netherlands, Singapore, Switzerland, Poland, United States and the UK. To learn more, visit https://www.tricentis.com or follow us on LinkedIn, Twitter, and Facebook.
Media Contact
Victoria Brown
Revere
+1 (650) 850-2009
Victoria.brown@revereagency.co