The latest Gartner Magic Quadrant for Test Automation focuses on what's needed to enable Continuous Testing, which they define as “the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.”
Here's a quick recap of the report’s key points on test automation and Continuous Testing:
- Test automation across development, QA, and performance testing are critical to achieving the level of Continuous Testing that DevOps requires.
- Avoiding the “test maintenance trap” is one of the greatest challenges of test automation. Practices like model-based test automation, synthetic test data generation, service virtualization, etc. can help.
- The test automation market has changed dramatically over the past year, and more changes are likely to come soon. DevOps will ultimately force organizations to find a sustainable test automation approach that works for their people, processes, and technologies.
- Open source test automation solutions such as Selenium and Appium have made a strong impact on test automation—primarily for developers focused on web and mobile front ends. They have provided a welcome level of "standardization," but aren't yet viable for non-developers such as subject matter experts and business analysts.
For teams looking to review or replace their existing test automation solution, the report also reviews and ranks what they believe are the top 9 test automation tools for large enterprises. The evaluation involved inquiries with Gartner clients, surveys of tool users, vendor responses to Gartner questions, and product evaluations. Gartner required tools to support native Windows desktop application testing and Android or iOS testing support as well as support 3 of the following: responsive web applications, mobile applications, packaged applications, API/web services.
Along with the vendor-specific reviews and rankings, the report provides the following recommendations:
- If you don’t already have a software testing tool deployed, start learning about the software testing tools market and evaluating tools that will meet your needs.
- If you already use a software testing tool and are considering switching, consider adding complementary software testing tools to extend your testing capabilities without the hassle of switching.
- If you’ve outgrown your legacy software testing tool, start considering alternative software testing tools based on your current and anticipated future (12 to 18 months out) needs.
- If your existing software testing tool is meeting your needs, reassess the tool’s effectiveness at least yearly.
Download the complete “Magic Quadrant for Software Test Automation” report to review the results of Gartner’s tool comparison, their insights on Selenium and other open source software testing tools, and their detailed recommendations for test automation.