Bitcoin Support Your contribution funds and supports Keyrox's website

For agile testing, fail fast with test impact analysis

“Failing fast” lies at the heart of agile. Teams want continuous integration (CI) to provide feedback on their latest updates as soon as possible. CI test results are the primary barometer that developers use to determine whether it’s safe to move on to the next development task, or if they inadvertently broke functionality that users have grown to rely on.

With more extensive and more effective regression testing during CI, you’re much more likely to spot critical problems as soon as they’re introduced—which is when they’re fastest, easiest and cheapest to fix. However, given the frequency of builds in most agile processes, there’s simply not much time available for test execution. Developers expect feedback in a matter of minutes, but most regression test suites—especially in Global 2000 organizations—take hours (or days!) to execute. This seems to force a trade-off: Settle for sluggish CI or scale down testing.

Having slow CI impacts productivity across all the team members waiting to review, extend, document, and test the newly integrated functionality. The closer the end-of-sprint deadline, the more painful each minute of waiting seems. Of course, you can (and should) accelerate the process with parallel/distributed test execution and similar technologies. But ultimately, if you want to accelerate the execution of a large enterprise test suite, you will need to make some hard decisions about what tests do not need to be executed for each build. 

Ways to speed up (and scale down) testing

One common approach to speeding up regression testing is to slash end-to-end tests. This is a huge mistake. As I’ve noted in past columns, I know that unit tests are valuable for exposing implementation errors. However, I just don’t see unit tests catching the issues that really impact the end user, which are largely integration issues. If you skip end-to-end tests designed by a professional tester who really understands the domain, you’ll inevitably miss a lot of issues that have a very direct impact on the user experience.

Read More…

Leave a Reply

one × five =