This post was based on a comment on another blog post about the StarWest 2011 conference.

Automation should primarily be used for regression testing & other scenarios that require repetition and accuracy.

The point I try to stress is that the goal of test automation is to create time for exploratory testing. In that perspective, a good measure of the effectiveness automation is that it:

1) doesn’t take more time than manual testing would — thus allowing time for manual exploratory testing,

2) catches and prevents defects. And it should do it very quickly. Ideally with each build.

If you’re not accomplishing both of these, I’d question the value you’re getting from automation, but even more, the value you’re missing from neglecting manual tests.   But there’s also a balance that it takes time to get to the point where automation pays off, both in experience and in infrastructure.

So while effective automation is essential to successful agile, the process of implementing automation is difficult to accomplish in an Agile way.