“Are you only interested in test automation?”
I was asked this question casually, and here is my (detailed) response:
My opinion is that test automation serves 3 main purposes:
1. Help developers move faster by giving them rapid feedback as to whether their changes “broke” anything.
2. Help testers move faster and find bugs better by doing the boring, repetitive work that takes up their time and dulls their senses.
3. Help operations know that deployments are successful and environments are working with all the pieces in place communicating with smoke tests and system integration tests.
In each case, the automated tests’ primary roll is making sure the system works as expected and providing rapid, repetitive feedback.
My opinion is that manual regression tests don’t necessarily make good automated regression tests — and that you can often develop them independently easier.
When I create automated tests, it is usually through the process of exploratory testing and analyzing requirements, and then determine when and whether to automate based on the following criteria:
Is this a check that will need to be done repeatedly?
(Are the manual setup or verification steps difficult & slow to perform manually, or do they lend themselves easily to automation — e.g. database queries, system configuration, API calls, etc?)
Is this interface (UI or API) going to be stable or rapidly changing ?
(Will you need to frequently update the automation steps?)
Will this test provide enough value to pay the up front cost of developing automation?
(Is it harder to automated than to manually test? Or will it only need tested once?)
The reason I don’t recommend translating manual regression tests into automated tests is because not all these criteria can be met, and also that often, not all of the manual steps need to be reproduced in automation — or automation can do them more efficiently.
For example: creating accounts via an API, verifying results via a SQL query, or bypassing UI navigation steps via deep linking, setting cookies, etc.
Automation also lends itself well to testing many data variations that are too tedious to perform manually, so you can end up testing some scenarios more thoroughly than you would with manual regression (due to time constraints).
And sometimes, the cost of automating something is just not worth the effort, and it’s easier to perform manual checks.
All this said, I’m not opposed to doing requirements analysis and manual testing — in fact I consider it an important part of deciding what and how to automate. I just don’t think that automation should be considered a secondary step — because of my initial premise that the goal of automation is to accelerate those other rolls of development, testing, and delivery.
Fijiaaron 