An Asynchronous Test Runner?

Here’s a conversation on LinkedIn talking about which programming language you should choose for a test framework — including comments about how automated tests are inherently synchronous (which I agree with) and why someone would write an asynchronous test framework in, for example JavaScript.

https://www.linkedin.com/posts/vikas-mathur_qa-testing-testautomation-activity-6871687300267474944-1MUL

Vikas Mathur

Typically, programming language for test automation can be any language, irrespective of which language the developers are using. However, to allow for more and effective collaboration with the developers, using the same language as them might make sense. Of course, there might be situations where it does make sense to use a different language – but in most cases, it makes more sense to use the same language. In case the language is different than the developers, it might make sense to use a language that has a good ecosystem for test automation. Something to think about while selecting a programming language for test automation.

(I agree with this)

Gabriele Cafiero 

I can’t still figure out why someone chose Javascript to test things. It’s asynchronous, and tests should be synced by definition. It has a weak typing so you have to double check any assertion of yours

(I agree with this also)


But here is where it sparked my own thought:

90% of the time, Javascript doesn’t need to be async, but library writers have a fetish for it because it was difficult for them to learn and so they want to show off that hard won knowledge.

But…there is a reason for asynchronous code and that’s efficient resource usage through task switching (i.e. the call stack).

Tests also have a need for this — although it’s not really utilized in any framework:

1. Tests need to run concurrently so you can get faster results
2. Tests take different times to complete, so you shouldn’t have to wait for a test blocking
3. Tests interact with asynchronous events (e.g. network, UI)
4. Test runs shouldn’t have to be discreet sequential loops.

Wouldn’t it be nice to have tests run continuously?
A test runner that listens for events to trigger tests and you don’t have to wait for (or kill) the previous test run to start another.

Imagine a continuous test queue that doesn’t depend on a specific job to run,

Imagine failing tests that dynamically rerun for stability.

Imagine commits that trigger specific tests, but don’t worry about full regression or smoke testing because that’s happening in the background.

So while an individual test needs to run synchronously (but also needs to await asynchronous events sometimes), having a test runner that operates in an asynchronous, event driven style is a great opportunity that hasn’t (to my knowledge) really been explored.