Test First Isn’t

Here’s an essay I wrote several years ago but apparently never posted. It didn’t get where I wanted it to go, and turned into a bit of a rant. Hope you enjoy it as much as I still do.

Test first development isn’t testing. It’s development. It’s prep work for a developer. It doesn’t test anything. And as a matter of fact, if you could write tests first that work without testing before running them, you’re probably a perfect coder, and writing tests would be superfluous.

Now, running a test after the code is written is testing, but in the simplest possible sense. In about the same way that spell-checking is editing. Or rather, you could consider the spell checker the compiler, and the notoriously bad grammar checker equivalent to the also notoriously bad warnings a compiler gives. But no one considers a document even “proof-read” because Microsoft Word didn’t underline anything. Actually, a document of any length –say half a page or more– is probably unreadable if there are no squiggly underlines in it.

What would be your own unit tests (or even automated functional tests) that could be done to a document? Pretty difficult to come up with something useful, isn’t it?

But code is structured, you complain? Stop complaining, so is English. As a matter of fact, supposedly your code is just a much longer description of a specification written in English (or Swahili). It’s longer because computers aren’t as smart as people, and because of that, you have to talk down to them and use lots of little words and a lot more punctuation.

Let me help you out. There are some things that can be automated. Such as word count. Formatting guidelines. Header style. Beginning, middle, and end. You could come up with run-on sentence algorithms or text analysis. But the real trick is turning a document into an executable, because that’s what code does.

How are documents executed? By modifying them. The aforementioned word count can make sure the document length doesn’t change — or if it does, that the changes occur in an expected area. Or that formatting changes (margins, fonts, etc.) are changed across the board.

Fairly useless stuff, right? But that’s all automated tests can do for software. Granted, there are a lot of variables in software that aren’t in documents, and code changes happen even more. So there is value in REGRESSION tests being automated. Because a code change is like a document edit. You don’t want to edit one paragraph and expect the format (or word count) of another paragraph to change.

Now, I’m not arguing against automated testing (it’s what I do for a living), but I’m arguing in favor of knowing what it can do, so it can do it well. The biggest travesty in quality assurance is the so called “test first” methodology. It’s really just a cop out that says you’re not going to test.

Supposedly, the idea is to do the things first that you never have time to finish. You know, like washing the dishes. By the time you’ve fixed dinner and eaten it, it’s time for your favorite television show (or online chat game, for you WoWers), and there just isn’t time to do the dishes.

Here’s a novel idea — why not do them before cooking? Cooking is what takes up all the time, and the cutting and mixing and associated tasks. Cooking is Very Important. After all, without the cooks, there’d be nothing to eat. Therefore, of course, cooks are the most important people in the cooking process, right? As such, they don’t have time to do the dishes, so you need to find someone else (less important) to do them. And do them before you start cooking. Makes sense, right?

Okay, you could test (I mean wash dishes) while cooking. Every time a cook dirties a dish, a tester (I mean dishwasher) could wash it. Then, everyone could sit down to dinner and all the dishes would be done, and no one would miss Wipeout (or another quest to chop wood or fight squirrels).

Except of course the dishwasher, who’d also miss dinner. Because while a spoon can get washed after every use, that big pot of goulash can’t get cleaned until after everyone has been served (and served seconds, if the cook is any good.)

And of course, there’s the little problem the cook never thought about of all the dinner plates, silverware, and glasses. Those need washed too. If only they could be washed first, or iteratively.

How can a QA Site help you?

Here’s the follow up post to “What is a QA Site?.

You can see the original document at:

How can a QA Site help you?

·     All your testing information together in one place

Your QA Site dashboard gathers information from the different tools that testers, developers, analysts, and project manager use into one place that increases project visibility and collaboration.

See project milestones and status, task completion, blocking issues, defects and resolutions, requirements changes, document updates, code check-ins,

automated builds, and test execution at a glance.

·     Integrate your existing tools & processes

Use your existing tools and integrate them with your dashboard.  Allow your testers & developers to work the way they want.

Your QA Site doesn’t try to change how your team works or dictate which tools they use.  It complements existing processes and enhances productivity.

·     Add the missing pieces to your QA toolset

What tools do you wish you had but you don’t have the time or resources to ad? Does your organization pass the “Joel Test“? Are some tools in silos that only some team members use?

Enable developers to see project goals, testers to use version control, and analysts to know what requirements are covered by tests.  Make your tools available anywhere with your QA Site.

·     No setup or infrastructure costs & no maintenance required

Your QA Site is a turn key package including everything you need from OS to software configuration.   Keep everything on site or hosted on your favorite cloud platform.

Training and support are available, and the system is maintained for you.

Request your QA site today


What is a QA Site?

I’m working on developing a testing tool I call a “QA Site.”  It consists of a set of hosted testing & development tools and a dashboard user interface. I wrote the following in an attempt to describe it.  

The original document can be downloaded at

What is a QA Site?

QA Site is a testing dashboard

that enables you to see at a glance your testing progress and project status.

Your QA Site dashboard is a home page that summarizes your QA efforts and provides links to your testing, development, and project management tools.

See project status, test execution results, defects, builds, and other notifications all in one place.

QA Site is a collection of software tools

that you use to improve quality and release with greater confidence.

Choose the tools you want to use, commercial and open source, working together.  No vendor lock-in.  You don’t have to change your existing infrastructure or processes.

Your QA Site can include:

  • Version Control
  • Defect Tracking
  • Test Cases
  • Automation
  • Continuous Integration
  • Code Analysis
  • Documentation
  • Project Management

QA Site is a hosted platform

that you don’t have to setup or maintain.

Deploy your QA Site on your favorite cloud platform or within your own network.  No setup or infrastructure costs.  No in-house expertise or maintenance required.

We will set it up and work to integrate your existing tools to build a custom dashboard & reports that fit your needs.

Training and support are available, and the system is maintained for you.

Request your QA site today


Selenium QC integration update

My blog post about connecting to QC from a client side script was mentioned on the PushToTest tutorial about selenium for beginners.

Thanks the Frank Cohen of PushToTest for mentioning me.   Although to be strictly accurate, I don’t talk about using OTA to run Selenium tests, I talk about using OTA  to update test results in Quality Center from a Selenium test (or other test that is run via JUnit, etc.).  It may sound like splitting hairs, but I get a lot of questions about that from confused people.

I should probably draw a diagram and post it on my blog.  I had one on my whiteboard that I used to explain to a test manager the actual workflow.

I’ve written several other blog posts on the subject and have been meaning to put them together into a long article (or short book) on the subject (with source code):

Integrating JUnit tests with HP/Mercury Quality Center
Integrating JUnit with HP Quality Center part 2
Upload Selenium/JUnit test results to Quality Center
Updating test results in QC using the QC OTA APK explained
Getting a QC test coverage report from JUnit

I also have some example code on GitHub that can be used off the shelf to batch update test results in QC from a spreadsheet (though a small effort is required have your tests output their results in the expected CSV format):


If you’re looking to integrate Quality Center with your open source testing tools, give me a call  or write me a note.

I’d be happy to discuss what I can do for you.