In my last post [Continuous Testing part 1] I described what a typical build script inside a continuous integration tool looks like. And how they’re often a nightmare to maintain.

I didn’t even get to the point I wanted to talk about — which is that
something about continuous integration is broken.

Part of the problem is that people are trying to code in text boxes and doing setup and configuration through a GUI.

With discipline and practice, you can have everything within a tidy, modular build script so that all your CI server is just calling that (and nothing else)

Your job looks like :

/path/to/build_script.sh $OPTION1 $OPTION2 $OPTION3

but no more than 3 options!

It becomes nothing more than a glorified GUI on top of a cron job.

Which is all a CI server should be.

But was it JWZ or ESR who said something about every program attempts to expand until it can send email (and parse lisp?)

—

I think that Continuous Integration promises something more.

That’s what “Continous Delivery” tries to reach for too — and has gotten bogged down in the task of provisioning and managing build & deployment environments.

We have two tasks here that are really quite separate:

1. Building a product.
2. Deploying it to a working environment and running it.

But they’re both trying to accomplish the same goal:

Testing

You want to build a product continuously so you can test it frequently. Both unit tests, and system tests.

The “I” in Continuous Integration stands for “Integration Testing”

Likewise, you want to deploy a production continuously so you can test it frequently.

The “D” in Continuous Delivery stands for “Integration Testing” too.

Because to test something completely you need to deploy it to a production-like environment and run it through it’s paces there.

It just so happens that the environment you build a product on is also almost always an environment you can run it on. Developers know this because that’s how they know it works when they’re building it.

This is called a development environment. Your build system needs the runtime to build it but it needs a lot of other things.

Developers need javac to compile their code, but they also need java to run it. If they’re building web apps, they need a web server to deploy it to, a browser to connect to the web server, and maybe a database or two to get data from. They probably also need to mock out a service or call a development version of a service that doesn’t scale.

But a production environment looks nothing like a development environment.

And a testing environment is always a compromise between the two.

Since developers build developer tools, they build them with themselves in mind.

They’re thinking about unit testing, mocking services, and deploying to dev environments — preferably with as few steps between compile and deploy as possible.

That’s why we have CI servers that work the way they do. They were build by developers for developers.

But dev environments aren’t good for integration testing.  Ever heard a developer say “it works on my box?”

Similarly, we have Continuous Delivery servers that work the way they do because they were build by (devops) for sysadmins. They’re interested in getting the tools and processes for provisioning and deploying products in place — because that’s what they care about.

But as I said, the purpose of CI & CD is to facilitate testing. What if we had a Continuous Testing server that was designed (if not build) by testers, for testers?

Maybe it’s a piece that fits in between (or after) Continuous Build and Continuous Delivery. Or maybe it’s part of the same tool. But I think it’s the next step.