Dependency Driven Development
We need to encourage dependency driven development. This includes not only the style of writing tests first: TDD, See Fowler. It also extends to testing development, staging, and production environments before deploying to them.
TDD is testing interaction with environment
At each level of testing: functional, unit≥ there is the concept of mocking something out. The practice of mocking is a way to isolate the code under test. Essentially testing is a way of providing environment which can be measured, so that we can test that our code under test interacts with the environment appropriately. Environment is key to testing.
The problem with running tests in different environments
Tests run differently in different environments. When we write automation tools to run our tests we strive to make the environment more and more consistent because it is only possible to observe a given behavior if we can control and isolate the environmental factors that might affect it.
For example, in Jenkins, we often use versioned folders to contain our code, so that it can be run in a fresh environment. That folder driven environment is good, but it leaves a lot to be desired. We end up adding most of our environmental dependencies to the Jenkins Job (when we should be adding it to our code). Also, Jenkins environment must conform the Jenkins slave it's running on. So we need to deploy Jenkins-slaves on all platforms. This makes scaling our Jenkins cluster difficult and complex.
Environment is a dependency
Test Driven Development formally applies to the practice of writing failing tests in a development environment, and then writing code to fulfill the assertions of those tests. We need to avoid talking about formal TDD (as it's been covered in depth). Instead I want to talk about the workflow that ensures the process is consistent across all platforms.
Writing tests is a good start. We need to also use a consistent development environment. I should be able to trust that code I write on my Macbook Pro, running OSX 10.10.5, will also run on your IBM Thinkpad, running Ubuntu.
Dependency Driven Development
I'd like to pen a new phrase: dependency driven development (DDD): a process where the interfaces provided by dependencies are driving the development and design of a product. Utilizing the principles of composition and of programming to an interface; in this practice a developer is encourage to compose the environment, injecting it into the application framework; encouraged to program to that injected environment through a consistent interface. We should be testing all the environments: development, staging, and production.
See Design Principles from Design Patterns for more on the benefits of programming to an interface and composition.
The environment should be controlled as closely as the mocks in a unit test. It should be provided consistently, and composed explicit, in order for the results to be deterministic. We should specify the interfaces of the environment we are designing to.
The objectives are:
- Designing to the interface of the environment
- Composing the application with the environment as a dependency
- Writing tests that cover environmental changes
- Writing tests that mock out important environmental conditions
- dev, prod, stage
Mocking out Environment
It should be done. Mocking out the environment is a process not unlike mocking dependencies in a unit test. Like unit tests, it has the benefits of:
- Documenting environmental interfaces
- Focusing tests on key functionality, in multiple environments
- Composing all environmental dependencies, and monitoring interactions
Essentially, something like Docker provides a way to ship consistent environment. So, that way, we can actually run our tests against all environments. We can ensure programmatically that our production, stage, and development configurations do what they should.
Mocking Out Services
By taking over DNS, an environment will direct requests to mocked services rather than the real ones. Managing all your application's URI dependencies could be made easier with the use of a Configuration service (such as Consul, by HashiCorp).
- Identify Service URLs
- Record a hash table of URI request, and responses
- Store that hash table for replay in functional tests
Development should involve as a first step identifying and resolving the interfaces of external APIs. Record each expected interaction. If you are designing to an interface, then that first step is to identify how that API interface behaves in a set of functional situations.