I once posted a semi-serious post entitled The 7 Phases of Unit Testing. The phases are:

  1. Refuse to unit test because "you don't have enough time"
  2. Start unit testing and immediately start blogging about unit testing and TDD and how great they are and how everyone should do it
  3. Unit test everything - make private methods internal and abuse the InternalsVisibleTo attribute. Test getters and setters or else you won't get 100% code coverage
  4. Get fed with how brittle your unit tests are and start writing integration tests without realizing it.
  5. Discover a mocking framework and make heavy use of strict semantics
  6. Mock absolutely everything that can possibly be mocked
  7. Start writing effective unit tests
I think the cycle I went through is extremely healthy - unit testing is something best learnt from practice and is something you refine over time. Judging by the comments on the original post, a lot of you agree.

Recently though, I've felt like adding another stage:
8 . Sometimes the best tests aren't unit tests
After awhile, it becomes obvious that some tests are significantly more meaningful when you expand your scope - say to include hitting an actual database. Narrow unit tests and wider integration tests can always work together; but I've found that, in some cases, more comprehensive tests can replace corresponding unit tests. This may not be a proper, but it is practical.

When I say unit test, I mean the smallest possible unit of code - generally a behavior. Most methods are made up of 1 or more behavior. I'd say that you shouldn't have too many methods with more than 6 behaviors (as a rough goal). As an obvious example, in NoRM this method helps identify the type of the items in a collection:

public static Type GetListItemType(Type enumerableType) { if (enumerableType.IsArray) { return enumerableType.GetElementType(); } if (enumerableType.IsGenericType) { return enumerableType.GetGenericArguments()[0]; } return typeof(object); } Clearly, this method is a good candidate for 3 or 4 unit tests (one when the type is an array, one when it's a generic, one when its something else, and maybe one when its null).

As your code moves closer to the boundaries of 3rd party components, the value of unit testing may suffer. You'll still get the benefits of flushing out coupling and enabling safe refactoring (which shouldn't be underestimated), but you'll likely miss out on making sure things will work like they should in production. The solution can be to expand the scope of your tests to include the 3rd party component.

The most common example is database code. Testing a Save method by mocking the underlying layer might work, but there's value in making sure that the object actually does get saved. That isn't to say that a single test that hits the database is good enough - your Save method might be made up of multiple behaviors, some which are better validated with one form of testing than another.

Really, that's one of the key things to remember as you walk down this path - don't think that just because your method is actually saving an object that your job is done. There are likely other behaviors that aren't being tested at all. It's easy to abuse these types of tests and get a false sense of security. The other key is to make sure that it runs like a unit test - namely, that its fast, doesn't require any manual setup, and isn't dependent or doesn't break any other test.

Lately, I've seen interest in using in-memory databases for this type of thing. The benefit is that they are super fast and don't leave stale data. They also don't require special setup. On the downside you still aren't truly testing the most fundamental behavior of your method - that an object will be saved to the database in production. Even with the best O/R tool I've seen code work against one database but not work against another - due to a bug on my part. Writing a script that can automatically and quickly setup and teardown against the final database, and having your team members set up a local database, may or may not work for you (it'll depend on the nature of your team and your system).

Ultimately, the most important thing is that you have automated tests which aren't a nightmare to setup, maintain or run. Integration tests have more dependency and thus are more fragile, but can be an efficient way to verify correctness.




More...