Your Unit Tests Should Mind Their Own Business

Hacker News LinkedIn Facebook Twitter by Abby Fichtner

Good neighbors make good fences As the unit testing debates continue on my project, I can’t help but noticing that people are spending all sorts of time pontificating over the right way to unit test, without stepping back to consider what they’re trying to achieve with unit testing. And because they don’t know where they’re going, they’re not able to reach any conclusions on how to get there. Sound familiar?

There are actually a number of fabulous reasons for doing unit testing:

1. They help us think about our design in a way that we might not otherwise.
2. They lead to cleaner code because it’s often easier to re-factor our code to be more testable then it is to write unit tests that exercise obfuscated or otherwise hard to access code.
3. They validate that our modules are behaving correctly.
4. They provide a safety net to ensure our existing modules continue to function properly as our application evolves.

These aren’t all intuitive and so a lot of people will only consider #3 when they think about unit testing. And yet, what’s interesting is that #3 typically provides the smallest return on our unit testing investment because we tend to find WAY more defects through #1 and #2 as we work out how to develop our tests, and then of course in #4 as we continue to change our code over the application’s lifetime. And, unfortunately, by focusing only on #3, it can lead us to make some faulty decisions when we try to reason about the best way to do unit tests.

Private Methods

One of the most common questions people ask is whether to unit test private methods. I struggle to find any reasons why we should, but I do know a lot of reasons why we should not. I’m not saying that there’s never a reason to unit test private methods, I’m just saying to make sure you understand what that reason is before you go to all the effort.

In my opinion, the best way to achieve our unit test objectives is by validating our class’s interfaces. And when I think about interfaces, I’m thinking about the public methods we expose to other modules in our application. If we can validate that each of these methods is behaving as expected, then I’m going to feel satisfied that the class works correctly. For the purposes of unit testing, I don’t really care how the method achieves the result, as long as it’s the right one. I don’t care if the method figures out the result in-line, or by calling external methods or through internal private methods. If I call calculator.add( 2, 2 ) and get 4 then I’m happy. If I get 5, there’s a problem.

3 unit test scenarios for calculator.add() - which implementation is most correct?

In fact, if we’re being good developers and remembering to re-factor our code as the application evolves, then there’s a good chance that the method we use to determine that result is going to change over the application’s lifetime. That gets back to reason #4 for having unit tests – they provide a safety net to ensure that our modules continue to function as we make changes to them. If those unit tests are dependent on how we provide results, then it means that every time we decide to change our class’s inner workings, we have to update our unit tests. And if we have to rewrite our unit tests, then what is happening to our safety net?

“being able to quickly create, move around, and change the functionality of private methods is vital to remaining agile”
Charles Miller, The Fishbowl

Okay, sure if we change our public interfaces, we have to update our unit tests. But that makes sense. It means our interface specifications have changed and so our tests better be updated accordingly. Changing our public interfaces should incur a significant cost because other code relies on them. In fact, one might even argue that having high costs (such as having to update our unit tests) provides a good reminder to us that it’s in our best interest to keep our public interfaces stable.

On the other hand, the private parts of our implementations – those details that have been hidden behind our class’s interface – should not be costly to change. If we give them the high cost of having to redo our unit tests then it puts up a barrier against refactoring that we really don’t want. If we find that we’re using the same logic in 3 different places in our class, we should be able to easily move that logic into a single private method and then simply re-run our unit tests as they already exist to make sure our class continues to function as expected. We shouldn’t have to go write another unit test at this point because we haven’t added any new functionality. We’ve simply refactored our existing code to be more maintainable.

“Pure” Unit Tests with Mock Objects

Another big question people ask about unit testing revolves around how pure our unit tests need to be. Pure unit testing requires isolating an individual unit to validate its correctness.

So, for example, if our class interacts with external servers, we might create a stub to take the place of that server (such as a dummy server that simply listens on a port and returns the expected response). Our unit test can then validate our class using the stub rather than the real server. This enables us to validate that our class is doing its part correctly without depending on the external server acting right.

But what about if our class interacts with other classes? How do we factor them out? A popular idea is to use mock objects. Say we want to unit test our Training class, which contains a method enroll( Student ). Rather than passing it a real Student object, we could create a MockStudent object that implements the same interface as Student. We then tell MockStudent what methods we expect to be called, such as getFirstName() and getLastName(), and what to return for them.

Once we call Training.enroll( MockStudent), our mock object validates that every method we expected to be called was, indeed, called. We even have the option of requiring that these methods are called in a specific order. If any of the expected methods were not called, or if any unexpected methods were called, our test fails.

using mock objects in place of real ones during unit testing

 

Well, I really like the idea of mock objects – up until that last little bit. We seem to be back to the same concern that I have with unit testing private methods, and that is who cares which methods in Student are called, much less in what order? All I care about is that the student was successfully enrolled in the training. Maybe Training.enroll() was only using the student’s first and last name to set a unique ID and the developer later realized that it’s a bad idea to have data in our key, so updated the method to use a GUID instead. Should the Training.enroll() unit test now break because that method is no longer calling Student.getFirstName()? I don’t think so. And, again, I think we’re setting up unnecessary barriers to refactoring.

So, this isn’t meant as a “never use mock objects” anymore than a “never test private methods.” Just a recommendation to consider your unit testing objectives and what’s going to best serve them before blindly diving in to the latest cool thing you just read on some schmo’s blog.


See Also:
» The Flawed Theory behind Unit Testing (Michael Feathers)
» Testing Private Methods with JUnit and SuiteRunner (great stuff, even if you’re not using these tools)
» Mocks Aren’t Stubs (Martin Fowler)

  • Pingback: Agen Judi Terpercaya()

  • Pingback: Agen Judi Terpercaya()

  • Pingback: free porn videos()

  • Pingback: best free porn videos()

  • Pingback: free porn sex videos()

  • Pingback: แทงบอลออนไลน์()

  • Pingback: แทงบอลออนไลน์()

  • joshilewis

    With regards to private methods: if changing your private members results in a test breaking, you have violated the principle of encapsulation, which is a fundamental tenet of object orientation, and also broken the abstraction of the object.

    The meaning of the term 'refactor' means to modify the internal workings of a unit, such that its interface is not affected, and it still performs the same. Hence, refactoring a class in the  true sense should break no unit tests.

  • joshilewis

    I agree with Steve about mocks. Part of the value mocks provide to me, via behaviour-based testing (instead of state-based testing), is it makes system boundaries very explicit; i.e. the unit of code you're trying to test in isolation has very explicit boundaries (cross-reference your post on unit test purity).

    This has a lot of value when you're using layered architectures, such as domain models, repositories etc, or in something like MVC. Mocks allow you to make very explicit what the interaction between a controller and a repository looks like, i.e. the interface. When you're testing a controller, all you want to assert is that the controller calls a method on the repository (interface), with specific parameters. You don't care what the repository implementation would return. Mocks allow you to do this.This is design by contract.

    With regards to design by contract and mocks, most mocking (or isolation) frameworks only let you mock and interface, or virtual methods on a class. This almost forces you to design to an interface, and not an implementation, which is design by contract.

  • Yes, that’s exactly what we’ve been saying but the message got lost somewhere–which is why we’re going through the pain of writing a book. Take a look at the rest of it and tell us what you think.

    I believe I’ve got a lab of some sort accepted at Agile, so there should be an opportunity there.

  • Abby Fichtner

    Steve, this is really interesting. Thank you for posting this. Am I understanding correctly that you’re saying mock objects can actually be used (ala TDD style) to help “tease out” what we want the interfaces of our collaborating objects to even look like? That’s a very cool idea.

    I am sorry that they’re not doing your Pragmatic Mocking Workshop at Agile 2009 – I am really curious on this topic and would like to learn more. Can you recommend any other resources (or perhaps other chapters in your book) that would speak to this further)?

  • I don’t think you’ve really understood the intention of TDD with Mocks. The point is to design in terms of the roles objects play and how they relate. You shouldn’t be making assertions about the values sucked out of a neighbouring object, only on actions that you’ve asked them to perform for you. More detail at http://www.mockobjects.com/book/tdd-with-objects-intro.html

    …and I just can’t think of a reason to test a private method directly.

  • Abby Fichtner

    hah, your use of the word “interesting” gives you away as a Sixer, you know. ;-)

    I agree re: the private methods should always be instantiated by a public method. I could imagine, however, an instance where the private method abstracts away some yuckness that only has to be dealt with in 1% of cases so that the method can focus on the 99% part, and thus be easier to understand. In such a case, it may indeed be “difficult” to come up with a test case that executes the private method. I’m not sure why, however, this would represent a code smell…? (well, unless it’s that yuckness itself, hmmm)

    (okay, i swear, i’m going to do a post on code smells here soon)

  • It seems to me we only care about classes to the extent that they are used by each other (and the system in which they live) — otherwise a test of the class would be a system test). That is, a class is only interesting because some other class created or did something with it? So by extension, anything that results from a private method was initiated at some level by a public call?

    Or am I on crack?

  • Abby Fichtner

    I’m leaning towards 100% branch coverage in my testing…

    Interesting, how do you go about selecting tests that will give you 100% branch coverage? Do you find that you run into some of the same issues as using mocks or testing private methods directly in that the tests are now dependent on the internals of your code?

    Might it not be possible to have a bug in a private method that doesn’t immediately (or doesn’t always) show up by the public method that uses it?

    I’ve read that if you’re not able to easily validate your private methods through your public methods that this is a code smell and indicates that your code should be re-factored. But I must admit that I don’t understand this reasoning. Would be curious if anyone could elaborate on that…

  • But at the same time I’m leaning towards 100% branch coverage in my testing…

    Might it not be possible to have a bug in a private method that doesn’t immediately (or doesn’t always) show up by the public method that uses it?

    Darn, now I’m going to have to take another look at my code to see if I can test enough of the private methods via the public ones…

    Great post.

  • Abby Fichtner

    Thank you! That is my point exactly.

    Now why did I need all those words?

  • Anonymous

    Testing private methods?

    Bad mojo, because I think unit test should be focused on the framework/api… if you change the inner workings, you change them until the unit tests pass, right? That could mean changing the private methods, which is outside the scope of a unit test, if you ask me…