Unit Testing ... Where to Start?
Before proceeding: many thanks to Joe Masilotti for so graciously reviewing this post and providing great feedback. Thanks, Joe! 💯 👍
The importance of testing our code cannot be overstated. However, this blog entry isn't about why unit testing is important. Instead, I'm hoping to share what I’ve learned regarding questions even a little more fundamental, such as how a unit of work is defined, and where and how we should add tests to an existing codebase.
- Part 1 will focus on exploring some of the basic concepts and terminology, such as: “What is a unit of work? What are stubs and mocks, and where are they used?”
Part 2 will focus on where and how to test our code. Given an existing codebase, how do prioritize and begin testing?
Part 3 will walk through testing of a small example app to showcase the concepts presented in Parts 1 and 2.
I'm sharing my own learnings in the hopes of: (1) solidifying my own understanding, and (2) helping others who might also be looking to come to better grips with this topic.
Caveat: There's a wealth of expertise, knowledge and opinion on this subject, and the insights presented here aren't intended to be definitive or exhaustive. I absolutely welcome clarifications, suggestions and corrections from the community - please feel free to leave a note in the comments or give me a shout on Twitter. I consider this a “living blog post" that will be updated over time as my own understanding evolves.
But for now, enjoy and let's get started, shall we? :)
- First and foremost, I couldn't more highly recommend Roy Osherove's book, The Art of Unit Testing, which helped me immensely. Roy's examples are written in C#, but because C# and Swift are both C-style languages, their overall syntax and structure are just similar enough, albeit broadly, to make Roy’s examples easy enough to follow. If you don't have a copy - please get one! It’s a worthy investment.
- Cate Huston's Unit Testing: Beyond the Model (written in Objective-C) is an excellent, to-the-point, iOS-specific resource.
- Major props to Joe Masilotti’s excellent series of blog posts on unit and UI-testing in Swift. Joe has written (and continues to write) many excellent articles on the subject of unit testing in Swift, although Better Unit Testing with Swift is a great place to start.
More so than any other definition I've come across, Roy Osherove’s definition of “a unit of work” really helps establish a concrete conceptual foundation from which to jump into the subject.
A unit of work is the sum of actions that take place between the invocation of a public method in the system and a single noticeable end result by a test of that system. A noticeable end result can be observed without looking at the internal state of the system and only through its public APIs and behavior.
-- Excerpt From: Roy Osherove. “The Art of Unit Testing, Second Edition: with examples in C#.”
In Swift-land, this could be interpreted to mean that - provided our code is well-written and we expose methods and properties only as necessary - our concern is to unit-test public-ly or internal-ly-visible methods and properties, not private declarations that aren't invoked outside of a class, struct or enum. Roy re-emphasizes this could mean that a unit can be as small as a single method or up to several classes and functions combined.
You better have good reason for exposing a piece of code outside its source file, because chances are you'll have to test it otherwise!
The above definition makes sense from a couple of different perspectives:
It allows us to refactor more confidently and efficiently by not having to worry about breaking our tests when we change the private details of an API. As long as the externally-visible interfaces of our APIs remain unchanged, our tests will inform us of whether or not the end result of unit of work is behaving as expected.
There is such a thing as “over-testing” a codebase to the point of being impractical. As Scott Norberg points out in this excellent article, “Code that is very highly tested is very tough to change. If unit tests are used excessively, software teams will find themselves considering the costs of changing the existing unit tests when changing the code, and these costs can spiral out of control.” Part of why I find Roy’s definition of a unit of work particularly appealing is it strikes a nice balance between practicality and thoroughness.
Roy’s definition also falls in line quite nicely with some best practices regarding documentation. For example, the Tumblr iOS team requires that, at minimum, all externally-visible properties and methods should be painstakingly documented. This is important because:
It helps our teammates (and our future selves!) understand the APIs we're writing, and
This makes us more more partial to declaring things as private unless absolutely necessary.
All that to say, you better have good reason for exposing a piece of code outside its source file, because chances are you'll have to test it otherwise!
SO TEST ALL THE PUBLIC STUFF? UMM ... NO.
As a general principle, pieces of code without any logic, for example a simple variable with its default getter and setter, don't require testing in and of themselves. And, as Cate Huston points out, we really should never have to test our data structures:
We wouldn’t mock an NSArray or an NSDictionary, so why would we mock our own data structures? Data structures should be simple and well tested, so we can trust them to behave as they should. It will be more work to mock them than just to use them.
iOS Unit Testing: Beyond the Model (Cate Huston)
However, immediately we add any sort of logic or control-flow code when reading from or writing to a variable, the variable becomes a candidate for unit-testing. (In Swift, one example might be a variable whose get or set accessors are overridden with some sort of conditional logic.)
sOME TERMINOLOGY WORTH SHARING
Dependency: An external input to the unit of work used to perform the method’s task.
Regression: When a component of your app used to work, and then doesn't. This is termed regressing to a less-desired (or broken!) state.
Mock: We mock a dependency in order to make assertions against the unit under test and verify that a piece of code is working as intended. For example, we mock a date input to verify that a method will print a given date in the expected format.
Stub: Continuing with the prior example, on our way to setting up the conditions necessary to assert against a mock date, we may have to fake one or more other dependencies. These tertiary dependencies are referred to as stubs.
An important point is that we test against mocks, not stubs! The terms “stub” and “mock” are often used interchangeably, but the distinction is worth keeping in mind.
UNIT VS INTEGRATION TESTS
An important feature of a unit test is that it should give us consistent results. This means that given the same inputs or dependencies, a test should provide the same outcome, no matter when or how frequently the test is run.
Integration tests are tests that use one or more real dependencies and therefore may not be consistent between trials. For example: testing a method with a date dependency. In Cocoaland, if we mocked a date input using NSDate() (i.e., the current date and time), the parameter supplied to the test would be different each time the test is run. Therefore, we are now in the realm of integrating real-world dependencies into our testing framework, hence the term "integration" testing. Integration testing is equally as important, but won't be the focus of this post.
we've got our definition down. now what?
Ideally, it's good practice to write your tests as you're writing your code. This helps reduce the build-up of technical debt and safeguards against regressions when we add to or refactor our code. To some, this means adding tests immediately after finalizing a piece of code. To others, this means writing tests before writing any code at all! (Loosely speaking, the latter falls within the camp of Test Driven Development or “TDD”.)
WRITING GOOD TESTS Vs when to write them
No matter which end of the "when-should-I-test" spectrum you fall into, Roy Osherove makes a good point of nurturing good testing practice by separating the skill of writing good tests from when to write them. To that end, I'm fostering my own development in this subject by first focusing on the core concepts and tools before worrying about where in the development cycle my tests should be applied. After all - it doesn't matter where in the development cycle I write my tests if said tests are badly-written, or simply aren’t useful. In almost any discipline, speed and efficiency are natural side effects of learning how to do things well first and foremost. Once we become confident in our ability to write good tests, we can more easily adapt to preferences regarding when exactly to insert unit tests into our code - and preferences or priorities regarding when (or if!) you write tests can change from one project to another.
Next: In Part 2, I'll explore some guiding principles for deciding where and how to add tests to an existing codebase. Stay tuned!