Code Reading: Shared Examples in Kiwi

Having examined the motivations for using shared examples and how they're implemented in RSpec, we're ready to examine how we can port the same functionality to Kiwi. This post takes approximately 7 minutes to read, and covers:

  1. How XCTest discovers instances of XCTestCase
  2. How Kiwi builds a suite of examples by hooking into this discovery
  3. How Kiwi implements shared examples

Readers familiar with using Kiwi to run Objective-C unit tests will get the most out of this post.

The Gist

Kiwi and RSpec are different beasts. RSpec is a pure testing framework that defines its own methods for test detection, while Kiwi runs alongside XCTest. Kiwi hooks into XCTest test detection to build example groups. Shared examples can be implemented just as in RSpec: by storing context nodes in a global registry and inserting them where itBehavesLike is used in the tests. A macro is used to ensure shared examples are registered before test execution.

Note that this post covers shared examples, a feature not yet in the official Kiwi release. See the pull request for information on when it might be available.

How XCTest discovers tests

XCTest is the framework behind the testing tools built into Xcode 5. Two classes relevant to this post are:

  1. XCTestCase: An individual test case.
  2. XCTestSuite: A collection of test cases.

If you've used XCTest before, you know that in order to implement test methods, you need to subclass XCTestCase and write methods that begin with -test. By default, XCTestSuite collects all subclasses of XCTestCase included in the test target. For each test method, it creates an instance of the test case and sends it the -invokeTest message. This executes the test case's -setUp method, then the test method, and finally the test case's -tearDown method.

XCTest is practically identical to SenTestingKit, the testing framework bundled with versions of Xcode 4.6 and prior–just replace XCTestCase with SenTestCase, and so on, in the explanations above. In posts from now on I'll only mention the points at which they differ.

How Kiwi hooks into XCTest test detection

Normally, XCTestSuite executes methods beginning with -test in each XCTestCase. Alternatively, subclasses of XCTestCase can override the class method +testInvocations to return a different set of methods to run during the test.

XCTestSuite adds all XCTestCase subclasses, then calls -invokeTest once for each of their +testInvocaiton invocations.

In fact, this is exactly what KWSpec does. When +testInvocations is called, each KWSpec subclass creates an instance of KWExampleSuite. In order to do so, KWSpec has the Kiwi DSL written between the SPEC_BEGIN and SPEC_END macros evaluated by a builder class called KWExampleSuiteBuilder. Functions like describe and context are C functions that add nodes–that is, blocks of code with a name parameter–to a stack maintained by the suite. Blocks passed to it functions are also turned into KWItNode objects and added to the stack.

When the example suite is finished constructing its stack of nodes, it returns an array of "dummy" invocations that match the number of KWItNode objects in the stack. Each of these invocations also carries a reference to the it node it represents, via a KWExample object.

When the -invokeTest method is called on each KWSpec, Kiwi grabs the KWExample associated with the current invocation being called. It then executes the KWItNode example by using the -[KWContextNode performExample:withBlock:] method. This method uses recursion to move up the context stack. The recursion exits when the entire context for the it node has been reconstructed, and the assertions in the it block are finally executed.

KWExample its KWItNode, which in turn traverses its parent contexts.

How shared examples are injected into the context stack

The previous post in this series described how RSpec shared examples are stored in a global registry. When RSpec encounters an #it_behaves_like method, it inserts the shared example as a context node in that spot.

Kiwi shared examples are implemented in the same way. The sharedExamplesFor method registers an instance of KWSharedExample in a global registry, an instance of KWSharedExampleRegistry. The itBehavesLike method inserts the shared example as a context block in the current stack.

Differences between RSpec and Kiwi implementations

Unfortunately, because Kiwi must adhere to the test execution flow imposed by XCTest, the implementation of shared examples isn't as clean as it is in RSpec. Whereas RSpec can load each spec file and process the shared examples in advance, Kiwi loads test case files in an order determined by XCTestSuite. As a result, special precautions must be taken to ensure that shared examples are registered before any example groups are executed.

The workaround for this limitation is to use the SHARED_EXAMPLES_BEGIN and SHARED_EXAMPLES_END macros. These define an arbitrary category and overload its +load method. This method is called when the class or category is added to the Objective-C runtime. This occurs before any tests are run, so it's an adequate place to register any shared examples:

SHARED_EXAMPLES_BEGIN(ReusableSpecHelpers)

sharedExamplesFor(@"something reusable", ^(Class describedClass) {
    // ...
});

SHARED_EXAMPLES_END

Of course, no one wants more macros in Kiwi. So it's worth considering whether there's a better way of ensuring shared examples are registered before any specs are run.

This concludes our exploration of shared examples in RSpec and Kiwi. Hopefully you got as much out of reading this series as I did writing it! If you have any feedback, please contact me on Twitter.

More on Kiwi