Secondary to system testing, it can be useful to unit test a topology. With good system test coverage this may not be necessary. If it is needed, then Creek can help.
The creek-kafka-streams-test library provides topology test helpers:
can be used to initialise Creek without a real Kafka cluster to talk to, and
TestTopics provides some factory
methods for creating the test topic instances used in the unit test.
Add a unit test
The aggregate template provided a shell
To add a basic unit test, add the code below to the class:
The above adds a single
shouldOutputHandleOccurrences unit test for our simple topology,
which produces a single record to the topologies input topic, and asserts the output is correct.
The eagle-eyed of you may have noticed an existing test at the bottom of
The intent of this test is to detect any unintentional changes to the Kafka Streams topology.
Warning: There are certain categories of topology changes that are not backwards compatible with earlier versions of a deployed service, e.g. those that change topic names. Creek recommends always naming operators in the Kafka Streams DSL to minimise the chance of unintentional changes to internal topics. See the Kafka Streams docs for more information.
The test compares the topology with the last know topology and fails if they differ.
If the change is intentional, then the
file can be updated to reflect the latest topology. For this tutorial, the file should be updated to contain:
Source: ingest-twitter.tweet.text (topics: [twitter.tweet.text])
Processor: extract-handles (stores: )
Sink: egress-twitter.handle.usage (topic: twitter.handle.usage)
If you find this test more of a hindrance than a help… delete it!
As before, test coverage can be calculated by running the following Gradle command:
This will execute the unit and system tests and use JaCoCo to calculate the test coverage.
The human-readable coverage report is saved at
In this case, the test coverage hasn’t improved by adding unit tests. Meaning, arguably, this test is superfluous.
While Creek recommends using system tests for functional testing, more complex real-world solutions often still benefit from unit testing of the topology, to cover branches that are hard to reach with system tests.
ProTip: The repository also has a Gradle task and GitHub workflow step to upload the coverage report to coveralls.io, should this be something you need, or the project can be customised to publish coverage reports elsewhere.