This step will add the business logic for the new service, in the form of a simple Kafka Streams topology.
As this is not the focus of this tutorial, this logic is kept deliberately trivial: it will filter the input topic, and produce the resulting records to the service’s output topic. The filter will exclude any records with Twitter handles not in a hardcoded list of accounts linked to presidents of the USA.
As a reminder, the input records store the Twitter handle in the record’s key and the number of occurrences in the record’s value. The service’s output topic will follow the same schema.
Define the stream topology
Add service module provided a shell
TopologyBuilder class in the new
Flesh out the class’s
build method to match what’s below:
ProTip: The example code deliberately names each step in the topology. This is good practice. Relying on default naming can result in topology evolution issues in the future. Internal store and topic names incorporate the step name. With default naming, the name of a step, and hence the store or topic, can change if steps are added or removed. This can lead to unintentional changes in internal topic names. If such a change was deployed, any unprocessed messages in the old topics would be skipped.
The above topology consumes the
HandleUsageStream defined in the service’s descriptor,
filters it using the
and produces any output to the
The Creek Kafka Streams extension provides type-safe access to the topic metadata & serde, and Kafka cluster properties, allowing engineers and the code to focus on the business logic.
The details of the
presidentsOnly method isn’t particularly important in the context of this tutorial.
A simple solution might look like this:
…and that’s the production code of the service complete!
Name instance defined in the
TopologyBuilder doesn’t add much in this example, but as topologies
get more complex, getting broken down into multiple builder classes, it really comes into its own.
Check out its JavaDoc to see how it can be used to help avoid topology node name clashes.
Topology builder unit test
Unit testing is not the focus of this tutorial. However, as it stands, the unit tests will fail. For completeness, this is addressed below.
Unit testing of Kafka Streams topologies is covered in more detail in the Basic Kafka Streams Tutorial .
Add service workflow added a new
TopologyBuilderTest for the new service’s topology.
This comes with a
shouldNotChangeTheTopologyUnintentionally test which, as it’s JavaDoc states, is there to capture
unintentional changes to the topology. Unintentional changes could introduce the possibility of data-loss, if
The test compares the topology with the last know topology and fails if they differ.
If the change is intentional, then the
file can be updated to reflect the latest topology.
For this tutorial, the test can simple be disabled or deleted.