Waterstream Fleet Demo
Waterstream is a fully fledged MQTT broker leveraging Kafka as its only storage and distribution engine.
Every MQTT message sent by a device is immediately available on Kafka topics without the need of any 'man in the middle' connector. Likewise, devices can instantly receive produced messages on Kafka by subscribing to MQTT topics. Waterstream instances are stateless and therefore easy to scale out. This makes of Waterstream a light platform guaranteeing better throughput, reduced latency between Kafka and devices as well as less operational effort.
The Fleet Demo
In this demo, we simulate thousands of trucks driving around Italy, you can see fifteen of them on the map.
To follow one truck in its route, just click on its label. Sending MQTT messages to the Waterstream, each truck continuously communicates its current position, the next way-point and the speed.
No man in the middle
There's no Kafka connector or other integration technology between Waterstream and Apache Kafka. As soon messages are received by Waterstream, they are available in Apache Kafka topics.
This allows to integrate quickly an IoT platform with operational systems or the data analytics without any other system in the middle.
Unlike available Kafka connectors, Waterstream allows bidirectional communication. Data written into topics can be pushed to devices.
Waterstream exposes MQTT topics and metrics with Prometeus
It's easy to collect and show data coming from Waterstream with Grafana. In the following graph, we show the current message rate and how many messages were received so far by the Waterstream broker.
Explore and manage data easily with kSQL
Hence MQTT messages are stored directly in Kafka, you can aggregate them with any streaming technology like Kafka Streams or kSQL.
Below we built a real time graph that shows where trucks are headed using current position and waypoint. Graph data comes from a kSQL query that aggregates MQTT messages saved on a Kafka topic.
The following are the ksqlDB queries we used to aggregate MQTT messages in real time.
CREATE TABLE fleet_table( plate STRING, current STRUCT<lat DOUBLE, lng DOUBLE>, waypoint STRUCT<lat DOUBLE, lng DOUBLE>, speed DOUBLE, distance DOUBLE, updateTimestamp BIGINT) WITH (KAFKA_TOPIC='waterstream_fleet_demo', VALUE_FORMAT='JSON', KEY='plate');
CREATE TABLE fleet_directions AS SELECT plate, CASE WHEN waypoint->lat >= current->lat AND waypoint->lng >= current->lng THEN 'NE' WHEN waypoint->lat >= current->lat AND waypoint->lng < current->lng THEN 'NW' WHEN waypoint->lat < current->lat AND waypoint->lng <= current->lng THEN 'SW' ELSE 'SE' END AS direction FROM fleet_table EMIT CHANGES;
CREATE TABLE directions_vehicles AS SELECT direction, COUNT(*) as vehicles_count FROM fleet_directions GROUP BY direction EMIT CHANGES;
And these ksqlDB queries represent the aggregates in MQTT-friendly format:
CREATE STREAM directions_vehicles_stream (direction STRING, vehicles_count INTEGER) WITH (KAFKA_TOPIC='DIRECTIONS_VEHICLES', VALUE_FORMAT='JSON', KEY='direction');
CREATE STREAM directions_vehicles_mqtt AS SELECT direction, vehicles_count, 'waterstream-fleet-demo/direction-stats/' + direction AS mqtt_topic FROM directions_vehicles_stream PARTITION BY mqtt_topic;
Do you want to know more?
Visit Waterstream product site at https://waterstream.io an learn how you can combine MQTT with Kafka, the standard de-facto streaming platform.
Want to know more about Waterstream? Check out https://waterstream.io!