Kafka Stream Backup

Elastio, as an automated data protection service, offers advanced backup solutions. For instance, the Elastio stream backup functionality can be integrated into Kafka service. The Elastio script creates a Kafka consumer with a unique consumer group ID and streams a Kafka topic to Elastio vault. The data is deduplicated, compressed, encrypted and catalogued as a recovery point for future use.

The Elastio script captures the first and the last message in the stream and stores both of them as recovery point tags. The script is agnostic to a message structure and captures raw data from the topic. During the next backup of the same topic the script identifies the last message offset by a recovery point tag and starts a new message stream from that offset. In this way, the script ensures that there are no duplicated backed up Kafka messages. This is crucial when it comes to restoring topic messages in case of any Kafka outage.

This tutorial will guide you through the process of creating a Kafka consumer to performing Elastio stream backup through Kafka, as well as provide some examples of your backups restore and automation.

---