atoti.Table.load_kafka()#

Table.load_kafka(bootstrap_server, topic, *, group_id, batch_duration=datetime.timedelta(seconds=1), consumer_config={})#

Consume a Kafka topic and stream its records in the table.

Note

This method requires the atoti-kafka plugin.

The records’ key deserializer default to StringDeserializer.

The records’ message must be a JSON object with columns’ name as keys.

Parameters:

  • bootstrap_server (str) – host[:port] that the consumer should contact to bootstrap initial cluster metadata.

  • topic (str) – Topic to subscribe to.

  • group_id (str) – The name of the consumer group to join.

  • batch_duration (timedelta | int) – Time spent batching received events before publishing them to the table in a single transaction.

  • consumer_config (Mapping[str, str]) – Mapping containing optional parameters to set up the KafkaConsumer. The list of available params can be found here.