KafkaConnect Sink OpenLineage SMT

[djakupovic]

Feature Request / Improvement

Debezium introduced support for an OpenLineage Single Message Transform (SMT) in July 2025 to provide standardized data lineage tracking across streaming pipelines. OpenLineage is an open standard for capturing metadata about data movement, enabling better observability, governance, and impact analysis across platforms.
Given this, would it be possible to develop a similar lineage exporter for the Iceberg Kafka Connect plugin? Implementing OpenLineage support for Iceberg would allow organizations to trace data flows end-to-end—from change data capture to persistence—using a consistent, interoperable lineage framework.

Query engine

Kafka Connect

Willingness to contribute

• I can contribute this improvement/feature independently
• I would be willing to contribute this improvement/feature with guidance from the Iceberg community
• I cannot contribute this improvement/feature at this time

[danielcweeks]

@djakupovic Can you provide any more detail as to what this would look like? Links to the OpenLinage standard, link to the SMT documentation in Debezium?

It's a little difficult to understand what is involved here without more specific context.

[djakupovic]

@danielcweeks
Sure.
Openlineage: https://openlineage.io/docs

For example here is an implementation from factor house with kotlin:
https://github.com/factorhouse/factorhouse-local/blob/main/resources/plugins/smt/ol-kafka-connect-ext/src/main/kotlin/io/factorhouse/smt/OpenLineageLifecycleSmt.kt

OpenLineage is used for data lineage events. so e.g. we fetch data with debezium into Kafka. Debezium sends events to an backend like datahub for its jobs, insertions etc. afterwards we use iceberg s3 sink but we are blind because there is no OpenLineage implementation