Skip navigation

Logical Decoding Protocol V2: Streaming Transactions, Schema Changes, Backfills and more

Brandon Mochama

Brandon Mochama Brandon Mochama

(Livestream 2)

My talk will examine the practical details of implementing a logical decoding consumer, drawing from my experience in building a client in Go using pglogrepl and from studying production CDC systems like PeerDB and Dolt. Since there are many talks, blog posts and docs that provide overviews, I want to specifically cover the non-obvious and undocumented tidbits on logical decoding that I encountered. Specifically:

  • Protocol V2 streaming semantics: handling interleaved transaction chunks, the StreamStart/StreamStop/StreamCommit message sequence, and why duplicate StreamAbort messages occur
  • TOAST column handling: the 'u' (unchanged) tuple type, backfill strategies, and the REPLICA IDENTITY tradeoffs different systems make
  • DDL & Schema change detection: reactive vs proactive, using RelationMessage deltas rather than DDL parsing, and how PeerDB propagates column additions to destinations
  • Initial data sync: leveraging the exported snapshot from slot creation for consistent backfills without race conditions
  • And more...

The talk assumes some familiarity with PG replication concepts and is aimed at developers building or maintaining CDC pipelines.

I have been writing about PostgreSQL on my blog for several years, this will be my first conference talk.

talk bubbles
Join the conversation

Use the hashtag #PosetteConf