Pingu
Computer MySQL PostgreSQL Books Publications
Spielereien Kanu Geopolitik Business TopoDB POI Klettersteigen History TransPool Thermal Baden Brokenstuben Goldwaschen
Blog Contact
Shinguz
Google
/ch/open

PostgreSQL Logical Replication

/ home / computer / postgresql / postgresql for mysql admins / .

Table of Contents

Details
  1. Concept
  2. Setting up the Publisher
    • Create role on Publisher
      • Create role on Publisher for whole instance/cluster
      • Create role on Publisher for one whole database
      • Create role on Publisher for one whole schema in one database
      • Create role on Publisher for several tables in one database
    • Publish tables
      • Publish all tables in the database
      • Publish all tables in a schema
      • Publish some tables
    • Copy tables
      • Copy all tables in the database to Subscriber
      • Copy all tables in a schema to Subscriber
      • Copy some tables to Subscriber
  3. Setting up a Subscriber
  4. Operations
  5. Monitoring
  6. Limitations
  7. Fail-over
  8. Advanced topics
    • Replication filtering

Concept

Logical Replication uses a:

  • Publish and Subscribe model.
  • One or more subscribers, subscribing to one or more publications on a publisher node.
  • Subscribers pull data for the publications they subscribe to.
  • Cascading replication or more complex configurations are possible.

Use-cases

  • Sending incremental changes in a single database or a subset of a database to subscribers as they occur.
  • Firing triggers for individual changes as they arrive on the subscriber.
  • Consolidating multiple databases into a single one.
  • Replicating between different major versions.
  • Replicating between different platforms (Linxu to Windows, 32-bit to 64-bit, little-endian to big-endian).
  • Giving access to replicated data to different groups of users.
  • Sharing a subset of the database between multiple databases. (multi-source?)

Publisher / Publication

  • The node where a publication is defined is referred to as publisher (change set or replication set).
  • Each publication exists in only one database.
  • Every publication can have multiple subscribers.
  • A published table must have a replica identity configured. By default, this is the primary key. Another unique index can also be set to be the replica identity.
    TODO: How to find without replica identity, how to find tables without a pk?
  • If the table does not have any suitable key: replica identiy FULL. Can lead to Full-Table-Scan? on the Subscriber side. Only as a fallback if no other solution is possible.

Subscriber / Subscription

  • The node where a subscription is defined is referred to as the subscriber.
  • A subscriber node may have multiple subscriptions if desired.
  • It is possible to define multiple subscriptions between a single publisher-subscriber pair.
  • Each subscription will receive changes via one replication slot.
  • A logical replication subscription can be a standby for synchronous replication.
  • Subscriptions are dumped by pg_dump.

Setting up the Publisher

We consider different scenarios here:

  1. Replicate the whole instance/cluster
  2. Replicate one whole database
  3. Replicate one whole schema in one database
  4. Replicate serveral tables in one database

Required settings for logical replication on Publisher:

#
# postgresql.conf
#
wal_level             = logical
max_replication_slots = 10   # default
max_wal_senders       = 10   # default

postgres=# show wal_level;
 wal_level 
-----------
 logical
postgres=# CREATE ROLE replication WITH REPLICATION PASSWORD 'secret';


postgres=# SELECT rolname AS role, rolsuper AS super, rolcanlogin AS login, rolreplication AS replication
  FROM pg_roles
 WHERE rolsuper = true or rolreplication = true
;
     role     | super | login | replication 
--------------+-------+-------+-------------
 postgres     | t     | t     | t
 replication  | f     | t     | t
#
# pg_hba.conf
#
# TYPE  DATABASE        USER            ADDRESS                 METHOD
host    all             replication     10.223.125.0/24         scram-sha-256

postgres=# SELECT pg_reload_conf();

postgres=# SELECT user_name AS role, address, netmask, database
  FROM pg_hba_file_rules
 WHERE type != 'local' AND address NOT IN ('127.0.0.1', '::1')
;
     role      |   address    |    netmask    | database 
---------------+--------------+---------------+----------
 {replication} | 10.223.125.0 | 255.255.255.0 | {all}
 {app}         | 10.223.125.0 | 255.255.255.0 | {test}

Create role on Publisher

TODO: Show all privileges of a row: https://dba.stackexchange.com/questions/315091/how-can-i-list-all-privileges-that-a-postgresql-role-has https://stackoverflow.com/questions/40759177/postgresql-show-all-the-privileges-for-a-concrete-user

Create role on Publisher for whole instance/cluster

postgres=# SELECT datname FROM pg_database WHERE datistemplate != true;
 datname  
----------
 prod
 qual

postgres=# \c prod
You are now connected to database "prod" as user "postgres".

prod=# SELECT nspname AS schema
  FROM pg_namespace
 WHERE nspname NOT IN ('pg_toast', 'pg_catalog', 'information_schema')
;
 schema  
---------
 erp
 sales
 finance

prod=# GRANT SELECT ON ALL TABLES IN SCHEMA erp TO replication;
prod=# GRANT SELECT ON ALL TABLES IN SCHEMA sales TO replication;
prod=# GRANT SELECT ON ALL TABLES IN SCHEMA finance TO replication;

prod=# \c qual
You are now connected to database "qual" as user "postgres".

qual=# SELECT nspname AS schema
  FROM pg_namespace
 WHERE nspname NOT IN ('pg_toast', 'pg_catalog', 'information_schema')
;
 schema  
---------
 erp
 sales
 finance

qual=# GRANT SELECT ON ALL TABLES IN SCHEMA erp TO replication;
qual=# GRANT SELECT ON ALL TABLES IN SCHEMA sales TO replication;
qual=# GRANT SELECT ON ALL TABLES IN SCHEMA finance TO replication;

Create role on Publisher for one whole database

prod=# SELECT nspname AS schema
  FROM pg_namespace
 WHERE nspname NOT IN ('pg_toast', 'pg_catalog', 'information_schema')
;
 schema  
---------
 erp
 sales
 finance

prod=# GRANT SELECT ON ALL TABLES IN SCHEMA erp TO replication;
prod=# GRANT SELECT ON ALL TABLES IN SCHEMA sales TO replication;
prod=# GRANT SELECT ON ALL TABLES IN SCHEMA finance TO replication;
-- GRANT pg_read_all_data TO replication;   -- for all schemas in database

Create role on Publisher for one whole schema in one database

prod=# SELECT nspname AS schema
  FROM pg_namespace
 WHERE nspname NOT IN ('pg_toast', 'pg_catalog', 'information_schema')
;
 schema  
---------
 erp
 sales
 finance

prod=# GRANT SELECT ON ALL TABLES IN SCHEMA erp TO replication;

Create role on Publisher for several tables in one database

TODO: Create tables per schema:

prod=# SELECT nspname AS schema
  FROM pg_namespace
 WHERE nspname NOT IN ('pg_toast', 'pg_catalog', 'information_schema')
;
 schema  
---------
 erp
 sales
 finance

prod=# GRANT SELECT ON TABLE table1, table2 TO replication;

Publish tables

Publish all tables in the database

Caution: a PUBLICATION is always per database!

postgres=# \connect test

test=# CREATE PUBLICATION db_test_publication FOR ALL TABLES;
CREATE PUBLICATION

test=# SELECT * FROM pg_publication;
  oid  |       pubname       | pubowner | puballtables | pubinsert | pubupdate | pubdelete | pubtruncate | pubviaroot | pubgencols 
-------+---------------------+----------+--------------+-----------+-----------+-----------+-------------+------------+------------
 16405 | db_test_publication |       10 | t            | t         | t         | t         | t           | f          | n

Publish all tables in a schema

-- CREATE PUBLICATION my_publication FOR TABLE table1, table2;

Publish some tables


Copy tables

Copy all tables in the database to Subscriber

$ pg_dump --host=<publisher-host> --user=postgres --schema-only --dbname=test > full_schema_test_dump.sql

postgres=# drop database test;
DROP DATABASE

postgres=# create database test;
CREATE DATABASE

$ psql --host=<subscriber-host> --user=postgres --dbname=test --file=full_schema_test_dump.sql

Copy all tables in a schema to Subscriber

The easiest way to copy a database/schema to a Subscriber ist the following:

$ # pg_dump --host=127.0.0. --user=postgres --schema-only --table=users --table=orders test > schema_test_dump.sql
$ # psql -h subscriber_host -U postgres -d mydb -f schema.sql

Copy some tables to Subscriber

Setting up a Subscriber

postgres=# \connect

test=# CREATE SUBSCRIPTION db_test_subscription
  CONNECTION 'host=10.223.125.65 port=5432 dbname=test user=replication password=secret'
  PUBLICATION db_test_publication;
CREATE SUBSCRIPTION

test=# \x
Expanded display is on.

test=# SELECT * FROM pg_subscription;
-[ RECORD 1 ]-------+--------------------------------------------------------------------------
oid                 | 24613
subdbid             | 24597
subskiplsn          | 0/0
subname             | db_test_subscription
subowner            | 10
subenabled          | t
subbinary           | f
substream           | p
subtwophasestate    | d
subdisableonerr     | f
subpasswordrequired | t
subrunasowner       | f
subfailover         | f
subconninfo         | host=10.223.125.65 port=5432 dbname=test user=replication password=secret
subslotname         | db_test_subscription
subsynccommit       | off
subpublications     | {db_test_publication}
suborigin           | any

During creation of a SUBSCRIPTION a snapshot of the tables is created! This can take a while…

postgres=# SHOW sync_replication_slots;
postgres=# SELECT pg_sync_replication_slots();

Managing the Publisher

postgres=# ALTER PUBLICATION my_publication ADD TABLE new_table;
postgres=# ALTER PUBLICATION my_publication DROP TABLE table1;

Managing the Subscriber

postgres=# ALTER SUBSCRIPTION my_subscription DISABLE;
postgres=# ALTER SUBSCRIPTION my_subscription ENABLE;

https://www.postgresql.org/docs/current/logical-replication-subscription.html#LOGICAL-REPLICATION-SUBSCRIPTION-EXAMPLES


Monitoring

On Subscriber:

test=# \x
Expanded display is on.

test=# SELECT * FROM pg_stat_subscription;
-[ RECORD 1 ]---------+------------------------------
subid                 | 24613
subname               | db_test_subscription
worker_type           | apply
pid                   | 6615
leader_pid            | 
relid                 | 
received_lsn          | 0/E00F8B0
last_msg_send_time    | 2026-08-03 20:17:41.945136+00
last_msg_receipt_time | 2026-08-03 20:17:41.94527+00
latest_end_lsn        | 0/E00F8B0
latest_end_time       | 2026-08-03 20:17:41.945136+00

Limitations

DDL, Sequences, …

Fail-over

Advanced topics

  • Replication filtering

Sources