You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+9-9Lines changed: 9 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,7 +15,7 @@
15
15
16
16
At PeerDB, we are building a fast, simple and the most cost effective way to stream data from Postgres to Data Warehouses, Queues and Storage engines. If you are running Postgres at the heart of your data-stack and move data at scale from Postgres to any of the above targets, PeerDB can provide value.
17
17
18
-
We support different modes of streaming - log based (CDC), cursor based (timestamp or integer) and XMIN based. Performance wise, we are 10x faster than existing tools. Features wise, we support native Postgres features such as comprehensive set of data-types incl. jsonb/arrays/geospatial, efficiently streaming toast columns, schema changes and so on.
18
+
We support different modes of streaming - log based (CDC), cursor based (timestamp or integer) and XMIN based. Performance wise, we are 10x faster than existing tools. Features wise, we support native Postgres features such as comprehensive set of data-types incl. jsonb/arrays/geospatial, efficiently streaming TOAST columns, schema changes and so on.
19
19
20
20
## Get started
21
21
@@ -61,15 +61,15 @@ Current data tools prioritize a wide range of connectors, often neglecting to op
61
61
62
62
PeerDB is an ETL/ELT tool built for PostgreSQL. We implement multiple Postgres native and infrastructural optimizations to provide a fast, reliable and a feature-rich experience for moving data in/out of PostgreSQL.
63
63
64
-
**For performance** - we can parallelize initial load for a large table, still ensuring consistency. Syncing 100s of GB reduces from days to minutes. Our architecture is designed for real-time syncs and implements multiple logical replication related optimizations (tuning Postgres configs, parallel reading of slot etc.). This enables 10x faster Change Data Capture with data-freshness of a few 10s of seconds even at large throughputs (10k+ tps).
64
+
**For performance** - we can parallelize initial load for a large table, still ensuring consistency. Syncing 100s of GB reduces from days to minutes. Our architecture is designed for real-time syncs and implements multiple logical replication related optimizations (e.g., tuning Postgres configs, parallel reading of slot). This enables 10x faster Change Data Capture with data-freshness of a few 10s of seconds even at large throughputs (10k+ tps).
65
65
66
-
**For reliability**, we have mechanisms in place for fault tolerance - state management, automatic retries, handling idempotency and consistency and so on (<https://blog.peerdb.io/using-temporal-to-scale-data-synchronization-at-peerdb>) Configurable batching and parallelism prevent out of memory (OOMs) and crashes.
66
+
**For reliability**, we have mechanisms in place for fault tolerance - state management, automatic retries, handling idempotency and consistency and so on (<https://blog.peerdb.io/using-temporal-to-scale-data-synchronization-at-peerdb>). Configurable batching and parallelism prevent out of memory (OOMs) and crashes.
67
67
68
-
**From a feature richness standpoint**, we support efficient syncing of tables with large (TOAST) columns. We support multiple streaming modes - Log based (CDC) based, Query based streaming etc. We provide rich data-type mapping and plan to support every possible (incl. Custom types) that Postgres supports to the best extent possible on the target data-store.
68
+
**From a feature richness standpoint**, we support efficient syncing of tables with large (TOAST) columns. We support multiple streaming modes - log based (CDC), cursor based (timestamp or integer) and XMIN based. We provide rich data-type mapping and plan to support every possible type (incl. Custom types) that Postgres supports to the best extent possible on the target data-store.
69
69
70
-
### Now available natively in ClickHouse Cloud (Private Preview)
70
+
### Now available natively in ClickHouse Cloud (Generally Available)
71
71
72
-
PeerDB is now available natively in ClickHouse Cloud (Public Preview). Learn more about it [here](https://clickhouse.com/cloud/clickpipes/postgres-cdc-connector).
72
+
PeerDB is now available natively in ClickHouse Cloud (Generally Available). Learn more about it [here](https://clickhouse.com/cloud/clickpipes/postgres-cdc-connector).
@@ -79,13 +79,13 @@ PeerDB is now available natively in ClickHouse Cloud (Public Preview). Learn mor
79
79
80
80
The Postgres-compatible SQL interface for ETL is unique to PeerDB and enables you to operate in a language you are familiar with. You can do ETL the same way you work with your databases.
81
81
82
-
You can use Postgres’ eco-system to manage your ETL —
82
+
You can use Postgres’ ecosystem to manage your ETL —
83
83
84
84
1. Client tools like pgAdmin, psql to run SQL commands.
85
85
2. BI tools like Grafana, Tableau to visually monitor syncs and transforms.
86
86
3. Database migration and versioning tools like Flyway to manage your ETL.
87
-
4. Any language (Python, Go, Node.js etc) and Scheduler (AirFlow) for development.
88
-
5. And many more
87
+
4. Any language (e.g., Python, Go, Node.js) and Scheduler (Airflow) for development.
0 commit comments