Skip to content

Commit d8abdb9

Browse files
authored
[DINO-53] Remove sync toggle from JSON pipeline creation UI (#2113)
* [DINO-53] Remove sync toggle from JSON pipeline creation UI * update previous change logs * update change log date * udpate notes * fix out of order * move change log to the botton
1 parent 5e67458 commit d8abdb9

File tree

1 file changed

+30
-6
lines changed

1 file changed

+30
-6
lines changed

pages/docs/data-pipelines/json-pipelines.mdx

Lines changed: 30 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -58,17 +58,15 @@ The following are examples of the output in **BigQuery** for each data source pi
5858

5959
## Events Data Sync
6060

61-
> Sync is disabled by default when creating events pipelines. You can enable it by toggling it in the **Advanced** session.
61+
Events Data Sync is enabled by default when creating JSON pipelines to ensure data consistency. This feature automatically detects data changes as soon as they are ingested and appends new files for new/late data to your storage/warehouses, helping keep the data fresh and minimizing missing data points.
6262

63-
Event data stored in Mixpanel’s datastore and in the export destination can fall out of sync.
64-
65-
The discrepancy can be attributed to several different causes:
63+
Event data can fall out of sync between Mixpanel's datastore and the export destination due to several causes:
6664

6765
- Late data can arrive multiple days later due to a mobile client being offline.
6866
- The import API can add data to previous days.
6967
- Delete requests related to GDPR can cause deletion of events and event properties.
7068

71-
Mixpanel is able to detect any changes in your data as soon as they are ingested and adds new files for new/late data in object storage and data warehouse, if applicable. Data sync helps keep the data fresh and minimizes missing data points.
69+
**Important limitations**: Data sync does not guarantee syncing GDPR Data Deletions. It is recommended to implement a strategy to remove all records of GDPR Deleted Users in your data warehouse.
7270

7371
## Backfill Historical Events
7472

@@ -128,4 +126,30 @@ The sync feature is designed for events to keep the exported data up-to-date wit
128126

129127
GDPR deletions do not automatically cascade deletions to data warehouses via pipelines. When a user is deleted from Mixpanel via the GDPR deletion API, this deletion is reflected in Mixpanel’s own storage, but the deletion does not propagate to data that has already been exported to data warehouses via pipelines.
130128

131-
To keep your synced warehouse data GDPR compliant, you will need to implement a process to delete the corresponding user and event data from your warehouse when a GDPR deletion occurs in Mixpanel.
129+
To keep your synced warehouse data GDPR compliant, you will need to implement a process to delete the corresponding user and event data from your warehouse when a GDPR deletion occurs in Mixpanel.
130+
131+
## Change Log
132+
133+
<details>
134+
<summary><strong>2025-09-29: Sync Enabled by Default for JSON Pipelines</strong></summary>
135+
136+
Removed sync toggle from JSON pipeline creation UI. All new JSON pipelines now have incremental sync enabled by default to ensure data consistency without requiring manual configuration.
137+
138+
**Impact**: New JSON pipelines automatically include sync functionality - no longer optional through UI toggle.
139+
</details>
140+
141+
<details>
142+
<summary><strong>2025-08-14: Incremental Export Deployed for US Pipelines</strong></summary>
143+
144+
We have deployed the Incremental Export Improvement for newly created pipelines with US data residency.
145+
146+
The rollout of these updates to existing pipelines in US residency is scheduled for the near future.
147+
Your pipeline will automatically transition to the new system when ready.
148+
Your data quality and completeness remain the same; only the processing method has improved.
149+
</details>
150+
151+
<details>
152+
<summary><strong>2025-06-26: Introducing Incremental Export for Event Pipelines</strong></summary>
153+
154+
We're rolling out an improved pipeline system to improve the efficiency and reliability of your data exports. We're deploying these improvements for newly created pipelines in our EU and IN data residency. New pipelines in projects with US residency and migrating existing pipelines in all regions will follow after. Your pipeline will automatically transition to the new system when ready. Your data quality and completeness remain the same - only the processing method has improved.
155+
</details>

0 commit comments

Comments
 (0)