Once you've setup the connection to a Data Source and created a tabular data asset, Monda enables you to both configure the refresh strategy and schedule, as well as monitor data refreshes for your data assets.
Refresh Strategies
Monda currently supports both full and incremental data refreshes.
Strategy | Explanation | Support |
Full refresh | Refresh the entire dataset. While this ensures the table on Monda represents the source dataset 100%, it's of course computationally intense and not performant for frequent refreshes. | ✅ |
Incremental append-only | Fully support this, based on | ✅ |
Incremental upsert | We do not support incremental upsert data refreshes currently. Reach out to us if you need this. | ❌ |
💡 Are you looking for supporting a currently unsupported refresh strategy? Reach out to us to discuss this.
Refresh Status
Every data refresh is monitored and categorized into one of four states:
Status | Description |
Processing | The refresh is currently running and fetching the latest data. No action is required while the system processes the update. |
Refreshed Successfully | The update completed without errors. The product data is current and ready for use. |
Refresh Cancelled | The update was manually stopped by a user or terminated by a system override before it could finish. |
Refresh Failed | The update encountered an error (e.g., network timeout, credential issue, or corrupted data) and stopped. |
Refresh Triggers
Data refreshes can be triggered in three different ways:
Trigger | Description |
⏰ Scheduled refresh triggers | You can configure automatic refreshes with schedules that are daily, weekly, or monthly. In addition, you can also set the refresh time, in order to trigger the refreshes on a predictable schedule. |
⚙️ Programmatic API refresh triggers | By using the API, you can programmatically trigger data refreshes, e.g. when you have a regular data processing job running on your end and you want to trigger a refresh as soon as the job has finished. |
👤 Manual ad-hoc UI refresh triggers | In the UI, you can also manually trigger a data refresh job. This is especially helpful when first setting up a table or when debugging data refreshes. |
When does a full data refresh happen?
A full data refresh happens under four scenarios: manual triggers, specific product configurations, structural edits, or errors that require you to run a full refresh.
Explicit user triggers
You intentionally request a complete data rebuild by:
Clicking "Full Refresh" in the Provider Portal.
Calling Admin API endpoints with
force_full_refresh = true.Force-refreshing a subscription, which clears existing data and rebuilds it from scratch.
"Full refresh only" configurations
The product is configured to never use incremental refreshes by:
Turning off "Incremental refresh on the product.
Not setting an "Updated At" column on the product.
Being a File Only Data Product. This type of data product supports only full refreshes.
Schema or settings modifications
Altering the structure of a product forces the very next refresh to automatically be a full refresh to align the data.
Column edits: Including/excluding columns, changing visibility, altering data types or datetime formats, renaming columns, or clicking "Reset Columns".
Partition edits: Changing the designated partition column or its aggregation interval (e.g., daily to monthly).
Customization edits: Adding, removing, or changing row filters, or modifying any customization options on the product (or its child products).
Error resolutions requiring manual intervention
When data drifts or structural mismatches occur, incremental syncs will fail. Monda cannot fix these automatically and you must manually trigger a full refresh to resolve the following errors.
⚠️ Error | 💡Cause |
Historical data may have been updated, please run a full refresh on the product. | Rows were deleted or backdated in the source. Because incremental syncs only look for new rows, they miss updates to historical records. |
Row count mismatch, this may be due to incremental settings. Please run a full refresh on the product. | The row-count verification step failed during the sync process. |
Row count from the product does not match row count on the subscription. Please run a full refresh on the subscription. | Delivered data and product data have drifted apart, usually due to a missed incremental sync window |
Column mismatch between source and destination tables during incremental refresh. Please trigger a column reset... | The source schema changed in a way that the destination table cannot handle as a simple append. Triggering a column reset fixes this and forces a full refresh as a side effect |