Create "release" Redivis dataset

[mikabr]

In addition to the individual datasets, the data validator should populate a dataset with data from 1) datasets marked as "redivis release" in the entities airtable; 2) that was collected at least 6 months ago. Not sure what we want to call it, something like levante-release?

[ezhang61]

I can first name it as levante-release and include all the sites data until 6 months ago. My understanding is this "release dataset" would be manually triggered only, with no cron job triggered at a frequency, right?

[mcfrank]

Yes, that's correct - but we should have something like `levante-sites-release-2025.1` where options are `sites` or `alldata` and 2025.1 is a version number that increments within each year. Mike

…

On Thu, Sep 18, 2025 at 1:45 PM Ethan Zhang ***@***.***> wrote: *ezhang61* left a comment (levante-framework/data-validator#23) <#23 (comment)> I can first name it as levante-release and include all the sites data until 6 months ago. My understanding is this "release dataset" would be manually triggered only, with no cron job triggered at a frequency, right? — Reply to this email directly, view it on GitHub <#23 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAI25F5C72NIG4P2C4Y4HED3TMKXXAVCNFSM6AAAAACG43362WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMBZGY3DQMRXGE> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[ezhang61]

Does 'alldata' mode mean only date_filter would be applied (that would also include all the testing data within the date range), and 'sites' mode mean date_filter and site_filter would both be applied?

[mcfrank]

exactly! Message ID: ***@***.***

…

com>

[mikabr]

I think we should think through how including the version tag in the dataset name interacts with the redivis native versioning system. I think it might make sense to have a continuously updated private "pending release" dataset and then periodically snapshot it for actual public release – let's discuss

[ezhang61]

Putting a note here: "alldata" mode may include all the test data, especially the sites and cohorts.

[ezhang61]

district table name as site, group->cohort
add district id in group tbl
user_assignment
check user_class

want data before 7.1.2025 COs
DE CA no date range