Conversation
antarcticrainforest
left a comment
antarcticrainforest
left a comment
There was a problem hiding this comment.
I don't think that this repo is the right place for this sort of thing!
…ax age and max size of cache cleanup out of box
|
@antarcticrainforest now auto cache cleanup logic has been eliminated from prims and we only kept the manual cleanup here as an util to be used standalone by its own or other apps that are using prism. It seems to be ready to go to main |
|
when it's merged and pypi and conda packages are released, we will implement the auto cache cleanup in the opened PR in data-loader of freva-nextgen |
| return cache_root | ||
| # 2. User-defined storage option | ||
| if storage_options: | ||
| user_cache = storage_options.get("simplecache", {}).get("cache_storage") |
There was a problem hiding this comment.
Why do you need to call it simplecache? Is that something that's standard. Couldn't you call it something like prism-options or something else that signals that it's an option for this backend?
There was a problem hiding this comment.
This is standard fsspec convention, unlike rasterio which introduces its own engine-specific kwargs that users must learn separately, prism deliberately avoids adding new keywords and reuses conventions users already know from xarray/fsspec. In other word, user is using the base xarray storage_options, instead of defining new argument and asking them to read the documentation to learn how to work with prism and it keeps the usage of prism simple.
2 participants
This PR implements a cache cleanup strategy that removes caches either manually or automatically based on two policies: TTL and a size cap. By default, the TTL is 7 days and the size cap is 10 GB. Both limits can be increased via environment variables.