-
Notifications
You must be signed in to change notification settings - Fork 68
Open
Description
This proposal would give a public API for reading and writing stream state for a Cache interface.
Signature might be something like:
class CacheBase:
...
def get_state(
stream_name: str,
) -> StreamState | None:
"""Return a stream state object for the provided stream name.
This is a thin wrapper around the internal `StateProvider` interface.
"""
...
def set_state(
stream_name: str,
stream_state: StreamState | dict,
) -> None:
"""Set a stream state object for the provided stream name.
This is a thin wrapper around the internal `StateWriter` interface.
"""
...
def migrate_state(
streams: list[str] | Literal["*"],
to_cache: CacheBase,
) -> None:
"""Copies all matching stream states to the specified `Cache` object.
This is a thin wrapper around the respective `Cache` objects'
`get_state` and `set_state` methods.
"""
...Other considerations:
- Since there are as of now many different strongly typed
Stateclasses (StateMessage,StreamState, etc.), we'd want to think carefully on which (if any) we feel comfortable to make a part of the public interface. - It might actually be cleaner to get/set values as
dictobjects, since that would fully avoid needing to expose a public API for the state object itself. (Tradeoff:dictobjects may be difficult to parse.) - It should be noted that manually modifying a
stateartifact for a stream basically always "voids the warranty", and sources and not guaranteed to have stablestateartifact interfaces over time. - This feature is actually more appropriate for migration of state - such as during renames or moving from one state backend to another, or from one table name/alias to a new one. For this reason, I've included a possible
migrate_state()method in the above which could streamline a "copy-all"-type operation. - Internally, states are often cached in memory by the
StateProviderclass. We'd need to take care to invalidate or refresh the caches after a "set" action.
Metadata
Metadata
Assignees
Labels
No labels