This python package provides access to a set of tables loaded from parquet. These are listed in https://github.com/ImagingDataCommons/idc-index/blob/main/idc_index/index.py#L128. Two of the tables, index and prior_versions_index, are included in the package and are installed automatically. The remaining indices are hard-coded. We need to switch from the hard-coded list of indices to automatic discovery of those tables based on the attachments for the idc-index-data dependency. Those tables are available as release attachments in parquet files, and are accessible at https://github.com/ImagingDataCommons/idc-index-data/releases/tag/<idc-index-data version> (for example, see https://github.com/ImagingDataCommons/idc-index-data/releases/download/23.0.1/analysis_results_index.parquet). The list of available index tables should be automatically discovered during package initialization and used to populate indices_overview.

Descriptions of the individual index tables should be populated from the schema files that are available from idc-index-data along with the parquet files. For example, analysis_results_index.parquet will be accompanied by analysis_results_index.json sibling attachment at https://github.com/ImagingDataCommons/idc-index-data/releases/download/23.0.1/analysis_results_index.json. Table description is available in the table_description attribute. The entire schema for the individual tables should also be cached, and made available to the user via idc-index API.

Discovered index list should be cached, the function that populates it should have an option to override and refresh the cached content.