Add bulk insert feature #73
Description
in case if its required to add/insert hundreds (or even thousands) Parquet files it will be very useful to have ability to get list of Parquet files from input file. Adding "--from-file" argument for INSERT command. Something like this
/ # ice insert poc.dns_mtree_repl_v2_0_test -p --no-copy --force-no-copy --from-file foo.list
cat foo.list
s3://altiella-lrgbxme0-iceberg/poc/dns_mtree_repl_v2_0/external-data/eventDate=2025-10-15/retention=30/20251015-30_160_301_3_9063CCDB08A83B68D11B2B4B0433AB4A.parquet
s3://altiella-lrgbxme0-iceberg/poc/dns_mtree_repl_v2_0/external-data/eventDate=2025-10-15/retention=30/20251015-30_302_445_3_5AFF7CB76394E6891BE09D9B3E2E972D.parquet
s3://altiella-lrgbxme0-iceberg/poc/dns_mtree_repl_v2_0/external-data/eventDate=2025-10-15/retention=30/20251015-30_446_594_3_82899265B4702C84B76110518E450FF3.parquet
s3://altiella-lrgbxme0-iceberg/poc/dns_mtree_repl_v2_0/external-data/eventDate=2025-10-15/retention=30/20251015-30_595_725_3_54EF298EDE565C6F775DB03E3FF823D4.parquet
s3://altiella-lrgbxme0-iceberg/poc/dns_mtree_repl_v2_0/external-data/eventDate=2025-10-15/retention=30/20251015-30_726_859_3_639202D66D3EB4643AED5E2F469AF266.parquet
It will be important to return negative exit code if at least one Parquet file was not inserted successfully.
Metadata
Metadata
Assignees
Labels
No labels