Incremental full refresh on UC managed Iceberg drops the table instead of create-or-replace (v1 materialization)

[cemsbr]

Describe the bug

Running dbt build --full-refresh on an incremental model that materializes as a Unity Catalog managed Iceberg table (use_managed_iceberg: true behavior flag + table_format: iceberg, no use_materialization_v2) executes drop table if exists followed by create or replace table ... as select, instead of a single atomic create or replace.

Because the drop happens first, the table does not exist for the entire duration of the CTAS — several minutes on large tables. Concurrent queries fail with TABLE_OR_VIEW_NOT_FOUND, and the table's history/time travel is lost. Grants survive only when they are inherited from the schema.

This looks like the v1-materialization / managed-Iceberg sibling of #1205 (v2 + managed Delta, fixed by #1210): the same "is this relation replaceable?" check fails, just for a different reason.

Root cause (from reading the shipped macros)

In macros/materializations/incremental/incremental.sql (v1 path):

{% set is_delta = (catalog_relation.file_format == 'delta' and existing_relation.is_delta) %}
{% set is_iceberg = (catalog_relation.file_format == 'iceberg' and existing_relation.is_iceberg) %}
{% set is_replaceable_format = is_delta or is_iceberg %}

With managed Iceberg, catalog_relation.file_format is always 'delta': DatabricksCatalogRelation defaults file_format to delta, and iceberg_table_properties even raises DbtConfigError if the user sets anything else ("When table_format is 'iceberg', cannot set file_format to other than delta."). Meanwhile the existing relation's metadata reports Provider = iceberg, so existing_relation.is_delta is false.

Result: is_delta is false (existing table is not delta) and is_iceberg is false (file_format is delta) → is_replaceable_format = false → the macro takes the drop-then-create branch, even though CREATE OR REPLACE TABLE works fine on managed Iceberg (it is exactly the statement the macro runs right after the drop).

The v2 path has the same shape (is_replaceable = existing_relation.can_be_replaced and is_replaceable_format), so it should be affected too.

Steps To Reproduce

dbt_project.yml:

flags:
  use_managed_iceberg: true
models:
  my_project:
    +table_format: iceberg

Model:

{{ config(materialized='incremental', incremental_strategy='merge', unique_key='id') }}
select 1 as id

1. dbt run (creates the managed Iceberg table)
2. dbt run --full-refresh --log-level debug
3. The debug log shows Applying DROP to: ... followed by drop table if exists ..., then the create or replace table ... using iceberg ... as select.

Expected behavior

A single create or replace table ... as select, with no drop — same as delta-on-delta full refreshes — so the table stays continuously queryable and keeps its identity.

Screenshots and log output

Sanitized debug log from a production-sized run (dbt-core 1.11.11, dbt-databricks 1.12.1, serverless SQL warehouse, UC managed Iceberg table that was itself created by dbt with this same config):

16:24:00.449339 [debug] MATERIALIZING INCREMENTAL
16:24:00.526970 [debug] Applying DROP to: `catalog`.`gold`.`my_incremental_model`
16:24:00.5      [debug] On model.my_project.my_incremental_model:
                        drop table if exists `catalog`.`gold`.`my_incremental_model`
                        ... followed by:
                        create or replace table `catalog`.`gold`.`my_incremental_model`
                        using iceberg ... as select ...
16:27:40.321475 [info ] 1 of 16 OK created sql incremental model gold.my_incremental_model [OK in 220.08s]

The table did not exist between 16:24:00 and 16:27:40 (~3m40s); queries against it during that window failed with TABLE_OR_VIEW_NOT_FOUND.

System information

The output of dbt --version:

Core:
  - installed: 1.11.11
Plugins:
  - databricks: 1.12.1

(Changelog for 1.12.2–1.12.5 does not mention this area, so latest appears affected as well.)

The operating system you're using: Linux

The output of python --version: Python 3.13.14

Additional context

• Related: materialization_v2: Incremental full refreshes now drop and recreate tables #1205 (same replaceability check failing for managed tables on the v2 path with Delta; fixed by fix: Use create or replace rather than drop/create for in Mat V2 on full refresh #1210). This report is the v1 path with managed Iceberg.
• Workaround we are using for planned full refreshes: compile the model and run the generated create or replace table ... as select manually, skipping the drop.

[cemsbr]

While validating #1669 on dbt-databricks 1.12.5 I found the same drop-then-create in the table materialization, and there it happens on every run, not only on --full-refresh.

materializations/table.sql gates the create or replace path (v2 branch and v1 branch alike) on:

existing_relation.can_be_replaced and adapter.resolve_file_format(config) in ('delta', 'iceberg')

With the use_managed_iceberg behavior flag on and table_format: iceberg, resolve_file_format returns parquet (impl.py, resolve_file_format), so the condition is always false: dbt runs DROP TABLE IF EXISTS and then a CREATE OR REPLACE TABLE ... USING ICEBERG CTAS that would have been atomic on its own. Log excerpt (UC managed Iceberg table, ~1M rows):

MATERIALIZING TABLE
Applying DROP to: `<catalog>`.`<schema>`.`<table>`        (6 ms later)
drop table if exists `<catalog>`.`<schema>`.`<table>`
create or replace table `<catalog>`.`<schema>`.`<table>` ... using iceberg ...
OK created sql table model <schema>.<table> [OK in 122.05s]

The table is missing for the whole CTAS (~2 minutes here, on every hourly rebuild), and concurrent readers get TABLE_OR_VIEW_NOT_FOUND. Every table model on managed Iceberg in a project is affected on every dbt run/dbt build.

existing_relation.is_iceberg is already True in this path (the relation cache fills Provider from information_schema.tables.data_source_format, and can_be_replaced has an is_iceberg arm), so the predicate introduced by #1669 (format_allows_create_or_replace) is exactly what table.sql needs as well. materializations/seeds/helpers.sql (databricks__reset_csv_table) has the same gate.

I have a draft PR stacked on #1669 that reuses that macro in both branches of table.sql: #1674. In our project we are running the same predicate as a root-project override of the table materialization as a workaround until the fix ships.