feat: add an option to skip validate lance table

[hfutatzhanghb]

Motivation

Recently, we are testing lance-spark with hive2 or hive3 namespace. We met an problem as below:
suppose we have two tables: t1 and t2. t1 is a lance table and t2 is a normal hive table. It ocurred exception when we execute below SQL:

INSERT OVERWRITE TABLE t1
SELECT
      xxx
FROM t2;

The output exception is t2 is not lance format table. After refering to the code, we found it it blocked by method validateLanceTable.

Solution

We should better add an option to control whether we should skip validate lance table or not.

[github-actions]

ACTION NEEDED
Lance follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

For details on the error please inspect the "PR Title Check" action.

[hfutatzhanghb]

Hi, @jackye1995 @wojiaodoubao could you please help review this PR when you have free time? Thanks a lot.

[jackye1995]

“t2 is not lance format table” This seems like a valid error since the table property table_type=lance is required. How did you create it with the property missing?

[hfutatzhanghb]

“t2 is not lance format table” This seems like a valid error since the table property table_type=lance is required. How did you create it with the property missing?

Table t2 is a non-lance format table. We just attempt to load data from non-lance table and insert those data into lance table t1.

[jackye1995]

I think there are 2 problems here, (1) INSERT OVERWRITE is for Hive only, for Spark DSv2 you should use CREATE OR REPLACE TABLE AS SELECT. (2) lance sits in its own Spark catalog, so you need something like CREATE TABLE lance.default.t1 AS SELECT * FROM t2