Skip to content

Core: Validate implementation classes loaded from properties - #3951

Open
akashchamp wants to merge 1 commit into
apache:mainfrom
akashchamp:fix/validate-implementation-types
Open

akashchamp wants to merge 1 commit into
apache:mainfrom
akashchamp:fix/validate-implementation-types

Conversation

@akashchamp

Copy link
Copy Markdown

Closes #3932

Rationale for this change

Table properties can name implementation classes that are imported and instantiated. Those values can come from table metadata, so loading an arbitrary importable class can pass table properties to an unrelated constructor. Each loader now verifies the expected base class before instantiation.

Are these changes tested?

  • Seven focused regression tests and the three affected unit-test files: 232 passed, 7 skipped, 52 deselected.
  • make lint passed.
  • Manual property-driven validation loaded valid custom FileIO, LocationProvider, and S3RetryStrategy implementations, then confirmed that a wrong-but-importable class was rejected in each path.
  • make test completed with 4,105 passed, 5 skipped, and 126 deselected. Three unrelated Hive Kerberos tests failed because this host lacks krb5-config and the native Kerberos module.

Are there any user-facing changes?

Yes. Invalid py-io-impl, write.py-location-provider.impl, and s3.retry-strategy-impl values now raise a clear ValueError before an unrelated importable class is instantiated. Valid custom implementations continue to load normally.

@rambleraptor rambleraptor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did some grepping around looking for other places in the codebase where this pattern occurs (taking some form of outside text and attempting a python import based off of it)

The other two places are in catalog/__init__.py for importing a catalog and for AuthManagers.

I don't see any harm is doing a similar check there. Importing a Catalog should conform with the Catalog class and AuthManager should conform with the AuthManager class.

I know they're not mentioned in the bug, but do you mind doing those? (I'm still approving since this looks great as-is)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implementation classes named in table properties are imported and instantiated without a type check

2 participants