Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 43 additions & 7 deletions docs/resources/api_integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,21 @@ description: >-
[Snowflake Documentation](https://docs.snowflake.com/en/sql-reference/sql/create-api-integration) | Snowcap CLI label: `api_integration`

Manages API integrations in Snowflake, allowing external services to interact with Snowflake resources securely.
This class supports creating, replacing, and checking the existence of API integrations with various configurations.
This class supports creating, replacing, and checking the existence of API integrations across multiple cloud providers
and Git HTTPS providers.

## Supported `api_provider` values

| `api_provider` | Required fields | Used for |
|---|---|---|
| `AWS_API_GATEWAY`, `AWS_PRIVATE_API_GATEWAY`, `AWS_GOV_API_GATEWAY`, `AWS_GOV_PRIVATE_API_GATEWAY` | `api_aws_role_arn` | External functions calling AWS API Gateway |
| `AZURE_API_MANAGEMENT` | `azure_tenant_id`, `azure_ad_application_id` | External functions calling Azure API Management |
| `GOOGLE_API_GATEWAY` | `google_audience` | External functions calling Google API Gateway |
| `GIT_HTTPS_API` | (none beyond `api_allowed_prefixes`) | Snowflake [Git repositories](git_repository.md) connecting to GitHub/GitLab/Bitbucket/etc. |

## Examples

### YAML
### YAML — AWS API Gateway

```yaml
api_integrations:
Expand All @@ -24,9 +33,19 @@ api_integrations:
api_allowed_prefixes: ["/prod/", "/dev/"]
api_blocked_prefixes: ["/test/"]
api_key: "ABCD1234"
comment: "Example API integration"
comment: "Example AWS API integration"
```

### YAML — GitHub (used by GitRepository)

```yaml
api_integrations:
- name: github_api_integration
api_provider: GIT_HTTPS_API
api_allowed_prefixes: ["https://github.com/some-org/"]
enabled: true
comment: "GitHub integration for git repos"
```

### Python

Expand All @@ -39,20 +58,37 @@ api_integration = APIIntegration(
api_allowed_prefixes=["/prod/", "/dev/"],
api_blocked_prefixes=["/test/"],
api_key="ABCD1234",
comment="Example API integration"
comment="Example API integration",
)
```


## Fields

* `name` (string, required) - The unique name of the API integration.
* `api_provider` (string or ApiProvider, required) - The provider of the API service. Defaults to AWS_API_GATEway.
* `api_aws_role_arn` (string, required) - The AWS IAM role ARN associated with the API integration.
* `api_key` (string) - The API key used for authentication.
* `api_provider` (string or ApiProvider, required) - The provider of the API service. See table above for supported values.
* `api_aws_role_arn` (string) - The AWS IAM role ARN. Required for AWS providers; omit for AZURE/GOOGLE/GIT_HTTPS_API.
* `azure_tenant_id` (string) - Azure AD tenant ID. Required for `AZURE_API_MANAGEMENT`.
* `azure_ad_application_id` (string) - Azure AD application registration ID. Required for `AZURE_API_MANAGEMENT`.
* `google_audience` (string) - GCP audience identifier. Required for `GOOGLE_API_GATEWAY`.
* `api_key` (string) - Optional API key used for authentication.
* `api_allowed_prefixes` (list) - The list of allowed prefixes for the API endpoints.
* `api_blocked_prefixes` (list) - The list of blocked prefixes for the API endpoints.
* `enabled` (bool, required) - Specifies if the API integration is enabled. Defaults to TRUE.
* `comment` (string) - A comment or description for the API integration.


## Granting on an integration

Snowflake's `GRANT USAGE ON INTEGRATION <name>` SQL is valid for any subtype. In YAML you may use either the
concrete subtype (`on: api integration <fqn>`) — preferred — or the generic umbrella (`on: integration <fqn>`):

```yaml
grants:
- priv: USAGE
on: api integration github_api_integration # preferred — explicit subtype
to: some_role
- priv: USAGE
on: integration github_api_integration # also supported (umbrella)
to: another_role
```
75 changes: 75 additions & 0 deletions docs/resources/git_repository.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
---
description: >-
A git repository in Snowflake.
---

# GitRepository

[Snowflake Documentation](https://docs.snowflake.com/en/sql-reference/sql/create-git-repository) | Snowcap CLI label: `git_repository`

A Git Repository in Snowflake represents an externally hosted Git repository (GitHub,
GitLab, Bitbucket, etc.) that has been registered for use with Snowflake's Git
integration. Once registered, files in the repository can be referenced via stage
syntax in `COPY`, `EXECUTE IMMEDIATE`, and other commands.

A git repository depends on an [APIIntegration](api_integration.md) whose
`api_allowed_prefixes` covers the repository's `origin` URL, and optionally on a
[Secret](generic_secret.md) (for private repos) referenced via `git_credentials`.


## Examples

### YAML

```yaml
git_repositories:
- name: some_git_repository
database: some_db
schema: some_schema
origin: https://github.com/some-org/some-repo.git
api_integration: some_api_integration
git_credentials: some_secret
comment: Example git repository
```


### Python

```python
git_repository = GitRepository(
name="some_git_repository",
database="some_db",
schema="some_schema",
origin="https://github.com/some-org/some-repo.git",
api_integration="some_api_integration",
git_credentials="some_secret",
comment="Example git repository",
)
```


## Fields

* `name` (string, required) - The name of the git repository.
* `origin` (string, required) - The URL of the externally hosted Git repository
(e.g., `https://github.com/some-org/some-repo.git`).
* `api_integration` (string, required) - The name of the API integration object
Snowflake will use to interact with the repository. The API integration's
`api_allowed_prefixes` must include the `origin` URL.
* `git_credentials` (string) - The name of a [Secret](generic_secret.md) holding
credentials for accessing a private repository. Optional for public repos.
* `comment` (string) - A comment for the git repository.
* `owner` (string or [Role](role.md)) - The owner role of the git repository.
Defaults to `SYSADMIN`.


## Grants

Snowcap supports `READ`, `WRITE`, and `OWNERSHIP` privileges on git repositories:

```yaml
grants:
- priv: READ
on: git repository some_db.some_schema.some_git_repository
to: some_role
```
2 changes: 2 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,8 @@ nav:
- AzureStorageIntegration: resources/azure_storage_integration.md
- GCSStorageIntegration: resources/gcs_storage_integration.md
- S3StorageIntegration: resources/s3storage_integration.md
- Git:
- GitRepository: resources/git_repository.md
- Orchestration:
- Alert: resources/alert.md
- Task: resources/task.md
Expand Down
75 changes: 72 additions & 3 deletions snowcap/data_provider.py
Original file line number Diff line number Diff line change
Expand Up @@ -1335,13 +1335,22 @@ def fetch_api_integration(session: SnowflakeConnection, fqn: FQN):
properties = _desc_type2_result_to_dict(desc_result, lower_properties=True)
owner = _fetch_owner(session, "INTEGRATION", fqn)

# Different api_provider values return different DESC properties:
# AWS_API_GATEWAY family -> api_aws_role_arn
# AZURE_API_MANAGEMENT -> azure_tenant_id, azure_ad_application_id
# GOOGLE_API_GATEWAY -> google_audience
# GIT_HTTPS_API -> none of the above
# Use .get() so missing fields fall back to None instead of crashing.
return {
"name": _quote_snowflake_identifier(data["name"]),
"api_provider": properties["api_provider"],
"api_aws_role_arn": properties["api_aws_role_arn"],
"api_aws_role_arn": properties.get("api_aws_role_arn") or None,
"azure_tenant_id": properties.get("azure_tenant_id") or None,
"azure_ad_application_id": properties.get("azure_ad_application_id") or None,
"google_audience": properties.get("google_audience") or None,
"enabled": properties["enabled"],
"api_allowed_prefixes": properties["api_allowed_prefixes"],
"api_blocked_prefixes": properties["api_blocked_prefixes"],
"api_allowed_prefixes": properties.get("api_allowed_prefixes"),
"api_blocked_prefixes": properties.get("api_blocked_prefixes"),
"owner": owner,
"comment": data["comment"] or None,
}
Expand Down Expand Up @@ -1595,6 +1604,34 @@ def fetch_event_table(session: SnowflakeConnection, fqn: FQN):
}


def fetch_integration(session: SnowflakeConnection, fqn: FQN):
"""
Fetch any integration regardless of concrete subtype.

Backs the generic `ResourceType.INTEGRATION` registered in RESOURCE_SCOPES so
that grants like `on: integration <fqn>` parse and resolve (Snowflake's
`GRANT USAGE ON INTEGRATION <name>` syntax accepts any subtype; this fetcher
returns the minimum SHOW INTEGRATIONS metadata so the grant's required-ref
check succeeds).
"""
show_result = execute(session, "SHOW INTEGRATIONS", cacheable=True)
show_result = _filter_result(show_result, name=fqn.name)
if len(show_result) == 0:
return None
if len(show_result) > 1:
raise Exception(f"Found multiple integrations matching {fqn}")
data = show_result[0]
owner = _fetch_owner(session, "INTEGRATION", fqn)
return {
"name": _quote_snowflake_identifier(data["name"]),
"type": data["type"],
"category": data["category"],
"enabled": data["enabled"] == "true",
"comment": data["comment"] or None,
"owner": owner,
}


def fetch_external_access_integration(session: SnowflakeConnection, fqn: FQN):
integrations = _show_resources(session, "EXTERNAL ACCESS INTEGRATIONS", fqn)
if len(integrations) == 0:
Expand Down Expand Up @@ -2348,6 +2385,23 @@ def fetch_schema(session: SnowflakeConnection, fqn: FQN, include_params: bool =
}


def fetch_git_repository(session: SnowflakeConnection, fqn: FQN):
show_result = _show_resources(session, "GIT REPOSITORIES", fqn)
if len(show_result) == 0:
return None
if len(show_result) > 1:
raise Exception(f"Found multiple git repositories matching {fqn}")
data = show_result[0]
return {
"name": _quote_snowflake_identifier(data["name"]),
"origin": data["origin"],
"api_integration": data["api_integration"],
"git_credentials": data.get("git_credentials") or None,
"comment": data["comment"] or None,
"owner": _get_owner_identifier(data),
}


def fetch_secret(session: SnowflakeConnection, fqn: FQN):
show_result = _show_resources(session, "SECRETS", fqn)
if len(show_result) == 0:
Expand Down Expand Up @@ -3311,6 +3365,17 @@ def list_dynamic_tables(session: SnowflakeConnection) -> list[FQN]:
return list_schema_scoped_resource(session, "DYNAMIC TABLES")


def list_integrations(session: SnowflakeConnection) -> list[FQN]:
"""List every integration in the account (any subtype).

Backs the generic `ResourceType.INTEGRATION` (umbrella). Concrete subtypes
(API/CATALOG/EXTERNAL_ACCESS/NOTIFICATION/SECURITY/STORAGE) still have their
own list_*_integrations functions for typed manifests.
"""
show_result = execute(session, "SHOW INTEGRATIONS", cacheable=True)
return [FQN(name=resource_name_from_snowflake_metadata(row["name"])) for row in show_result]


def list_external_access_integrations(session: SnowflakeConnection) -> list[FQN]:
return list_account_scoped_resource(session, "EXTERNAL ACCESS INTEGRATIONS")

Expand Down Expand Up @@ -3830,6 +3895,10 @@ def list_schemas(session: SnowflakeConnection, database=None) -> list[FQN]:
raise


def list_git_repositories(session: SnowflakeConnection) -> list[FQN]:
return list_schema_scoped_resource(session, "GIT REPOSITORIES")


def list_secrets(session: SnowflakeConnection) -> list[FQN]:
return list_schema_scoped_resource(session, "SECRETS")

Expand Down
9 changes: 8 additions & 1 deletion snowcap/privs.py
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,12 @@ class SecretPriv(Priv):
USAGE = "USAGE"


class GitRepositoryPriv(Priv):
OWNERSHIP = "OWNERSHIP"
READ = "READ"
WRITE = "WRITE"


class SequencePriv(Priv):
ALL = "ALL"
OWNERSHIP = "OWNERSHIP"
Expand Down Expand Up @@ -389,7 +395,7 @@ class WarehousePriv(Priv):
ResourceType.FAILOVER_GROUP: FailoverGroupPriv,
ResourceType.FILE_FORMAT: FileFormatPriv,
ResourceType.FUNCTION: FunctionPriv,
ResourceType.GIT_REPOSITORY: None,
ResourceType.GIT_REPOSITORY: GitRepositoryPriv,
ResourceType.GRANT: None,
ResourceType.HYBRID_TABLE: None,
ResourceType.ICEBERG_TABLE: IcebergTablePriv,
Expand Down Expand Up @@ -445,6 +451,7 @@ class WarehousePriv(Priv):
ResourceType.FAILOVER_GROUP: AccountPriv.CREATE_FAILOVER_GROUP,
ResourceType.FILE_FORMAT: SchemaPriv.CREATE_FILE_FORMAT,
ResourceType.FUNCTION: SchemaPriv.CREATE_FUNCTION,
ResourceType.GIT_REPOSITORY: SchemaPriv.CREATE_GIT_REPOSITORY,
# ResourceType.GRANT: AccountPriv.CREATE_GRANT,
ResourceType.MATERIALIZED_VIEW: SchemaPriv.CREATE_MATERIALIZED_VIEW,
ResourceType.NETWORK_POLICY: AccountPriv.CREATE_NETWORK_POLICY,
Expand Down
2 changes: 2 additions & 0 deletions snowcap/resources/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
from .failover_group import FailoverGroup
from .file_format import CSVFileFormat, JSONFileFormat, ParquetFileFormat
from .function import JavascriptUDF, PythonUDF
from .git_repository import GitRepository
from .grant import DatabaseRoleGrant, Grant, RoleGrant
from .hybrid_table import HybridTable
from .iceberg_table import SnowflakeIcebergTable
Expand Down Expand Up @@ -99,6 +100,7 @@
"GCPOutboundNotificationIntegration",
"GCSStorageIntegration",
"GenericSecret",
"GitRepository",
"GlueCatalogIntegration",
"Grant",
"HybridTable",
Expand Down
Loading