Production deployment documentation #3712
Conversation
Signed-off-by: Mangirdas Judeikis <[email protected]> On-behalf-of: @SAP [email protected]
kcp-ci-bot
added
kind/documentation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
kcp-ci-bot
added
the
size/XXL
| garbageCollectionPolicy: Exponential | ||
| garbageCollectionPeriod: 43200s | ||
| deltaSnapshotPeriod: 300s | ||
| deltaSnapshotMemoryLimit: 1Gi |
There was a problem hiding this comment.
Not much important because this is about backups, but maybe it would be useful to have a comment why are these values chosen (e.g. if they're default, changed because it's better in prod this way, etc).
There was a problem hiding this comment.
This was taken from etcd-druid docs. would be good if they could comment on this. Never teted this myself :/
|
|
||
| The core `kcp-dev/kcp` repository is a monorepo containing the kcp core and some close to the core libraries. | ||
| See the [monorepo document](./monorepo/) for more details. | ||
| See the [monorepo document](./monorepo.md) for more details. |
There was a problem hiding this comment.
Why do we need .md extension here? The web page URL doesn't have it, e.g. https://docs.kcp.io/kcp/main/contributing/monorepo/
There was a problem hiding this comment.
mkdocs logs were complainings about this:
Doc file 'contributing/index.md' contains an unrecognized relative link './monorepo', it was left as is. Did you mean 'monorepo.md'?
There was a problem hiding this comment.
I think this is one of these: It works due to backwards compatability but mkdocs gives warrning
|
@mjudeikis: The following test failed, say
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
|
||
| This directory contains assets and configuration files for production deployment of kcp. | ||
|
|
||
| !!! Note: We, understand that maintaining static assets in the repository can be challenging. If you have noticed any discrepancies between these assets and the latest version of the kcp - please open an issue or submit a pull request to help us keep them up to date. |
There was a problem hiding this comment.
| !!! Note: We, understand that maintaining static assets in the repository can be challenging. If you have noticed any discrepancies between these assets and the latest version of the kcp - please open an issue or submit a pull request to help us keep them up to date. | |
| !!! Note: We understand that maintaining static assets in the repository can be challenging. If you have noticed any discrepancies between these assets and the latest version of the kcp - please open an issue or submit a pull request to help us keep them up to date. |
There was a problem hiding this comment.
Sharding and High Availability use the term workload, I have a feeling this is kind of wrong and it associates with Pods, containers, etc. Maybe we should try to find a better term just to avoid confusion. (This was present prior to this PR, I just had a chance to read this doc as a whole)
|
|
||
| **CloudFlare CA Certificate**: Download the CloudFlare edge certificate for extended trust: | ||
|
|
||
| **Important**: Verify this URL with CloudFlare documentation before production use. |
There was a problem hiding this comment.
Nit: I'd put this in a note box.
| - "*.alpha-peer.kcp-comer.svc" | ||
| - "*.alpha-peer.kcp-comer.svc.cluster.local" | ||
| issuerRef: | ||
| name: etcd-ca |
There was a problem hiding this comment.
Why is this not using etcd-ca from the kcp-comer namespace, similar to etcd-tls-ca?
| secretName: etcd-client-tls | ||
| commonName: etcd-client | ||
| issuerRef: | ||
| name: etcd-ca |
There was a problem hiding this comment.
Same here, why is this not using etcd-ca from the kcp-comer namespace, similar to etcd-tls-ca?
There was a problem hiding this comment.
Does it make sense that we put comments such as # CHANGE ME for fields that should be changed (e.g. dnsNames)? That way, people can easier find it, and we can say in docs search for CHANGE ME and change those values.
| apiVersion: operator.kcp.io/v1alpha1 | ||
| kind: FrontProxy | ||
| metadata: | ||
| name: frontproxy-internal |
There was a problem hiding this comment.
Out of curiosity, why is this called frontproxy-internal if it's exposed to the internet?
| - https://root-client.kcp-comer.svc.cluster.local:2379 | ||
| tlsConfig: | ||
| secretRef: | ||
| name: etcd-ca-tls |
There was a problem hiding this comment.
Out of curiosity, shouldn't this have client TLS? Just CA is not enough to do things in etcd AFAIK.
| username: kcp-admin | ||
| groups: | ||
| - system:kcp:admin | ||
| validity: 8766h |
There was a problem hiding this comment.
Just so it's clear:
| validity: 8766h | |
| validity: 8766h # 1 year |
| username: kcp-admin | ||
| groups: | ||
| - system:kcp:admin | ||
| validity: 8766h |
There was a problem hiding this comment.
Same here:
| validity: 8766h | |
| validity: 8766h # 1 year |
| kubectl apply -f contrib/production/kcp-comer/kcp-front-proxy-internal.yaml | ||
| ``` | ||
|
|
||
| **Get the LoadBalancer IP**: |
There was a problem hiding this comment.
I'm not sure if I'm a fan of bold-ing these, I think it would be better to have numbered list, like 1,2...
| Deploy kcp with dual front-proxy and CDN integration for enterprise environments requiring edge acceleration and advanced networking. | ||
| --- | ||
|
|
||
| # kcp-comer: Dual Front-Proxy with CDN Integration |
There was a problem hiding this comment.
I have a little hard time understanding what do you gain by using CDN. The power of CDN is in caching assets, think of images, files... What would be cached here exactly?
|
|
||
| ### 5. Configure DNS for Front-Proxy | ||
|
|
||
| 1. **Get the LoadBalancer IP**: |
There was a problem hiding this comment.
This is personally the way I like it, with numbered steps, but still, I think too much bold can be annoying.
|
|
||
| ```bash | ||
| # macOS | ||
| brew install int128/kubelogin/kubelogin |
There was a problem hiding this comment.
I'd also mention krew as an option.
| root https://root-kcp.kcp-dekker.svc.cluster.local:6443 https://api.dekker.example.com:6443 11m | ||
| ``` | ||
|
|
||
| ### Install kubectl OIDC Plugin |
There was a problem hiding this comment.
Shouldn't we have this for kcp-comer as well?
|
|
||
| ## Deployment Steps | ||
|
|
||
| ### 1. Configure DNS Records |
There was a problem hiding this comment.
This is confusing to me, how can you create these records when you don't know the IP addresses yet?
|
|
||
| Because we use Let's Encrypt, and since kubectl needs explisit CA configuration, we need to deploy kcp components with extended CA bundle trust. This mighgt be different in your environment. | ||
|
|
||
| ```bash |
| ```bash | ||
| curl -k https://api.vespucci.example.com:6443/healthz | ||
| ``` |
There was a problem hiding this comment.
Nit: I don't think this should be indented.
| The main API endpoint for clients to access kcp. This is the main entry point for all external consumer clients. | ||
|
|
||
| **Communication patterns:** | ||
| - Configured in both shards and front-proxy |
There was a problem hiding this comment.
This line doesn't read well to me, what did you mean exactly here?
| - Set `--externalHostname` or `spec.external.hostname` in front-proxy or shard configurations | ||
|
|
||
| ### Shards | ||
| Individual kcp shards that host workspaces. Shards can be exposed publicly or kept private. |
There was a problem hiding this comment.
It might be useful to explain why you might want to consider having public or private shard.
| ### Shards | ||
| Individual kcp shards that host workspaces. Shards can be exposed publicly or kept private. | ||
|
|
||
| **Configuration:** |
There was a problem hiding this comment.
How are shards registering with the front proxy? Is it done automatically? Is there anything needed to configure? Might be useful to clarify that
| ### Virtual Workspaces | ||
| kcp supports running virtual workspaces outside shards, but the recommended approach is to run virtual workspaces inside shards. | ||
|
|
||
| **Configuration:** |
There was a problem hiding this comment.
| **Configuration:** | |
| **Configuration for external virtual workspaces:** |
(I assume you don't need to do anything for internal VWs)
There was a problem hiding this comment.
What did you use to generate those diagrams? Are there sources anywhere?
| https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.26/releases/cnpg-1.26.0.yaml | ||
| ``` | ||
|
|
||
| ### 4.2. Deploy PostgreSQL Database |
There was a problem hiding this comment.
Why can't we just use etcd or even sqlite for Dex? PostgreSQL is quite an overhead, operational and resource usage wise.
| -f contrib/production/oidc-dex/values.yaml | ||
| ``` | ||
|
|
||
| ### 5. DNS Configuration |
There was a problem hiding this comment.
Isn't this done along the way in each of these setups?
| issuer: https://auth.example.com | ||
|
|
||
| logger: | ||
| level: "debug" |
There was a problem hiding this comment.
Is debug level really needed in prod? I'd guess something like info should be fine and safer.
3 participants
Summary
This PR adds detailed deployment documentation on how to deploy kcp in a production configuration.
/kind documentation
What Type of PR Is This?
Related Issue(s)
Fixes #
Release Notes