Skip to content

Commit 0d747ce

Browse files
committed
docs(azure): add self-managed Azure setup without external DNS
Add comprehensive documentation for setting up self-managed Azure HyperShift without External DNS. This provides a simpler alternative path for development and testing environments. Changes: - Add setup-management-cluster-without-external-dns.md guide - Add create-self-managed-azure-cluster-without-external-dns.md guide - Update self-managed-azure-index.md with DNS options and dual paths The without-external-dns approach uses Azure LoadBalancer DNS for the API server and Routes for other services, eliminating the need for External DNS operator, service principals, and DNS zone configuration. Signed-off-by: Bryan Cox <[email protected]> Commit-Message-Assisted-by: Claude (via Claude Code)
1 parent e5d781f commit 0d747ce

File tree

5 files changed

+477
-205
lines changed

5 files changed

+477
-205
lines changed
Lines changed: 270 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,270 @@
1+
# Create a Self-Managed Azure HostedCluster Without External DNS
2+
3+
!!! note "Developer Preview in OCP 4.21"
4+
5+
Self-managed Azure HostedClusters are available as a Developer Preview feature in OpenShift Container Platform 4.21.
6+
7+
This document describes how to create a self-managed Azure HostedCluster without using External DNS for automatic DNS record management.
8+
9+
## Prerequisites
10+
11+
Before creating a hosted cluster, ensure you have completed:
12+
13+
1. **[Azure Workload Identity Setup](azure-workload-identity-setup.md)** - Created workload identities and OIDC issuer
14+
2. **[Management Cluster Setup Without External DNS](setup-management-cluster-without-external-dns.md)** - Installed HyperShift operator
15+
16+
Additionally, you need:
17+
18+
- Azure CLI (`az`) configured with appropriate permissions
19+
- HyperShift CLI binary
20+
- OpenShift pull secret
21+
- SSH key (or use `--generate-ssh`)
22+
- `workload-identities.json` file from Phase 1
23+
24+
## Environment Setup
25+
26+
Set the required environment variables:
27+
28+
```bash
29+
# Cluster configuration
30+
CLUSTER_NAME="my-hosted-cluster"
31+
LOCATION="eastus"
32+
BASE_DOMAIN="example.com"
33+
PULL_SECRET="/path/to/pull-secret.json"
34+
35+
# Azure subscription and resource groups
36+
SUBSCRIPTION_ID=$(az account show --query id -o tsv)
37+
TENANT_ID=$(az account show --query tenantId -o tsv)
38+
PERSISTENT_RG_NAME="os4-common" # From Phase 1
39+
MANAGED_RG_NAME="${CLUSTER_NAME}-rg"
40+
41+
# Workload Identity configuration (from Phase 1)
42+
WORKLOAD_IDENTITIES_FILE="/path/to/workload-identities.json"
43+
OIDC_ISSUER_URL="https://yourstorageaccount.blob.core.windows.net/oidc-issuer"
44+
45+
# Networking (optional - will be created if not specified)
46+
VNET_ID="" # Leave empty to create new VNet
47+
SUBNET_ID="" # Leave empty to create new subnet
48+
NSG_ID="" # Leave empty to create new NSG
49+
50+
# DNS Zone (optional - for custom ingress domain)
51+
DNS_ZONE_RG_NAME="" # Leave empty if not using custom DNS zone
52+
53+
# Release configuration
54+
RELEASE_IMAGE="quay.io/openshift-release-dev/ocp-release:4.21.0-x86_64"
55+
```
56+
57+
## Create the HostedCluster
58+
59+
### Basic Cluster Creation (Without Custom DNS)
60+
61+
Create a hosted cluster using default DNS provided by Azure LoadBalancers:
62+
63+
```bash
64+
hypershift create cluster azure \
65+
--name "${CLUSTER_NAME}" \
66+
--location "${LOCATION}" \
67+
--base-domain "${BASE_DOMAIN}" \
68+
--pull-secret "${PULL_SECRET}" \
69+
--release-image "${RELEASE_IMAGE}" \
70+
--node-pool-replicas 2 \
71+
--generate-ssh \
72+
--workload-identities-file "${WORKLOAD_IDENTITIES_FILE}" \
73+
--oidc-issuer-url "${OIDC_ISSUER_URL}" \
74+
--resource-group-name "${MANAGED_RG_NAME}"
75+
```
76+
77+
!!! note "No External DNS Domain Flag"
78+
79+
Notice that we **do not** specify the `--external-dns-domain` flag. This tells HyperShift to:
80+
81+
- Use a LoadBalancer for the API server (gets Azure-provided DNS automatically)
82+
- Use Routes for OAuth, Konnectivity, and Ignition services
83+
- Not create any custom hostname annotations for External DNS
84+
85+
### With Custom Networking
86+
87+
If you have pre-created VNet, Subnet, and NSG resources:
88+
89+
```bash
90+
hypershift create cluster azure \
91+
--name "${CLUSTER_NAME}" \
92+
--location "${LOCATION}" \
93+
--base-domain "${BASE_DOMAIN}" \
94+
--pull-secret "${PULL_SECRET}" \
95+
--release-image "${RELEASE_IMAGE}" \
96+
--node-pool-replicas 2 \
97+
--generate-ssh \
98+
--workload-identities-file "${WORKLOAD_IDENTITIES_FILE}" \
99+
--oidc-issuer-url "${OIDC_ISSUER_URL}" \
100+
--resource-group-name "${MANAGED_RG_NAME}" \
101+
--vnet-id "${VNET_ID}" \
102+
--subnet-id "${SUBNET_ID}" \
103+
--network-security-group-id "${NSG_ID}"
104+
```
105+
106+
### With Custom DNS Zone for Ingress
107+
108+
If you want to use a custom DNS zone for the ingress domain (*.apps):
109+
110+
```bash
111+
hypershift create cluster azure \
112+
--name "${CLUSTER_NAME}" \
113+
--location "${LOCATION}" \
114+
--base-domain "${BASE_DOMAIN}" \
115+
--pull-secret "${PULL_SECRET}" \
116+
--release-image "${RELEASE_IMAGE}" \
117+
--node-pool-replicas 2 \
118+
--generate-ssh \
119+
--workload-identities-file "${WORKLOAD_IDENTITIES_FILE}" \
120+
--oidc-issuer-url "${OIDC_ISSUER_URL}" \
121+
--resource-group-name "${MANAGED_RG_NAME}" \
122+
--dns-zone-rg-name "${DNS_ZONE_RG_NAME}"
123+
```
124+
125+
!!! info "DNS Zone Resource Group"
126+
127+
The `--dns-zone-rg-name` flag specifies where your Azure DNS zone for the ingress domain resides. This is required by the ingress controller to manage DNS records for `*.apps.${CLUSTER_NAME}.${BASE_DOMAIN}`.
128+
129+
This is **different** from External DNS - it's used by the hosted cluster's own ingress controller.
130+
131+
## Verify Cluster Creation
132+
133+
Monitor the hosted cluster deployment:
134+
135+
```bash
136+
# Watch the HostedCluster resource
137+
oc get hostedcluster -n clusters -w
138+
139+
# Check the hosted control plane pods
140+
oc get pods -n clusters-${CLUSTER_NAME}
141+
142+
# Get the cluster status
143+
oc get hostedcluster ${CLUSTER_NAME} -n clusters -o jsonpath='{.status.conditions[?(@.type=="Available")].status}'
144+
```
145+
146+
## Get Cluster Access Information
147+
148+
### API Server Endpoint
149+
150+
Without External DNS, the API server uses an Azure LoadBalancer with an Azure-provided DNS name:
151+
152+
```bash
153+
# Get the API server LoadBalancer service
154+
API_LB_SERVICE=$(oc get svc -n clusters-${CLUSTER_NAME} kube-apiserver -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
155+
echo "API Server: https://${API_LB_SERVICE}:6443"
156+
157+
# Or get the IP address
158+
API_LB_IP=$(oc get svc -n clusters-${CLUSTER_NAME} kube-apiserver -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
159+
echo "API Server IP: ${API_LB_IP}"
160+
```
161+
162+
### Get Kubeconfig
163+
164+
Retrieve the kubeconfig for the hosted cluster:
165+
166+
```bash
167+
hypershift create kubeconfig --name ${CLUSTER_NAME} > ${CLUSTER_NAME}-kubeconfig
168+
```
169+
170+
The kubeconfig will use the Azure LoadBalancer DNS name for the API server endpoint.
171+
172+
### Get Admin Password
173+
174+
```bash
175+
oc get secret -n clusters ${CLUSTER_NAME}-kubeadmin-password \
176+
-o jsonpath='{.data.password}' | base64 -d
177+
```
178+
179+
## Optional: Create Custom DNS Records
180+
181+
If you want to use a custom domain name for the API server instead of the Azure-provided name:
182+
183+
### Create DNS A Record for API Server
184+
185+
```bash
186+
# Get the LoadBalancer public IP
187+
API_LB_IP=$(oc get svc -n clusters-${CLUSTER_NAME} kube-apiserver \
188+
-o jsonpath='{.status.loadBalancer.ingress[0].ip}')
189+
190+
# Create DNS A record (example using Azure DNS)
191+
az network dns record-set a add-record \
192+
--resource-group YOUR_DNS_ZONE_RG \
193+
--zone-name ${BASE_DOMAIN} \
194+
--record-set-name api.${CLUSTER_NAME} \
195+
--ipv4-address ${API_LB_IP}
196+
197+
echo "Custom API endpoint: https://api.${CLUSTER_NAME}.${BASE_DOMAIN}:6443"
198+
```
199+
200+
### Update Kubeconfig with Custom DNS
201+
202+
If you created a custom DNS record, you can manually edit the kubeconfig to use your custom domain:
203+
204+
```bash
205+
# Edit the kubeconfig server field
206+
sed -i "s|https://.*:6443|https://api.${CLUSTER_NAME}.${BASE_DOMAIN}:6443|" ${CLUSTER_NAME}-kubeconfig
207+
```
208+
209+
## DNS Architecture Without External DNS
210+
211+
Understanding how DNS works in this setup:
212+
213+
| Service | Publishing Type | DNS Handling |
214+
|---------|----------------|--------------|
215+
| **API Server** | LoadBalancer | Azure-provided DNS (e.g., `abc123.eastus.cloudapp.azure.com`)<br>Or manually created DNS A record |
216+
| **OAuth** | Route | Management cluster ingress (e.g., `oauth-clustername.apps.mgmt-cluster.com`) |
217+
| **Konnectivity** | Route | Management cluster ingress |
218+
| **Ignition** | Route | Management cluster ingress |
219+
| **Ingress (*.apps)** | Managed by hosted cluster | Uses DNS zone specified in `--dns-zone-rg-name`<br>Or manual DNS configuration |
220+
221+
## Comparison: With vs Without External DNS
222+
223+
| Aspect | With External DNS | Without External DNS |
224+
|--------|------------------|---------------------|
225+
| **API Server** | Route with custom hostname<br>`api-cluster.external-domain.com` | LoadBalancer with Azure DNS<br>`abc123.region.cloudapp.azure.com` |
226+
| **Setup Complexity** | Higher (service principal, DNS zones, external-dns operator) | Lower (no additional operator) |
227+
| **DNS Management** | Automatic via external-dns | Manual or Azure-provided |
228+
| **Custom Domains** | Built-in | Manual DNS record creation |
229+
| **Best For** | Production, multi-cluster, custom branding | Development, testing, simpler setups |
230+
231+
## Troubleshooting
232+
233+
### API Server Not Accessible
234+
235+
```bash
236+
# Check LoadBalancer service status
237+
oc get svc -n clusters-${CLUSTER_NAME} kube-apiserver
238+
239+
# Check if LoadBalancer has external IP/hostname
240+
oc describe svc -n clusters-${CLUSTER_NAME} kube-apiserver
241+
242+
# Verify network security group allows traffic on port 6443
243+
az network nsg rule list \
244+
--resource-group ${MANAGED_RG_NAME} \
245+
--nsg-name ${CLUSTER_NAME}-nsg \
246+
--query "[?destinationPortRange=='6443']"
247+
```
248+
249+
### Ingress Controller Issues
250+
251+
```bash
252+
# Check ingress controller pods in hosted cluster
253+
oc --kubeconfig ${CLUSTER_NAME}-kubeconfig get pods -n openshift-ingress
254+
255+
# Check ingress controller configuration
256+
oc --kubeconfig ${CLUSTER_NAME}-kubeconfig get ingresscontroller -n openshift-ingress-operator
257+
```
258+
259+
## Next Steps
260+
261+
- [Configure Additional NodePools](../automated-machine-management/nodepool-lifecycle.md)
262+
- [Upgrade Your Hosted Cluster](../upgrades.md)
263+
- [Monitor Your Hosted Cluster](../per-hostedcluster-dashboard.md)
264+
265+
## Related Documentation
266+
267+
- [Azure Workload Identity Setup](azure-workload-identity-setup.md) - Phase 1 prerequisites
268+
- [Management Cluster Setup Without External DNS](setup-management-cluster-without-external-dns.md) - Phase 2
269+
- [Self-Managed Azure Overview](self-managed-azure-index.md) - Architecture and prerequisites
270+
- [With External DNS](create-azure-cluster-on-aks.md) - Alternative setup with automatic DNS

0 commit comments

Comments
 (0)