Skip to content

AGENT-1207: automate OVE cluster installation #1796

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 20, 2025
Merged

AGENT-1207: automate OVE cluster installation #1796

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion agent/01_agent_requirements.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,5 +55,6 @@ if [[ "${AGENT_E2E_TEST_BOOT_MODE}" == "ISCSI" ]]; then
fi

if [[ "${AGENT_E2E_TEST_BOOT_MODE}" == "ISO_NO_REGISTRY" ]]; then
sudo dnf -y install xorriso coreos-installer syslinux skopeo
sudo dnf -y install epel-release
sudo dnf -y install chromium skopeo
fi
38 changes: 6 additions & 32 deletions agent/06_agent_create_cluster.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -646,41 +646,15 @@ case "${AGENT_E2E_TEST_BOOT_MODE}" in

check_assisted_install_UI

# Temporarily create a dummy kubeconfig and kubeadmin-password file for the CI
auth_dir=$SCRIPTDIR/$OCP_DIR/auth
mkdir -p $auth_dir
cfg=$auth_dir/kubeconfig
cat << EOF >> ${cfg}
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBGSUNBVLS0tLQo=
server: https://api.test.redhat.com:6443
name: test
contexts:
- context:
cluster: test
user: admin
name: admin
current-context: admin
preferences: {}
users:
- name: admin
user:
client-certificate-data: LS0tLS1CRUdJTiBNBVEUtLS0tLQo=
client-key-data: LS0tLS1CRUdJTiURSBVktLS0tLQo=
EOF
echo "dummy-kubeadmin-password" > $auth_dir/kubeadmin-password
mkdir -p $OCP_DIR/auth
rendezvousIP=$(getRendezvousIP)
get_vips
# Simulate user actions as done on the webUI and start cluster installation
CLUSTER_NAME=$CLUSTER_NAME BASE_DOMAIN=$BASE_DOMAIN RENDEZVOUS_IP=$rendezvousIP OCP_DIR=$OCP_DIR INGRESS_VIP=$INGRESS_VIPS API_VIP=$API_VIPS go run agent/isobuilder/ui_driven_cluster_installation/main.go
exit 0
;;
esac

if [[ "${AGENT_E2E_TEST_BOOT_MODE}" == "ISO_NO_REGISTRY" ]]; then
# Current goal is to only verify if the nodes are booted fine,
# TUI sets the rendezvous IP correctly and UI is accessible.
# The next goal is to simulate adding the cluster details via UI
# and complete the cluster installation.
exit 0
fi

if [ ! -z "${AGENT_TEST_CASES:-}" ]; then
run_agent_test_cases
fi
Expand Down
7 changes: 5 additions & 2 deletions agent/agent_post_install_validation.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,10 @@ SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )"

source $SCRIPTDIR/common.sh

# Temp code skip the execution flow as cluster is not really installed
if [[ "${AGENT_E2E_TEST_BOOT_MODE}" == "ISO_NO_REGISTRY" ]]; then
exit 0
oc wait clusterversion version --for=condition=Available=True --timeout=60m
oc get csv -A
oc get packagemanifests -n openshift-marketplace
Copy link
Member

@andfasano andfasano Sep 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are these two commands for? It doesn't seem they are testing anything. I was expecting a more explicit check on the list of installed operators

Copy link
Contributor Author

@pawanpinjarkar pawanpinjarkar Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oc get csv -A is not useful seems like. oc get packagemanifests -n openshift-marketplace gives a list of operators , e.g.

oc get packagemanifests -n openshift-marketplace 
Member
NAME                                CATALOG   AGE
kubernetes-nmstate-operator                   3d20h
kubevirt-hyperconverged                       3d20h
node-healthcheck-operator                     3d20h
fence-agents-remediation                      3d20h
mtv-operator                                  3d20h
cluster-kube-descheduler-operator             3d20h
node-maintenance-operator                     3d20h

Copy link
Contributor Author

@pawanpinjarkar pawanpinjarkar Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are these two commands for? It doesn't seem they are testing anything. I was expecting a more explicit check on the list of installed operators

Could you suggest what else we should be testing?

Copy link
Contributor Author

@pawanpinjarkar pawanpinjarkar Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andfasano Created this task https://issues.redhat.com/browse/AGENT-1304

Copy link
Member

@andfasano andfasano Oct 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you suggest what else we should be testing?

Sorry missed that comment. This part must be changed to verify that the expected list of OLM operators have been successfully installed.
Specifically, the script must check that for each operator listed here (the list can be hard-coded, since it's not going to change so often)

fi

installed_control_plane_nodes=$(oc get nodes --selector=node-role.kubernetes.io/master | grep -v AGE | wc -l)
Expand All @@ -18,3 +19,5 @@ if (( $NUM_MASTERS != $installed_control_plane_nodes )); then
echo "Post install validation failed. Expected $NUM_MASTERS control plane nodes but found $installed_control_plane_nodes."
exit 1
fi

oc get clusterversion
Copy link
Member

@andfasano andfasano Sep 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And also this one

Copy link
Contributor Author

@pawanpinjarkar pawanpinjarkar Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought this to be a good way to see the cluster version
e.g. oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.19.7 True False 3d19h Error while reconciling 4.19.7: the cluster operator insights is not available

Copy link
Member

@andfasano andfasano Oct 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be preferable, for each required OLM operator, to explicitly verify if it was installed or not, and report it.

Copy link
Contributor Author

@pawanpinjarkar pawanpinjarkar Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracked via https://issues.redhat.com/browse/AGENT-1376

6 changes: 3 additions & 3 deletions agent/gather.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ done < "${OCP_DIR}"/hosts
num_screenshots=$(find "${OCP_DIR}" -type f -name "*.ppm" | wc -l)
if [[ "$num_screenshots" -gt 0 ]]; then
archive_name="agent-gather-console-screenshots.tar.xz"
echo "Gathering console screenshots to $archive_name"
tar -cJf $archive_name ${OCP_DIR}/*.ppm
echo "Gathering screenshots to $archive_name"
tar -cJf $archive_name ${OCP_DIR}/*.ppm ${OCP_DIR}/*.png
else
echo "No console screenshots found. Skipping screenshot gather."
echo "No screenshots found. Skipping screenshot gather."
fi
20 changes: 20 additions & 0 deletions agent/isobuilder/ui_driven_cluster_installation/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
module github.com/openshift-metal3/dev-scripts/ui-driven-cluster-installation

go 1.22.3

require (
github.com/go-resty/resty/v2 v2.16.5
github.com/go-rod/rod v0.116.2
github.com/pkg/errors v0.9.1
github.com/sirupsen/logrus v1.9.3
)

require (
github.com/ysmood/fetchup v0.2.3 // indirect
github.com/ysmood/goob v0.4.0 // indirect
github.com/ysmood/got v0.40.0 // indirect
github.com/ysmood/gson v0.7.3 // indirect
github.com/ysmood/leakless v0.9.0 // indirect
golang.org/x/net v0.33.0 // indirect
golang.org/x/sys v0.28.0 // indirect
)
40 changes: 40 additions & 0 deletions agent/isobuilder/ui_driven_cluster_installation/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/go-resty/resty/v2 v2.16.5 h1:hBKqmWrr7uRc3euHVqmh1HTHcKn99Smr7o5spptdhTM=
github.com/go-resty/resty/v2 v2.16.5/go.mod h1:hkJtXbA2iKHzJheXYvQ8snQES5ZLGKMwQ07xAwp/fiA=
github.com/go-rod/rod v0.116.2 h1:A5t2Ky2A+5eD/ZJQr1EfsQSe5rms5Xof/qj296e+ZqA=
github.com/go-rod/rod v0.116.2/go.mod h1:H+CMO9SCNc2TJ2WfrG+pKhITz57uGNYU43qYHh438Mg=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/ysmood/fetchup v0.2.3 h1:ulX+SonA0Vma5zUFXtv52Kzip/xe7aj4vqT5AJwQ+ZQ=
github.com/ysmood/fetchup v0.2.3/go.mod h1:xhibcRKziSvol0H1/pj33dnKrYyI2ebIvz5cOOkYGns=
github.com/ysmood/goob v0.4.0 h1:HsxXhyLBeGzWXnqVKtmT9qM7EuVs/XOgkX7T6r1o1AQ=
github.com/ysmood/goob v0.4.0/go.mod h1:u6yx7ZhS4Exf2MwciFr6nIM8knHQIE22lFpWHnfql18=
github.com/ysmood/gop v0.2.0 h1:+tFrG0TWPxT6p9ZaZs+VY+opCvHU8/3Fk6BaNv6kqKg=
github.com/ysmood/gop v0.2.0/go.mod h1:rr5z2z27oGEbyB787hpEcx4ab8cCiPnKxn0SUHt6xzk=
github.com/ysmood/got v0.40.0 h1:ZQk1B55zIvS7zflRrkGfPDrPG3d7+JOza1ZkNxcc74Q=
github.com/ysmood/got v0.40.0/go.mod h1:W7DdpuX6skL3NszLmAsC5hT7JAhuLZhByVzHTq874Qg=
github.com/ysmood/gotrace v0.6.0 h1:SyI1d4jclswLhg7SWTL6os3L1WOKeNn/ZtzVQF8QmdY=
github.com/ysmood/gotrace v0.6.0/go.mod h1:TzhIG7nHDry5//eYZDYcTzuJLYQIkykJzCRIo4/dzQM=
github.com/ysmood/gson v0.7.3 h1:QFkWbTH8MxyUTKPkVWAENJhxqdBa4lYTQWqZCiLG6kE=
github.com/ysmood/gson v0.7.3/go.mod h1:3Kzs5zDl21g5F/BlLTNcuAGAYLKt2lV5G8D1zF3RNmg=
github.com/ysmood/leakless v0.9.0 h1:qxCG5VirSBvmi3uynXFkcnLMzkphdh3xx5FtrORwDCU=
github.com/ysmood/leakless v0.9.0/go.mod h1:R8iAXPRaG97QJwqxs74RdwzcRHT1SWCGTNqY8q0JvMQ=
golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/time v0.6.0 h1:eTDhh4ZXt5Qf0augr54TN6suAUudPcawVZeIAPU7D4U=
golang.org/x/time v0.6.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
Loading