Flavor: foreman-proxy-content

[arvind4501]

Why are you introducing these changes? (Problem description, related links)

What are the changes introduced in this pull request?

• Add deploy-proxy sub-command to deploy proxy specific flavors(ex: foreman-proxy-content)
• Add proxy specific parameters to deploy-proxy ex: certs tar file and foreman fqdn
• Make health checks, post-install tasks, and pull-images flavor-aware instead of hardcoded to katello
• Support certificate extraction from tar bundles on proxy nodes
• Add httpd vhosts for proxying Pulp content and RHSM requests
• Add flavor-specific test directories, --flavor pytest option, and proxy CI job
• Refactor deploy metadata into reusable obsah includes (_foreman, _foreman_proxy, _pulp, _flavors/katello)

How to test this pull request

• ./foremanctl deploy

• ./foremanctl certificate-bundle proxy.example.com to generate bundle

• ./foremanctl deploy-proxy --flavor foreman-proxy-content --certificate-bundle /path-to-tar --foreman-fqdn quadlet.example.com

• Observe only relevent services are deployed

Steps to reproduce:

• ./foremanctl deploy-proxy --flavor foreman-proxy-content --certificate-bundle /path-to-tar --foreman-fqdn quadlet.example.com

Checklist

• Tests added/updated (if applicable)
• Documentation updated (if applicable)

[ehelms]

There are definitely some good nuggets of changes that would make for good, go-ahead, stand-alone PRs to get added. This also points to the need for flavor specific parameters that are only shown for a given flavor.

[arvind4501]

I have been facing one issue which i already have a workaround implemented but i wanted to know if we should solve it properly?

so we have a static quadlet target in our playbooks(which may be fine for today as it points to localhost), but in CI and development we need to deploy on different servers not localhost. currently the workaround is to pass a extra var in command and use that to determine the host.

What about exposting a --target obsah option to manage that, but the issue is its visible to users

[Gauravtalreja1]

When I ran foremanctl deploy-proxy without --certificate-bundle or --foreman-fqdn options, it fails with below error, instead i think it should have thrown a validation error that this are mandatory options, right?

[root@foreman-proxy-server ~]# foremanctl deploy-proxy
.
..
TASK [pulp : Configure Foreman Proxy] *********************************************************************************************************************************************************
fatal: [localhost]: FAILED! =>
    changed: false
    msg: |-
        Failed to connect to Foreman server: DocLoadingError: Could not load data from https://foreman-proxy-server.example.com: 503 Server Error: Service Unavailable for url: https://foreman-proxy-server.example.com/apidoc/v2.json
                          - is your server down?

PLAY RECAP ************************************************************************************************************************************************************************************
localhost                  : ok=126  changed=82   unreachable=0    failed=1    skipped=24   rescued=0    ignored=0

[arvind4501]

Yes, that might be concern, looking at https://obsah.readthedocs.io/en/latest/development.html#constraints there is no way to say give these params otherwise deployment won't work currently. but having said that i also expect user to follow steps for deployment for at least first time. i also wonder if we had such validations in foreman-installer?

[arvind4501]

Also, we would not want to make --certificate-bundle or --foreman-fqdn options as hard validation to be present each time we do deploy-proxy. for next runs if i want to add any feature i would just do foremanctl deploy-proxy --add-feature bmc and expect it to work without any issue

[ehelms]

I tend to agree -- parameters that are required the first time it ever runs but not necessarily after that is tricky. I think we should look into solving that but as a follow up and not in this initial PR. Please file a Github issue with that concern.

[Gauravtalreja1]

When I ran foremanctl deploy-proxy with random dir/nonexistnet --certificate-bundle it fails with below error, instead i think it should have thrown a validation error file not exist or with a proper error, right?

[root@foreman-proxy-server ~]# foremanctl deploy-proxy --certificate-bundle ~/mycerts
.
..
TASK [pulp : Configure Foreman Proxy] *********************************************************************************************************************************************************
fatal: [localhost]: FAILED! =>
    changed: false
    msg: |-
        Failed to connect to Foreman server: DocLoadingError: Could not load data from https://foreman-proxy-server.example.com: 503 Server Error: Service Unavailable for url: https://foreman-proxy-server.example.com/apidoc/v2.json
                          - is your server down?

PLAY RECAP ************************************************************************************************************************************************************************************
localhost                  : ok=126  changed=82   unreachable=0    failed=1    skipped=24   rescued=0    ignored=0

[arvind4501]

I think thats the issue of not passing fqdn, rather than certificate bundle. and validation exist for certificate bundle https://github.com/theforeman/foremanctl/pull/571/changes#diff-01885bb15c1a59c6f49bbaa057e4d11075238b4bf9ce1968f4cc4c47de259690R8

[ehelms]

I think we could look into some better pre-install checks that helps fail fast but as a follow up instead of in this PR. From your testing, checks I think we should consider as a follow up:

• validating certificate bundle actually exists (and possibly that it's valid)
• that the foreman-fqdn is reachable

[Gauravtalreja1]

When I pass --flavor foreman-proxy-content --foreman-fqdn foreman.example.com along with --certificate-bundle ~/mycerts, the validation works to check if certificate bundle actually exists and deploy-proxy fails with below error, but validation for foreman-fqdn is reachable still makes sense

TASK [certificates : Check path to certificate tar file exists] *******************************************************************************************************************************
ok: [localhost]

TASK [certificates : Fail if path to certificate tar file does not exist] *********************************************************************************************************************
fatal: [localhost]: FAILED! =>
    changed: false
    msg: 'Path to certificate tar file not found: /root/mycerts'