Add Liveness entrypoint

[felixleo22]

I’m trying to deploy SATOSA on Kubernetes, and I want to add a liveness probe to my deployment.

I noticed that SATOSA only becomes truly available after it receives data from the identity provider. Therefore, the following Kubernetes liveness probe:

livenessProbe:
  httpGet:
    path: /liveness
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 10

is not sufficient.

If SATOSA takes a long time to receive data from the identity provider, the server does not start until it has received that data, and the port is therefore not accessible until the server is fully up.

Is it possible to add a /liveness route that returns "ok" or something else only after data has been received from the identity provider?

Thanks

[c00kiemon5ter]

Hello Felix, what is the data from the identity provider that is needed by the proxy/SATOSA? I am guessing you're talking about the metadata?

If that is the case, for a production service you should be using an MDX service. The MDX service is collecting and maintaining the metadata and its validity. The proxy is querying the MDX service every time a service/SP or provider/IdP sends a request towards the proxy.
That means that when the proxy comes up it is ready to get to work. The MDX should already be up and available when the proxy boots.

Let us know what you're thinking.

[felixleo22]

My bad,

By "data", I mean metadata from RENATER Identity provider.

Actually, to check if our SATOSA is available, I only use the readinessProbe by checking the HTTP server. But the HTTP server doesn't start until it has received metadata.

What do you mean by MDX service ? Shouldn't the proxy fetch metadata by itself already?

[c00kiemon5ter]

The metadata of a Federation (or even an IdP) is not a static artefact. It is ever changing and evolving. In practice, many federations and inter-federations (like eduGAIN) have policies about how often the metadata should be updated. IIRC, for eduGAIN that is every 8hours.

To do this, you most often use a metadata aggregator service that support the Metadata Queue (MDQ) protocol. This facilitates metadata exchange (MDX). pyFF (python Federation Feeder) can do this. Of course you'd need to configure it properly and schedule it to refetch and revalidate the metadata per your federation policies.

So, while the proxy can load metadata (from a local file or remote endpoint), it does this once and not in the most efficient way. This functionality is supported but is intended for dev/testing scenarios and cases when the metadata will not be updated often. That way of operation is also much simpler than configuring and running a separate MDX component. But it has its tradeoffs as you're already seeing.

[oxypomme]

Thanks for the answer, we'll look into setting up a MDX service soon (probably pyFF as you suggest) !

If we find a suitable liveness probe we'll share it here.

EDIT : Might be a good idea to add a section into the proxy documentation about MDX

[vladimir-mencl-eresearch]

Hi @oxypomme ,

Depending on what metadata you are trying to load, it might already be available.

It appears RENATER operates an MDQ service: https://services.renater.fr/federation/documentation/generale/metadata/mdq

And while SATOSA should perhaps document the configuration as well, configuring MDQ is covered in the pysaml2 documentation (pysaml2 is the underlying library used by the SATOSA SAML backend): https://pysaml2.readthedocs.io/en/latest/howto/config.html#metadata

Hope this helps.

Cheers,
Vlad

[oxypomme]

Hello again,

We successfully used the MDQ service from RENATER, thanks for the help !

However, to return to the original topic, we added a (simple) liveness probe and attempted to find a (functional but not optimal) readiness probe to our Kubernetes deployment :

livenessProbe:
  httpGet:
    path: /.well-known/openid-configuration
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 10

readinessProbe:
  httpGet:
    path: /SAML2/ACS/POST
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 10

As the proxy serves a the OpenID configuration as long as the proxy is up (even tho it can't proxify requests), we can use it as a route to get if SATOSA is running with minimal configuration.

We did thought of using the assertion_consumer_service endpoint to get if the proxy is ready to proxify requests, we did found out that the endpoint returns 500 until a user attempted to log in through SATOSA.

We think it would be nice to get the status of the services running inside SATOSA (is the frontend ready ? are the metadata for the backend available ? etc.). We probably could get around with some scripts added to a custom Docker image, but it would be easier to have endpoints dedicated for that.

Any thoughts about that feature ?