Support for layer.add_loss()

[han-ol]

With the add_loss API (https://keras.io/api/losses/#the-addloss-api) layers can add arbitrary losses that need not depend on the output of the network.
This is a useful feature for example if a custom layer needs to be regularized in a special way.

It basically works like this:
layer_1.add_loss adds loss tensors to a layer_1.losses list and parent layers/models have a joint losses attribute containing all these layer losses.

Currently, these losses are silently ignored in approximator.fit.
We can actually support add_loss with minimal changes by taking the self.losses into account in approximator.compute_metrics.

[LarsKue]

Unless I am mistaken, all keras losses are supervised and rely on the y_pred = f(x); loss = g(y, y_pred) structure. This does not work for us since our losses are inherently generative, i.e., unsupervised or self-supervised: loss = f(x). Is there a specific use case for this that you have in mind? Otherwise, I would not put too much effort into supporting this.

[han-ol]

The functionality of add_loss is actually really flexible. You can compute some quantity however you like and add it to the loss. So it allows for arbitrary customization with respect to regularization, finetuning, quantization.

I am sure there are a bunch of things this flexibility is useful for, but the concrete idea I can quote right now would be to incentivize some but not all weights to have discrete values, which is interesting if you later want to quantizing them with minimal accuracy decline.

[paul-buerkner]

I am in favor of having this feature, especially since the code overhead appears to be minimal.