Locking for mutations (PUT/PATCH/DELETE) should leverage k8s dry-run to avoid side effects

[gberche-orange]

While experimenting with the proxy and experiencing hangs reported into #106 while inserting relationships into spicedb, I wonder how much the locking modes (pessimistic or optimistic) are compatible with mutations.

The code suggests that a transactional approach is implemented

spicedb-kubeapi-proxy/pkg/authz/distributedtx/workflow.go

Lines 125 to 128 in cb46daa

	// PessimisticWriteToSpiceDBAndKube ensures that a write exists in both SpiceDB
	// and kube, or neither, using locks. It prevents multiple users from writing
	// the same object/fields at the same time
	func PessimisticWriteToSpiceDBAndKube(ctx workflow.Context, input *WriteObjInput) (*KubeResp, error) {

Reading the code suggests that a rollback strategy is applied when writes fails in spicedb or kube

spicedb-kubeapi-proxy/pkg/authz/distributedtx/workflow.go

Line 128 in cb46daa

func PessimisticWriteToSpiceDBAndKube(ctx workflow.Context, input *WriteObjInput) (*KubeResp, error) {

However, I'm observing that when insert hangs in spicedb, the objects are not rollbacked in k8s.

I'm wondering whether a rollback on k8s calls is currently implemented.

Besides, mutations operations on k8s resources (such as custom resources) have side effects. Therefore a rollback don't prevent the side effects from being applied.

Using the k8s dryRun support (see https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/576-dry-run/README.md) seems a way to avoid this side effect.

However, I'm not yet seeing use of dryRun in the current code base, see https://github.com/search?q=repo%3Aauthzed%2Fspicedb-kubeapi-proxy%20dryrun&type=code