List requests do not map HTTP status codes to error variants, so 403 and 404 both become `Error::Generic`

[TomNicholas]

What happens

A list that fails with 403 or 404 comes back as Error::Generic. The same failure on get or head comes back as PermissionDenied or NotFound.

This means you cannot tell "I am not allowed to read this bucket" apart from "this bucket does not exist" apart from "the network is down", unless you parse the error string.

Why

In src/aws/client.rs, only two variants are passed to the status-code mapper:

match err {
    Error::CompleteMultipartRequest { source, path } => source.error(STORE, path),
    Error::DeleteObjectsRequest { source, paths }    => source.error(STORE, paths.join(",")),
    _ => Self::Generic { store: STORE, source: Box::new(err) },
}

Error::ListRequest hits the _ arm, so it never reaches RetryError::error(), which is the function that turns 404 into NotFound, 403 into PermissionDenied and 401 into Unauthenticated.

src/gcp/client.rs has the same problem. There, GetRequest and Request are mapped and ListRequest falls through.

How to reproduce

Reproduced with obstore 0.9.2, against real S3:

import asyncio, obstore
from obstore.store import S3Store

async def main():
    # a private bucket -> S3 returns 403
    s = S3Store(bucket="<private-bucket>", region="us-east-1", skip_signature=True)

    try:
        await obstore.list_with_delimiter_async(s)
    except Exception as e:
        print(type(e).__name__)   # GenericError

    try:
        await obstore.head_async(s, "probe")
    except Exception as e:
        print(type(e).__name__)   # PermissionDeniedError

asyncio.run(main())

Same bucket, same 403, two different error variants.

A bucket that does not exist returns 404, and list reports that as generic too.

Suggested fix

Add a ListRequest arm that goes through source.error(...), the same way #365 / #366 did for the HTTP backend.

I am happy to open a PR if this looks right.

[kylebarron]

Seems reasonable to add match cases for 403 and 404