Location header is parsed unconditionally even when follow_redirects is disabled

[d0gkiller87]

Description

When the server responds with a malformed Location header (for example, some misconfigured IIS web server sets it to C:\intput\wwwroot\xxx.aspx) in 30X redirections, httpx will parse it and fails.
The problem is that this behavior cannot be disabled by setting follow_redirects to False.

Reproduce Setup

Server (flask)

from flask import Flask, redirect

app = Flask(__name__)

@app.route('/')
def hello():
    return redirect("C:\\",code=302)

if __name__ == '__main__':
  app.run(host='127.0.0.1', port=8080)

Client (httpx)

import httpx
httpx.get('http://127.0.0.1:8080/', follow_redirects=False)

Result

>>> httpx.get('http://127.0.0.1:8080/')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/python/lib/site-packages/httpx/_api.py", line 196, in get
    return request(
  File "/usr/python/lib/site-packages/httpx/_api.py", line 104, in request
    return client.request(
  File "/usr/python/lib/site-packages/httpx/_client.py", line 828, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/python/lib/site-packages/httpx/_client.py", line 915, in send
    response = self._send_handling_auth(
  File "/usr/python/lib/site-packages/httpx/_client.py", line 943, in _send_handling_auth
    response = self._send_handling_redirects(
  File "/usr/python/lib/site-packages/httpx/_client.py", line 1000, in _send_handling_redirects
    raise exc
  File "/usr/python/lib/site-packages/httpx/_client.py", line 989, in _se    request = self._build_redirect_request(request, response)
  File "/usr/python/lib/site-packages/httpx/_client.py", line 466, in _build_redirect_request
    url = self._redirect_url(request, response)
  File "/usr/python/lib/site-packages/httpx/_client.py", line 518, in _redirect_url
    url = url.copy_with(host=request.url.host)
  File "/usr/python/lib/site-packages/httpx/_urls.py", line 354, in copy_with
    return URL(self, **kwargs)
  File "/usr/python/lib/site-packages/httpx/_urls.py", line 117, in __init__
    self._uri_reference = url._uri_reference.copy_with(**kwargs)
  File "/usr/python/lib/site-packages/httpx/_urlparse.py", line 134, in copy_with
    return urlparse("", **defaults)
  File "/usr/python/lib/site-packages/httpx/_urlparse.py", line 252, in urlparse
    validate_path(path, has_scheme=has_scheme, has_authority=has_authority)
  File "/usr/python/lib/site-packages/httpx/_urlparse.py", line 370, in validate_path
    raise InvalidURL("For absolute URLs, path must be empty or begin with '/'")
httpx.InvalidURL: For absolute URLs, path must be empty or begin with '/'

Discussion

Here's the relevant function that handles redirection responses: https://github.com/encode/httpx/blob/0.27.0/httpx/_client.py#L991

def _send_handling_redirects(
    self,
    request: Request,
    follow_redirects: bool,
    history: typing.List[Response],
) -> Response:
    while True:
        if len(history) > self.max_redirects:
            raise TooManyRedirects(
                "Exceeded maximum allowed redirects.", request=request
            )

        for hook in self._event_hooks["request"]:
            hook(request)

        response = self._send_single_request(request)
        try:
            for hook in self._event_hooks["response"]:
                hook(response)
            response.history = list(history)

            if not response.has_redirect_location:
                return response

            request = self._build_redirect_request(request, response)
            history = history + [response]

            if follow_redirects:
                response.read()
            else:
                response.next_request = request
                return response

        except BaseException as exc:
            response.close()
            raise exc

As we can see, follow_redirects is only used after parsing the Location header and checking infinite recursions.
In my opinion, if the user decides to handle the redirection manually (i.e. by setting follow_redirects to False), httpx shouldn't inspect the Location header then fails. Any thoughts?

[devl00p]

Proposed PR: #3706