Provide an option for merging of URL query string with params

[jhominal]

In light of the resolution of #3433, as I still believe that it is a common need to merge the query string from the URL with a params object, I propose implementing an option that would allow API users to specify the way that a query string in the URL will interact with a params dictionary.

• I would call that url_query_params_mode;
• I would like to implement 3 modes for this parameter:
  • replace: that is the behaviour from httpx >=0.28 - the query component from the URL is replaced wholesale;
  • merge: that is an approximation of the behaviour of httpx<0.28:
    • The query string from the URL is parsed, as a dictionary;
    • The parameters specified in client.params are then merged with the dictionary from the previous step
      • One of the peculiarities of merging is that, in the case where a parameter to merge has the same name as a parameter in the source, the incoming value will replace the previous one (as if merging dictionaries using |);
    • The parameters specified in request.params are then also merged with the dictionary from the previous step;
    • That final dictionary is then serialised to a query string, and set on the URL;
  • append: that is the behaviour that I personally find the most interesting, basically:
    • The query string from the URL is kept as is (no attempt is made to parse it);
    • The parameters specified in client.params are serialised to a query string, and appended to the URL;
    • The parameters specified in request.params are serialised to a query string, and appended to the URL:
    • I find that mode appealing for the following reasons:
      • This mode does not change any of the URL parameters that are already set on the URL string representation;
      • All parameters and all values specified at any level (client or request) will be found in the final URL;

What do you think? If anyone at @encode/maintainers thinks that idea is interesting, I would be happy to work on a PR.

[Weugene]

Hi,

I have noticed a different behaviour from the requests package too, as you @jhominal. It overrides parameters that are already included in the URL with parameters that have been sent as params. A similar issue was opened and resolved in 2019: #652

The URL is sent to me by another service, therefore, there is no chance to separately send the pure URL and query parameters. It would be nice to add this feature. In this line of code, problems appear:
self.url = URL(url) if params is None else URL(url, params=params)

Let me copy the script that has been reported in the issue:

Here is a simple script:

import requests
import httpx
import asyncio

URL = 'https://httpbin.org/get?a=1'

r = requests.get(URL, params={'b': 2})
print('requests', r.json()['args'])

def f():
    r = httpx.get(URL, params={'b': 2})
    print('httpx', r.json()['args'])
f()

Results:

requests {'a': '1', 'b': '2'}
httpx {'b': '2'}

[Weugene]

Hello @jhominal,
If I suggest a pull request, can somebody review it? This feature is required in our company. We use SIREN, which is a specification for representing entities in hypermedia APIs. Our services use actions instead of hardcoded links. Some links have query parameters. From version httpx>0.25.2, it removes all previous parameters and adds new query parameters, making the link unusable. This feature is really required, can we add a query parameter?

Best regards,
Evgenii

[jhominal]

As the person who suggested this idea, I am of course in favor of it, but what I am trying to do here is to get a signal that I would not be the only person in encode interested in such a feature. However I am not sure how to navigate the situation as the discussion has died down.

[Kludex]

I think the previous behavior was also more intuitive.

I think this comment is relevant: #3433 (comment)