`network.responseCompleted` is emitted too early

[sadym-chromium]

According to the fetch specification, the WebDriver BiDi response completed steps should be invoked during "flush algorithm".

Currently, ChromiumBiDi relies on CDP event Network.responseReceived, which in turn relies on DataReceived, which:

is called each time a chunk of data is received

It looks like the right event is the Network.loadingFinished.

This seems to be the root cause of puppeteer/puppeteer#14017.

[OrKoN]

Currently, ChromiumBiDi relies on CDP event Network.responseReceived, which in turn relies on DataReceived, which:

this does not sound correct

[OrKoN]

Also, I don't think loadingFinished will be helpful. It emitted way too late (would be after navigation commit) and it does not get reported for redirect hops only for the final loading state of the resource request.

[sadym-chromium]

Currently, ChromiumBiDi relies on CDP event Network.responseReceived, which in turn relies on DataReceived, which:

this does not sound correct

I can easily have been mistaken in my research and would appreciate corrections.

[OrKoN]

I looked at the behavior and I observed the following:

1. Network.responseReceived is emitted once for a redirect chain. Network.responseReceivedExtraInfo are emitted for each redirect hop. I am not sure we correctly handle it right now, we need to add unit tests for this.

2. network.responseCompleted is indeed emitted too early. IIRC we should emit it for each redirect hop in WebDriver BiDi? In CDP we do not get a loading finished for each hop so loading finish is implicit. We get it once for the final response once the response finished streaming.

So LoadingFinished would help here but we need to handle redirects differently.

[OrKoN]

After trying to fix this with loading events, I think the following need to be clarified:

when is the flush algorithm actually called? is it called unconditionally or only if the body is consumed? for example, would void fetch(); result in the flush algorithm being invoked?

[OrKoN]

Spec question whatwg/fetch#1540 (comment)