API: Allow passing firstpass_image to the txt2img endpoint #16889
+2
−0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
I noticed a while ago that the docs for the txt2img generation endpoint has a
firstpass_imageparameter, and from what I could understand based off the source it was/is intended to effectively "skip" recreating the original image when using HiRes Fix since it has already been provided. Looked perfect for my application, although I couldn't quite figure out how to get the image passed in. Since it is listed in the docs as a string parameter, my first thought was that it was just the base64 string returned by txt2img, then I tried using a path to an existing image on disk. Both resulted in the following:{ "error": "ValueError", "detail": "", "body": "", "errors": "could not convert string to float: $INPUT" }(Where
$INPUTis whatever was passed into thefirstpass_imageparameter)After taking a closer look, it seemed to me the parameter was just never actually getting converted into an image to then be used by
processing.py. My PR just does a quick check when invoking the txt2img endpoint to see if the parameter was supplied, and if it was, then it will use the already existingdecode_base64_to_imagefunction to turn it back into an image.I've been using this for a few months now and have not discovered any ill-effects from the change. Passing in a malformed string to the
firstpass_imagewill result in a failure, just as it does before this patch. Wanted to make sure before submitting this PR, of course! The only thing I noticed was that when using HiRes Fix with this parameter passed in, the console assumes that it will need to render the first pass so the progress current and total step displays what would beoriginal steps + HR stepsas the total number of steps, and thus looks like it stops half way - but the/sdapi/v1/progressAPI endpoint correctly returns the right number of steps so I do not consider this to be an issue.I suppose the only other issue that could exist is if I just have the completely wrong idea of how to use this parameter, then this change would overwrite the usage of that - but I could not figure out how the parameter is supposed to be used otherwise. Should that be the case, then I think this would still be immensely valuable to have as another parameter.
Since the existing
decode_base64_to_imagefunction is already used, in theory you could also pass an http/https link to an image to the parameter and have it work as well, so long asapi_enable_requestsis turned on which is a cool bonus, though admittedly I have not tested this as that doesn't line up with my current use-case (which is just "replaying" a previous request, but withfirstpass_imagetossed in that has the contents of the image string returned by the previous request).Lastly to note, in regards to the checklist below: I did run both the linter and test runner, and they both showed issues, but none of them seem to be a result of my change (the output is the same before and after the patch). For the tests, I believe it is due to my PC using an AMD graphics card instead of an Nvidia card given the
HIPerrors present. The following gist documents the results of both before and after.Simple enough of a change, but if I've missed something, please let me know.
Screenshots/videos:
N/A (I believe?) - No UI tweaks are made with this PR.
Checklist: