🐛 [Bug]: Every model I try results in an endless stream of garbage output

[drichline]

Bug description

I've been running Serge for 2 years, and it's almost never seemed to work correctly. I've tried 7b and 13b versions of Vicuna, WizardLM, and llama2.

Steps to reproduce

Install Serge using podman-compose. Download one of the three models and give it a prompt

Environment Information

Podman on Void Linux with a 4th gen i7

Screenshots

Relevant log output

Confirmations

• I'm running the latest version of the main branch.
• I checked existing issues to see if this has already been described.

[gaby]

@drichline This project is no longer maintained. I recommend using Open-WebUI.