EasieST

Easy-to-use SillyTavern Starter, based on Docker Compose.

Usage

clone

git clone https://github.com/moeru-ai/easiest.git
cd easiest

config

cp intel.docker-compose.yml docker-compose.yml # Intel oneAPI SYCL
# cp rocm.docker-compose.yml docker-compose.yml # AMD ROCm (TODO)
# cp cuda.docker-compose.yml docker-compose.yml # NVIDIA CUDA (TODO)
# cp vulkan.docker-compose.yml docker-compose.yml # Vulkan (TODO)

nano docker-compose.yml # edit config

up / down

sudo docker compose up -d
# podman compose up -d # if you use podman

sudo docker compose down
# podman compose down # if you use podman

FAQ

Recommended Models (2024-08-09)

If this section hasn't been updated in a long time, I recommend looking for a new model.

For GGUF format, I recommend Q5_K_M or Q4_K_M (and imatrix).

• 8B: Lewdiculous/L3-8B-Stheno-v3.3-32K-GGUF-IQ-Imatrix
  • ST Presets:
    • [LLAMA-3-Context]Roleplay-v1.9.json => ./sillytavern/data/default-user/context
    • [LLAMA-3-Instruct]Roleplay-v1.9.json => ./sillytavern/data/default-user/instruct
    • [Sai]Roleplay.json => ./sillytavern/data/default-user/TextGen Settings (from here)
• 11B: mradermacher/Fimbulvetr-11B-v2.1-16K-i1-GGUF
  • ST Presets:
    • Lewdicu-Context-3.0.2-eros.json => ./sillytavern/data/default-user/context
    • Lewdicu-Instruct-Alpaca-3.0.2-tentative.json => ./sillytavern/data/default-user/instruct
    • Lewdicu-Samplers-3.0.2.json => ./sillytavern/data/default-user/TextGen Settings

Why llama.cpp?

llama.cpp provides the official docker image for Intel Arc Graphics.

I may change to ollama or koboldcpp later.

License

Unlicense