Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions llamacppllmbasev3/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
apiVersion: v2
appVersion: b9879
appVersion: b10068
description: Generic llama.cpp + llm-init base; the GGUF model is supplied at install time via env
name: llamacppllmbasev3
type: application
version: 1.2.22
version: 1.2.24
12 changes: 7 additions & 5 deletions llamacppllmbasev3/OlaresManifest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ metadata:
description: "Generic llama.cpp engine base. Pick any GGUF model at install via env."
appid: llamacppllmbasev3
title: llama.cpp Engine Base
version: '1.2.22'
version: '1.2.24'
categories:
- AI
sharedEntrances:
Expand All @@ -33,9 +33,11 @@ workloadReplicas:
llminit: 1
spec:
onlyAdmin: true
versionName: 'b9879'
versionName: 'b10068'
upgradeDescription: |
Drop accelerator mode `cpu` (GPU-only: `nvidia` / `nvidia-gb10`). Chart 1.2.22.
Bump llama.cpp image to server-cuda12-b10068.

Move HF_TOKEN into a Kubernetes Secret and inject via secretKeyRef (engine + llm-init). Chart 1.2.24.
fullDescription: |
**IMPORTANT NOTE**
This app is a template and cannot be used on its own. To use it, set the environment variables below to connect a specific model.
Expand Down Expand Up @@ -186,12 +188,12 @@ envs:
required: false
applyOnChange: true
valueFrom:
envName: OLARES_USER_HUGGINGFACE_SERVICE
envName: OLARES_SYSTEM_HUGGINGFACE_SERVICE
- envName: HF_TOKEN
required: false
applyOnChange: true
valueFrom:
envName: OLARES_USER_HUGGINGFACE_TOKEN
envName: OLARES_SYSTEM_HUGGINGFACE_TOKEN
- envName: LLAMACPP_CPU_REQUEST # default: 200m
required: true
default: "200m"
Expand Down
7 changes: 5 additions & 2 deletions llamacppllmbasev3/templates/llamacpp.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ spec:
defaultMode: 0555
containers:
- name: llamacpp
image: docker.io/beclab/ggml-org-llama.cpp:server-cuda12-b9879
image: docker.io/beclab/ggml-org-llama.cpp:server-cuda12-b10068
imagePullPolicy: IfNotPresent
command: ["/bin/sh", "/llm-init/wrappers/llamacpp.sh"]
env:
Expand All @@ -95,7 +95,10 @@ spec:
{{- end }}
{{- if $hfToken }}
- name: HF_TOKEN
value: "{{ $hfToken }}"
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-secrets
key: HF_TOKEN
{{- end }}
- name: PYTHONUNBUFFERED
value: "1"
Expand Down
5 changes: 4 additions & 1 deletion llamacppllmbasev3/templates/llm-init.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,10 @@ spec:
{{- end }}
{{- if $hfToken }}
- name: HF_TOKEN
value: "{{ $hfToken }}"
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-secrets
key: HF_TOKEN
{{- end }}
- name: RUN_DIR
value: /run/llm-init
Expand Down
15 changes: 15 additions & 0 deletions llamacppllmbasev3/templates/secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{{- $oe := .Values.olaresEnv | default dict -}}
{{- $hfToken := $oe.HF_TOKEN | default "" -}}
{{- if $hfToken }}
apiVersion: v1
kind: Secret
metadata:
name: {{ .Release.Name }}-secrets
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: llamacppllmbasev3
app.kubernetes.io/instance: {{ .Release.Name }}
type: Opaque
stringData:
HF_TOKEN: {{ $hfToken | quote }}
{{- end }}