Repository navigation
llama.cpp v0.8.0 #7033
|
Some AI models , for example, LiquidAI LFM3 model card says that LFM3's unique tri-hybrid architecture is only supported in llama.cpp v0.8.0 (build b11960) and newer. The pre-compiled wheel hosted on Flet's binary package registry is locked at version 0.3.3. Could we possibly has a version for 0.80+ ? Thanks! |
Replies: 2 comments 1 reply
|
llama-cpp-python 0.3.36 is now available on pypi.flet.dev (recipe update), but it won't load LFM3 yet. The Our wheels build whatever llama.cpp upstream bundles, so LFM3 support depends on a llama-cpp-python release that includes |
|
llama-cpp-python 0.3.37 is now available on pypi.flet.dev for Android and iOS (flet-dev/mobile-forge#136), and it can load LFM3 models. This release bundles a llama.cpp from after To use it, pin the version in your dependencies = [
"flet",
"llama-cpp-python==0.3.37",
]I checked it with a small Flet app that loads Since you already run LFM3 through ONNX, it would be great to hear how llama.cpp's speed compares on your phone. Let me know if you hit anything. |
llama-cpp-python 0.3.37 is now available on pypi.flet.dev for Android and iOS (flet-dev/mobile-forge#136), and it can load LFM3 models. This release bundles a llama.cpp from after
b11960, the build that added thelfm3architecture.To use it, pin the version in your
pyproject.toml:I checked it with a small Flet app that loads
LFM3-1.7B-Q4_K_M.ggufand answers a prompt, on an Android arm64 emulator and an iPhone simulator. Both load the model aslfm3and reply correctly. The GGUF is about 1.1 GB, so download it intoFLET_APP_STORAGE_DATAon first run instead of bundling it with the app. Load the model and run inference inpage…