Skip to content
Discussion options

You must be logged in to vote

llama-cpp-python 0.3.37 is now available on pypi.flet.dev for Android and iOS (flet-dev/mobile-forge#136), and it can load LFM3 models. This release bundles a llama.cpp from after b11960, the build that added the lfm3 architecture.

To use it, pin the version in your pyproject.toml:

dependencies = [
    "flet",
    "llama-cpp-python==0.3.37",
]

I checked it with a small Flet app that loads LFM3-1.7B-Q4_K_M.gguf and answers a prompt, on an Android arm64 emulator and an iPhone simulator. Both load the model as lfm3 and reply correctly. The GGUF is about 1.1 GB, so download it into FLET_APP_STORAGE_DATA on first run instead of bundling it with the app. Load the model and run inference in page…

Replies: 2 comments 1 reply

Comment options

You must be logged in to vote
1 reply
@yunguangli
Comment options

Comment options

You must be logged in to vote
0 replies
Answer selected by ndonkoHenri
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants