Skip to content

Commit 1b51641

Browse files
committed
Add additional install instruction details, closes #2
1 parent db677ed commit 1b51641

File tree

1 file changed

+11
-2
lines changed

1 file changed

+11
-2
lines changed

README.md

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -65,10 +65,14 @@ This project also works well with papers from [PubMed](https://pubmed.ncbi.nlm.n
6565

6666
### Setup
6767

68-
The examples below use an AutoAWQ model, make sure to install that dependency first.
68+
Install the following.
6969

70-
```
70+
```bash
71+
# Change autoawq[kernels] to "autoawq autoawq-kernels" if a flash-attn error is raised
7172
pip install annotateai autoawq[kernels]
73+
74+
# macOS users should run this instead
75+
pip install annotateai llama-cpp-python
7276
```
7377

7478
The primary input parameter is the path to the LLM. This project is backed by [txtai](https://github.com/neuml/txtai) and it supports any [txtai-supported LLM](https://neuml.github.io/txtai/pipeline/text/llm/).
@@ -78,6 +82,11 @@ from annotateai import Annotate
7882

7983
# This model works well with medical and scientific literature
8084
annotate = Annotate("NeuML/Llama-3.1_OpenScholar-8B-AWQ")
85+
86+
# macOS users should run this instead
87+
annotate = Annotate(
88+
"bartowski/Llama-3.1_OpenScholar-8B-GGUF/Llama-3.1_OpenScholar-8B-Q4_K_M.gguf"
89+
)
8190
```
8291

8392
### Annotate paper "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks"

0 commit comments

Comments
 (0)