add claude and local llm functionality

[ginberg]

see #46

[ginberg]

did you have the possibility to try this out @Faptimus420?

[Faptimus420]

did you have the possibility to try this out @Faptimus420?

Unfortunately, not yet - we are missing some system-level dependencies required by one of the R packages required by the tool on our system at the moment. Will test once that's been resolved

[Faptimus420]

Hi @ginberg. Apologies this took so long, getting all the dependencies required by the lock file on our system took a few days, as we're working on an offline system.

Nonetheless, I got the package running. I can point it to our local ollama models - when I create a new chat, I get "✔ ollama chat created with llama3.1:8b". However, when I try to actually send a prompt, I get
Error in req_perform() at ellmer/R/httr2.R:23:3: ! HTTP 500 Internal Server Error. ℹ Failed to create new sequence: unable to create sampling context
This is only happening from within PatientGenerator - if I prompt the same model directly using the ellmer library, I get a response.

I only set the model, provider, and echo arguments, I did not touch jsonSchemaPath or system_prompt. Let me know if you have any ideas - I'll ask the sysadmin to see if I can get any more detailed logs.

[ginberg]

hi @Faptimus420, thanks for trying this out!
When searching for this error, I found a few github issues this one for example. But I am not totally sure, how we need to adjust this package. We will look into it more. But please let us know if you have any ideas.