I've used https://github.com/PromtEngineer/localGPT for this and thought it was nice. So I packaged it in a docker container for easy use.

docker run -itd --gpus all -p $(PORT):5111 --name llm-local-wizardlm-7b obald/llm-launcher:0.0.2

just use localhost:port in the browser and upload docs then ask questions in the gui.

Really nice for easy lookup of rules in boardgames and such. As it provides the relevant text from the docs in addition to the query answer.

https://gitlab.com/PeterHedman/llm-local

I would take the d out of the -itd, otherwise I get no logs and can't see that it downloads pytorch_model before giving me access.

Otherwise, great job.

Good suggestion I am so used to running docker logs manually. But will update the git readme as you suggest.

I explored privateGPT today and gave it a source of documents in txt format. It worked well for some queries but for some it gave responses which were not part of the source documents. Any tips on fixing this? There is an open issue for this problem - https://github.com/imartinez/privateGPT/issues/517