Compile llama.cpp, download a small GGML LLM model and you will have a quite intelligent assiatant running into your phone.
Would that actually be decent? Even 6b models feel way too rudimentary after experiencing 33+b models and/or chatgpt. I haven't tried those really scaled down and optimized models, though!
They're decent for text completion purposes, e.g. generating some corpspeak for an email, or generating some "wikipedia"-like text. You have to know how to write good prompts, don't try to treat it like ChatGPT.
For example if i want to know about the history of Puerto Rico I would put:
"The history of puerto rico starts in about 480BC when"
Decent enough for a model 50 times smaller than ChatGPT. I use orca_mini_3b.
I got llama to compile but it crashes every time I tried running it.
Do we have any llama apk/deb/flatpack/appimage where you just choose the llm to use?
lamacpp is kind of this. you give it a path to a model and a prompt. download a smallish model GGUF file by thebloke. lamacpp main exec has an interactive mode too which can function like a chatbot. i never built it on android though maybe you need android NDK
Compile llama.cpp, download a small GGML LLM model and you will have a quite intelligent assiatant running into your phone.
Would that actually be decent? Even 6b models feel way too rudimentary after experiencing 33+b models and/or chatgpt. I haven't tried those really scaled down and optimized models, though!
They're decent for text completion purposes, e.g. generating some corpspeak for an email, or generating some "wikipedia"-like text. You have to know how to write good prompts, don't try to treat it like ChatGPT.
For example if i want to know about the history of Puerto Rico I would put:
"The history of puerto rico starts in about 480BC when"
Decent enough for a model 50 times smaller than ChatGPT. I use orca_mini_3b.
I got llama to compile but it crashes every time I tried running it.
Do we have any llama apk/deb/flatpack/appimage where you just choose the llm to use?
Idk if GPT4All is what you want.
Do we have apt install gtp4qll?
https://gpt4all.io/index.html
Says ubuntu, works fine on my non-apt distro.
lamacpp is kind of this. you give it a path to a model and a prompt. download a smallish model GGUF file by thebloke. lamacpp main exec has an interactive mode too which can function like a chatbot. i never built it on android though maybe you need android NDK