Compare commits
1
Commits
6cf5a08187
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f568e43362
|
@@ -0,0 +1,77 @@
|
||||
# Install llama.cpp
|
||||
|
||||
This recipe builds and installs llama.cpp from source on Debian/Ubuntu, including build dependencies and the `llama-server` binary.
|
||||
|
||||
## Install Build Dependencies
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y build-essential cmake git libcurl4-openssl-dev
|
||||
```
|
||||
|
||||
## Clone Repository
|
||||
|
||||
```bash
|
||||
git clone https://github.com/ggml-org/llama.cpp
|
||||
cd llama.cpp
|
||||
```
|
||||
|
||||
## Build Project
|
||||
|
||||
```bash
|
||||
cmake -B build
|
||||
cmake --build build --config Release -j $(nproc)
|
||||
```
|
||||
|
||||
## Install Binaries
|
||||
|
||||
```bash
|
||||
sudo cmake --install build
|
||||
sudo ldconfig
|
||||
```
|
||||
|
||||
## Verify Installation
|
||||
|
||||
```bash
|
||||
llama-server --version
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
version: xxxx (xxxxxxxx)
|
||||
```
|
||||
|
||||
## Verify Binary Location
|
||||
|
||||
```bash
|
||||
which llama-server
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
/usr/local/bin/llama-server
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Command not found
|
||||
|
||||
Verify that the binary was installed.
|
||||
|
||||
```bash
|
||||
ls /usr/local/bin/llama-server
|
||||
```
|
||||
|
||||
If it is missing, repeat the build and install steps.
|
||||
|
||||
### Build fails with missing dependency
|
||||
|
||||
Verify the build dependencies are installed.
|
||||
|
||||
```bash
|
||||
dpkg -l | grep -E "build-essential|cmake|libcurl4-openssl-dev"
|
||||
```
|
||||
|
||||
If any are missing, repeat the dependency installation step.
|
||||
@@ -0,0 +1,94 @@
|
||||
# Run Models on llama.cpp
|
||||
|
||||
This recipe runs a model with `llama-server` inside a persistent `tmux` session, downloading it directly from Hugging Face.
|
||||
|
||||
## Install tmux
|
||||
|
||||
```bash
|
||||
sudo apt install -y tmux
|
||||
```
|
||||
|
||||
## Start a tmux Session
|
||||
|
||||
```bash
|
||||
tmux new -s llama
|
||||
```
|
||||
|
||||
## Run the Model Server
|
||||
|
||||
Replace:
|
||||
|
||||
- `<REPO>` with the Hugging Face model repo (e.g. `deepreinforce-ai/Ornith-1.0-9B-GGUF`)
|
||||
- `<HOST>` by default `0.0.0.0` hear all network interfaces
|
||||
- `<PORT>` with the port to serve on (e.g. `8000`)
|
||||
- `<CONTEXT_SIZE>` with the context window size (e.g. `262144`)
|
||||
|
||||
```bash
|
||||
llama-server -hf <REPO> --host <HOST> --port <PORT> -c <CONTEXT_SIZE>
|
||||
```
|
||||
|
||||
## Detach from the Session
|
||||
|
||||
Press:
|
||||
|
||||
```text
|
||||
Ctrl+B, D
|
||||
```
|
||||
|
||||
## List tmux Sessions
|
||||
|
||||
```bash
|
||||
tmux ls
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
llama: 1 windows (created ...)
|
||||
```
|
||||
|
||||
## Reattach to the Session
|
||||
|
||||
```bash
|
||||
tmux attach -t llama
|
||||
```
|
||||
|
||||
## Verify Server is Running
|
||||
|
||||
Replace `<PORT>` with the configured port.
|
||||
|
||||
```bash
|
||||
curl http://localhost:<PORT>/health
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```json
|
||||
{"status":"ok"}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Port already in use
|
||||
|
||||
Verify that no other process is bound to the port.
|
||||
|
||||
```bash
|
||||
sudo ss -tulpn | grep <PORT>
|
||||
```
|
||||
|
||||
### Session not found
|
||||
|
||||
List active sessions and verify the name.
|
||||
|
||||
```bash
|
||||
tmux ls
|
||||
```
|
||||
|
||||
### Server not responding
|
||||
|
||||
Reattach to the session and check the server logs.
|
||||
|
||||
```bash
|
||||
tmux attach -t llama
|
||||
```
|
||||
@@ -0,0 +1,55 @@
|
||||
# Uninstall llama.cpp
|
||||
|
||||
This recipe completely removes llama.cpp from Ubuntu, including installed binaries, libraries, and the cloned source repository.
|
||||
|
||||
## Stop Running Sessions
|
||||
|
||||
```bash
|
||||
tmux kill-session -t llama
|
||||
```
|
||||
|
||||
## Remove Installed Binaries
|
||||
|
||||
```bash
|
||||
sudo rm -f /usr/local/bin/llama*
|
||||
```
|
||||
|
||||
## Remove Installed Libraries
|
||||
|
||||
```bash
|
||||
sudo rm -f /usr/local/lib/libllama*
|
||||
sudo rm -f /usr/local/lib/libggml*
|
||||
sudo rm -rf /usr/local/include/llama.h /usr/local/include/ggml*
|
||||
sudo ldconfig
|
||||
```
|
||||
|
||||
## Remove Cloned Repository
|
||||
|
||||
Replace `<PATH>` with the path where the repository was cloned (e.g. `~/llama.cpp`).
|
||||
|
||||
```bash
|
||||
rm -rf <PATH>
|
||||
```
|
||||
|
||||
## Remove Build Dependencies (Optional)
|
||||
|
||||
Only run this if the packages are not needed by other projects.
|
||||
|
||||
```bash
|
||||
sudo apt purge -y build-essential cmake libcurl4-openssl-dev
|
||||
sudo apt autoremove -y
|
||||
```
|
||||
|
||||
## Verify Removal
|
||||
|
||||
Check that the binary is no longer available.
|
||||
|
||||
```bash
|
||||
which llama
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```text
|
||||
(no output)
|
||||
```
|
||||
Reference in New Issue
Block a user