.profile/recipes/llama.cpp/02_run-models-on-llama-cpp.md

Run Models on llama.cpp

This recipe runs a model with llama-server inside a persistent tmux session, downloading it directly from Hugging Face.

Install tmux

sudo apt install -y tmux

Start a tmux Session

tmux new -s llama

Run the Model Server

Replace:

• <REPO> with the Hugging Face model repo (e.g. deepreinforce-ai/Ornith-1.0-9B-GGUF)
• <HOST> by default 0.0.0.0 hear all network interfaces
• <PORT> with the port to serve on (e.g. 8000)
• <CONTEXT_SIZE> with the context window size (e.g. 262144)

llama-server -hf <REPO> --host <HOST> --port <PORT> -c <CONTEXT_SIZE>

Detach from the Session

Press:

Ctrl+B, D

List tmux Sessions

tmux ls

Expected output:

llama: 1 windows (created ...)

Reattach to the Session

tmux attach -t llama

Verify Server is Running

Replace <PORT> with the configured port.

curl http://localhost:<PORT>/health

Expected output:

{"status":"ok"}

Troubleshooting

Port already in use

Verify that no other process is bound to the port.

sudo ss -tulpn | grep <PORT>

Session not found

List active sessions and verify the name.

tmux ls

Server not responding

Reattach to the session and check the server logs.

tmux attach -t llama