2 min read

Llama.cpp with Intel

Self hosted llm on Intel GPUs

Table of Contents

Info

Recently bought a Intel Arc B50 and decided to push it to the limits by trying to run some AI models on it

Setup

If you are using proxmox make sure you passthrough the GPU you wish to run AI on

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
features: fuse=1,nesting=1
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/card0 dev/dri/card0 none bind,optional,create=file
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
lxc.idmap: u 0 100000 262144
lxc.idmap: g 0 100000 44
lxc.idmap: g 44 44 1
lxc.idmap: g 45 100045 948
lxc.idmap: g 993 993 1
lxc.idmap: g 994 100994 261150

for rootless setup update

/etc/subgid

1
2
3
root:100000:262144
root:44:1
root:993:1

For simplicity and ease of upgrade I run this inside a docker compose container to keep it simple. First lets prep the environment with the neccesary packages

1
2
apt update && apt install curl intel-gpu-tools
curl -L https://get.docker.com |sh

Here is the docker-compose.yml I used

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
services:
  llama:
    image: ghcr.io/ggml-org/llama.cpp:server-intel
    container_name: llama
    restart: unless-stopped
    ports:
      - 11434:8080
    volumes:
      - ./models:/models
    devices:
      - /dev/dri/:/dev/dri/
    group_add:
      - "993" # render
      - "44" # video
    environment:
      # Hardware Acceleration
      SYCL_DEVICE_FILTER: "level_zero:gpu"
      ONEAPI_DEVICE_SELECTOR: "level_zero:gpu"
      SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS: "1"
      SYCL_PI_LEVEL_ZERO_USE_COPY_ENGINE: "1"

      # Performance Tuning B50
      GGML_SYCL_FORCE_MMQ: "1"
      GGML_SYCL_F16_AS_F32: "0"
      LLAMA_ARG_FLASH_ATTN: "on"
      LLAMA_ARG_CACHE_TYPE_K: "q4_0"
      LLAMA_ARG_CACHE_TYPE_V: "q4_0"
      LLAMA_ARG_BATCH: 2048
      LLAMA_ARG_UBATCH: 512
      LLAMA_ARG_N_PARALLEL: "1"
      # LLAMA_ARG_CTX_SIZE: "8000"
      LLAMA_ARG_CACHE_RAM: 0

      LLAMA_ARG_PORT: 8080
      LLAMA_ARG_MODEL: /models/gemma-4-12b-it-Q4_K_M.gguf
      LLAMA_ARG_MMAP: false
      LLAMA_ARG_N_GPU_LAYERS: "99"

Before we start the container make sure you download a model you with to run.

llama.cpp only supports “GGUF” models

Download models uwint “wget” from https://huggingface.co/models

Once you obtained a model, update teh LLAMA_ARG_MODEL to point to your downloaded file.

then run

1
docker compose up -d

once it is running go to page http://your_ip:11434 try a prompt. You can monitor your GPU using gputop