git/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp synced 2026-03-26 09:00:59 +01:00
Code Issues Packages Projects Releases Wiki Activity
a28e3c7567
llama.cpp/docs/ops.md
Reese Levine fd57b24c0f
ggml webgpu: unary op suppport, code refactoring, ops support (#17764) 
* Squashed commit of the following:

commit b3c6bf4b0450d8d452b934df27a0fb7cb53cd755
Author: Abhijit Ramesh <abhijitramesh2k@gmail.com>
Date:   Mon Dec 1 18:29:00 2025 -0800

    ggml webgpu: fix xielu parameter passing (#11)

    The XIELU operation was incorrectly using static_cast to convert
    float parameters to uint32_t, which converted numeric values instead
    of preserving IEEE 754 bit patterns. This caused incorrect values
    to be interpreted by the GPU shader.

    * Use reinterpret_cast to preserve float bit patterns when passing
      through uint32_t params buffer
    * Update WGSL shader parameter types from u32 to f32
    * Re-enable XIELU support (was disabled due to numerical issues)

    Fixes NMSE test failures for XIELU operation on WebGPU backend.

commit 5ca9b5e49e
Author: neha-ha <137219201+neha-ha@users.noreply.github.com>
Date:   Tue Nov 18 12:17:00 2025 -0800

    Refactored pipelines and workgroup calculations (#10)

    * refactored pipelines

    * refactored workgroup calculation

    * removed commented out block of prior maps

    * Clean up ceiling division pattern

    ---------

    Co-authored-by: Neha Abbas <nehaabbas@eduroam-169-233-141-223.ucsc.edu>
    Co-authored-by: Reese Levine <reeselevine1@gmail.com>

Author: James Contini <jamescontini@gmail.com>
Date:   Wed Oct 29 23:13:06 2025 -0700

    formatted embed wgsl and ggml-webgpu.cpp

commit e1f6baea31
Author: James Contini <jamescontini@gmail.com>
Date:   Wed Oct 29 23:08:37 2025 -0700

    implemented REPL_Template support and removed bug in unary operators kernel

commit 8c70b8fece
Author: James Contini <jamescontini@gmail.com>
Date:   Wed Oct 15 16:14:20 2025 -0700

    responded and dealt with PR comments

commit f9282c660c
Author: James Contini <jamescontini@gmail.com>
Date:   Sun Oct 12 13:41:41 2025 -0700

    removed unnecesarry checking if node->src[1] exists for unary operators

commit 4cf28d7dec
Author: James Contini <jamescontini@gmail.com>
Date:   Sun Oct 12 13:32:45 2025 -0700

    All operators (inlcluding xielu) working

commit 74c6add176
Author: James Contini <jamescontini@gmail.com>
Date:   Fri Oct 10 13:16:48 2025 -0700

    fixed autoconfig

commit 362749910b
Author: James Contini <jamescontini@gmail.com>
Date:   Fri Oct 10 13:10:46 2025 -0700

    removed vestigial files

commit cb08583337
Author: James Contini <jamescontini@gmail.com>
Date:   Fri Oct 10 12:59:32 2025 -0700

    abides by editor-config

commit 5360e2852a
Author: James Contini <jamescontini@gmail.com>
Date:   Fri Oct 10 12:45:57 2025 -0700

    rms_norm double declaration bug atoned

commit 7b09baa4aa
Merge: 8a6ec843 74b8fc17
Author: James Contini <jamescontini@gmail.com>
Date:   Fri Oct 10 11:50:03 2025 -0700

    resolving merge conflicts

commit 8a6ec843a5
Author: James Contini <jamescontini@gmail.com>
Date:   Wed Oct 8 18:06:47 2025 -0700

    unary operators pass ggml tests

commit c3ae38278a
Author: James Contini <jamescontini@gmail.com>
Date:   Wed Oct 1 16:22:40 2025 -0700

    neg passes backend test

commit aa1c9b2f88
Author: James Contini <jamescontini@gmail.com>
Date:   Tue Sep 30 23:55:27 2025 -0700

    neg f16xf32xip builds and runs, havent actually ran a model that uses neg kernel yet though

Co-authored-by: James Contini <jamescontini@gmail.com>
Co-authored-by: Neha Abbas <neabbas@ucsc.edu>
Co-authored-by: Abhijit Ramesh <abhijitramesh2k@gmail.com>

* Remove extra code and format

* Add ops documentation (finally)

* Update ggml/src/ggml-webgpu/wgsl-shaders/embed_wgsl.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: James Contini <jamescontini@gmail.com>
Co-authored-by: Neha Abbas <neabbas@ucsc.edu>
Co-authored-by: Abhijit Ramesh <abhijitramesh2k@gmail.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2025-12-05 12:25:51 -08:00

11 KiB
Raw Blame History

GGML Operations

List of GGML operations and backend support status.

How to add a backend to this table:

  1. Run test-backend-ops support --output csv with your backend name and redirect output to a csv file in docs/ops/ (e.g., docs/ops/CUDA.csv)
  2. Regenerate /docs/ops.md via ./scripts/create_ops_docs.py

Legend:

  • ✅ Fully supported by this backend
  • 🟡 Partially supported by this backend
  • ❌ Not supported by this backend
Operation BLAS CANN CPU CUDA Metal OpenCL SYCL Vulkan WebGPU zDNN
ABS ❌ ✅ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ❌
ACC ❌ ✅ ✅ ✅ ✅ ❌ ✅ ✅ ❌ ❌
ADD ❌ ✅ ✅ ✅ 🟡 🟡 ✅ ✅ ✅ ❌
ADD1 ❌ ✅ ✅ ✅ ❌ ❌ ✅ ✅ ❌ ❌
ADD_ID ❌ ❌ ✅ ✅ ✅ ❌ ❌ ✅ ❌ ❌
ARANGE ❌ ✅ ✅ ✅ ✅ ❌ ✅ ✅ ❌ ❌
ARGMAX ❌ ✅ ✅ ✅ ✅ ❌ ✅ ✅ ❌ ❌
ARGSORT ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
CEIL ❌ ❌ ✅ 🟡 ❌ ❌ 🟡 🟡 ❌ ❌
CLAMP ❌ ✅ ✅ ✅ 🟡 🟡 🟡 🟡 ❌ ❌
CONCAT ❌ ✅ ✅ 🟡 ✅ 🟡 ✅ ✅ ❌ ❌
CONT ❌ 🟡 ✅ ✅ ✅ 🟡 🟡 ✅ 🟡 ❌
CONV_2D ❌ ❌ ✅ ✅ ✅ ✅ ❌ ✅ ❌ ❌
CONV_2D_DW ❌ ❌ ✅ ✅ ❌ ❌ ❌ ✅ ❌ ❌
CONV_3D ❌ ❌ ✅ ❌ ❌ ❌ ❌ ❌ ❌ ❌
CONV_TRANSPOSE_1D ❌ ✅ ✅ ✅ ✅ ❌ ✅ ✅ ❌ ❌
CONV_TRANSPOSE_2D ❌ ❌ ✅ ✅ ✅ ❌ ❌ ✅ ❌ ❌
COS ❌ ✅ ✅ ✅ 🟡 ❌ 🟡 🟡 ❌ ❌
COUNT_EQUAL ❌ ✅ ✅ ✅ ❌ ❌ ✅ ✅ ❌ ❌
CPY ❌ 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 ❌
CROSS_ENTROPY_LOSS ❌ ❌ ✅ ✅ ❌ ❌ ❌ ❌ ❌ ❌
CROSS_ENTROPY_LOSS_BACK ❌ ❌ ✅ ✅ ❌ ❌ ❌ ❌ ❌ ❌
CUMSUM ❌ ❌ ✅ ❌ ✅ ❌ ❌ ✅ ❌ ❌
DIAG_MASK_INF ❌ ✅ ✅ ✅ ❌ 🟡 ✅ ✅ ❌ ❌
DIV ❌ ✅ ✅ ✅ 🟡 🟡 ✅ ✅ ✅ ❌
DUP ❌ ✅ ✅ 🟡 🟡 🟡 ✅ ✅ ❌ ❌
ELU ❌ ✅ ✅ 🟡 🟡 ❌ ✅ ❌ ✅ ❌
EXP ❌ ✅ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ❌
EXPM1 ❌ ❌ ✅ 🟡 🟡 ❌ ❌ ❌ ❌ ❌
FILL ❌ ❌ ✅ ❌ ✅ ❌ ❌ ✅ ❌ ❌
FLASH_ATTN_EXT ❌ 🟡 ✅ 🟡 🟡 ❌ ❌ 🟡 ❌ ❌
FLOOR ❌ ❌ ✅ 🟡 ❌ ❌ 🟡 🟡 ❌ ❌
GATED_LINEAR_ATTN ❌ ❌ ✅ ✅ ❌ ❌ ✅ ❌ ❌ ❌
GEGLU ❌ ✅ ✅ ✅ 🟡 ✅ ✅ 🟡 ✅ ❌
GEGLU_ERF ❌ ✅ ✅ ✅ 🟡 ✅ ✅ 🟡 ✅ ❌
GEGLU_QUICK ❌ ✅ ✅ ✅ 🟡 ✅ ✅ 🟡 ✅ ❌
GELU ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ❌
GELU_ERF ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ❌
GELU_QUICK ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ❌
GET_ROWS ❌ 🟡 ✅ 🟡 ✅ 🟡 🟡 🟡 🟡 ❌
GET_ROWS_BACK ❌ ❌ 🟡 🟡 ❌ ❌ ❌ ❌ ❌ ❌
GROUP_NORM ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
GROUP_NORM_MUL_ADD ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌
HARDSIGMOID ❌ ✅ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ❌
HARDSWISH ❌ ✅ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ❌
IM2COL ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
IM2COL_3D ❌ ❌ ✅ ✅ ❌ ❌ ❌ ✅ ❌ ❌
L2_NORM ❌ ❌ ✅ ✅ ✅ ❌ ✅ ✅ ❌ ❌
LEAKY_RELU ❌ ✅ ✅ ✅ 🟡 ❌ ✅ 🟡 ❌ ❌
LOG ❌ ✅ ✅ ✅ 🟡 ❌ 🟡 ✅ ❌ ❌
MEAN ❌ ✅ ✅ ✅ ✅ ❌ ✅ ✅ ❌ ❌
MUL ❌ ✅ ✅ ✅ 🟡 🟡 ✅ ✅ ✅ ❌
MUL_MAT 🟡 🟡 🟡 🟡 ✅ 🟡 🟡 🟡 🟡 🟡
MUL_MAT_ID ❌ 🟡 ✅ ✅ ✅ 🟡 🟡 ✅ ❌ ❌
NEG ❌ ✅ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ❌
NORM ❌ ✅ ✅ ✅ ✅ ✅ ✅ 🟡 ❌ ❌
NORM_MUL_ADD ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌
OPT_STEP_ADAMW ❌ ❌ ✅ ✅ ✅ ❌ ❌ ✅ ❌ ❌
OPT_STEP_SGD ❌ ❌ ✅ ✅ ✅ ❌ ❌ ✅ ❌ ❌
OUT_PROD 🟡 ❌ 🟡 🟡 ❌ ❌ 🟡 ❌ ❌ ❌
PAD ❌ ✅ ✅ 🟡 🟡 ✅ 🟡 ✅ ❌ ❌
PAD_REFLECT_1D ❌ ✅ ✅ ✅ ✅ ❌ ✅ ❌ ❌ ❌
POOL_2D ❌ 🟡 ✅ ✅ ✅ ❌ ✅ ✅ ❌ ❌
REGLU ❌ ✅ ✅ ✅ 🟡 ✅ ✅ 🟡 ✅ ❌
RELU ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ❌
REPEAT ❌ ✅ ✅ 🟡 ✅ 🟡 ✅ 🟡 ❌ ❌
REPEAT_BACK ❌ ❌ ✅ ✅ ❌ ❌ ✅ ✅ ❌ ❌
RMS_NORM ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌
RMS_NORM_BACK ❌ ❌ ✅ ✅ ❌ ❌ ✅ ✅ ❌ ❌
RMS_NORM_MUL_ADD ❌ ✅ ❌ ❌ ❌ ✅ ❌ ❌ ❌ ❌
ROLL ❌ ❌ ✅ ✅ ❌ ❌ ✅ ✅ ❌ ❌
ROPE ❌ 🟡 ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌
ROPE_BACK ❌ ❌ ✅ ✅ ❌ ❌ ❌ ✅ ❌ ❌
ROUND ❌ ❌ ✅ 🟡 ❌ ❌ 🟡 🟡 ❌ ❌
RWKV_WKV6 ❌ ❌ ✅ ✅ ✅ ❌ ✅ ✅ ❌ ❌
RWKV_WKV7 ❌ ❌ ✅ ✅ ✅ ❌ ✅ ✅ ❌ ❌
SCALE ❌ 🟡 ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌
SET ❌ ❌ ✅ ✅ ❌ ❌ 🟡 ❌ ❌ ❌
SET_ROWS ❌ ❌ 🟡 🟡 🟡 🟡 🟡 🟡 🟡 ❌
SGN ❌ ✅ ✅ 🟡 🟡 ❌ ✅ ❌ ✅ ❌
SIGMOID ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ❌
SILU ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ❌
SILU_BACK ❌ ❌ ✅ ✅ ❌ ❌ ❌ ✅ ❌ ❌
SIN ❌ ✅ ✅ ✅ 🟡 ❌ 🟡 🟡 ❌ ❌
SOFTCAP ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌
SOFTPLUS ❌ ❌ ✅ 🟡 🟡 ❌ ❌ 🟡 ❌ ❌
SOFT_MAX ❌ 🟡 ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌
SOFT_MAX_BACK ❌ ❌ 🟡 🟡 ❌ ❌ 🟡 ✅ ❌ ❌
SOLVE_TRI ❌ ❌ ✅ ❌ ❌ ❌ ❌ 🟡 ❌ ❌
SQR ❌ ✅ ✅ ✅ 🟡 ❌ 🟡 🟡 ❌ ❌
SQRT ❌ ✅ ✅ ✅ 🟡 ❌ 🟡 🟡 ❌ ❌
SSM_CONV ❌ ❌ ✅ ✅ ✅ ❌ ✅ ✅ ❌ ❌
SSM_SCAN ❌ ❌ ✅ ✅ ✅ ❌ ❌ 🟡 ❌ ❌
STEP ❌ ✅ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ❌
SUB ❌ ✅ ✅ ✅ 🟡 🟡 ✅ ✅ ✅ ❌
SUM ❌ ✅ ✅ 🟡 🟡 ❌ 🟡 🟡 ❌ ❌
SUM_ROWS ❌ ✅ ✅ 🟡 ✅ ✅ 🟡 ✅ ❌ ❌
SWIGLU ❌ ✅ ✅ ✅ 🟡 ✅ ✅ 🟡 ✅ ❌
SWIGLU_OAI ❌ ❌ ✅ ✅ ✅ ❌ ❌ 🟡 ✅ ❌
TANH ❌ ✅ ✅ 🟡 🟡 ✅ ✅ 🟡 ✅ ❌
TIMESTEP_EMBEDDING ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
TOP_K ❌ ❌ ❌ ❌ ✅ ❌ ❌ 🟡 ❌ ❌
TRI ❌ ❌ ✅ ❌ ✅ ❌ ❌ ✅ ❌ ❌
TRUNC ❌ ❌ ✅ 🟡 ❌ ❌ 🟡 🟡 ❌ ❌
UPSCALE ❌ 🟡 ✅ ✅ 🟡 ✅ 🟡 🟡 ❌ ❌
XIELU ❌ ❌ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ❌