Go to file
2026-05-02 15:06:09 +03:00
.github ci : re-enable mac workflows (#0) 2026-04-14 15:27:40 +03:00
ci ggml : fix conv2d_dw SVE path (#1380) 2025-11-04 20:40:52 +02:00
cmake cmake : add FindNCCL.cmake (#0) 2026-04-21 10:59:04 +03:00
docs Update gguf specification to synchronize the ggml_types declaration shown in the doc with the actual one. (#1342) 2025-09-16 13:42:24 +02:00
examples examples : update to Q1_0 2026-05-01 11:53:27 +03:00
include CUDA: manage NCCL communicators in context (llama/21891) 2026-04-21 10:59:04 +03:00
scripts sync : whisper.cpp 2026-05-02 15:06:09 +03:00
src ggml : remove obsolete rms_norm.wgsl (#0) 2026-05-02 08:52:03 +03:00
tests vulkan: Support asymmetric FA in coopmat2 path (llama/21753) 2026-05-02 08:41:22 +03:00
.editorconfig gguf : add file format specification (#302) 2023-11-01 19:01:49 +02:00
.gitignore gitignore : ignore idea files (#1339) 2025-09-09 13:17:07 +02:00
.gitmodules git : remove kompute submodule (#1300) 2025-07-12 16:12:49 +03:00
AUTHORS authors : update 2025-02-04 13:03:55 +02:00
CMakeLists.txt ggml : bump version to 0.10.2 (#1474) 2026-05-02 08:45:46 +03:00
CONTRIBUTING.md contrib : recommend PRs to llama.cpp (#1312) 2025-07-25 07:05:38 +03:00
ggml.pc.in pkg-config: include the new GGML_VERSION as a version (#1348) 2025-09-25 18:59:38 +02:00
LICENSE docs : Minor cleanups (llama/19252) 2026-02-07 10:37:38 +02:00
README.md readme : simplify 2026-03-16 14:54:53 +02:00
requirements.txt ci : update requirements.txt 2024-12-03 21:05:37 +02:00

ggml

Manifesto

Tensor library for machine learning

Note that this project is under active development.
Some of the development is currently happening in the llama.cpp and whisper.cpp repos

Features

  • Low-level cross-platform implementation
  • Integer quantization support
  • Broad hardware support
  • Automatic differentiation
  • ADAM and L-BFGS optimizers
  • No third-party dependencies
  • Zero memory allocations during runtime

Build

git clone https://github.com/ggml-org/ggml
cd ggml

# install python dependencies in a virtual environment
python3.10 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# build the examples
mkdir build && cd build
cmake ..
cmake --build . --config Release -j 8

GPT inference (example)

# run the GPT-2 small 117M model
../examples/gpt-2/download-ggml-model.sh 117M
./bin/gpt-2-backend -m models/gpt-2-117M/ggml-model.bin -p "This is an example"

For more information, checkout the corresponding programs in the examples folder.

Resources