git/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp synced 2026-03-11 17:51:30 +01:00
Code Issues Packages Projects Releases Wiki Activity
8,096 Commits 549 Branches 5,693 Tags 1.3 GiB
8a70973557
Commit Graph

2 Commits

Author SHA1 Message Date
Diego Devesa
5931c1f233
ggml : add support for dynamic loading of backends (#10469) 
* ggml : add support for dynamic loading of backends

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-11-25 15:13:39 +01:00
Kawrakow
5ecff35151
Adding a simple program to measure speed of dot products (#1041) 
On my Mac, the direct Q4_1 product is marginally slower
(~69 vs ~55 us for Q4_0). The SIMD-ified ggml version
is now almost 2X slower (~121 us).

On a Ryzen 7950X CPU, the direct product for Q4_1 quantization
is faster than the AVX2 implementation (~60 vs ~62 us).

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 2023-04-18 19:00:14 +00:00