diff options
| author | Zenix <zenixls2@gmail.com> | 2023-05-20 18:02:48 +0900 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-05-20 12:02:48 +0300 |
| commit | 07e9ace0f9da424d82e75df969642522880feb92 (patch) | |
| tree | 652979898352212059f9bef9778e52367ac8f268 /README.md | |
| parent | ec2e10c4443209da56b431b24dd0845b60e757fb (diff) | |
feature : add blis and other BLAS implementation support (#1502)
* feature: add blis support
* feature: allow all BLA_VENDOR to be assigned in cmake arguments. align with whisper.cpp pr 927
* fix: version detection for BLA_SIZEOF_INTEGER, recover min version of cmake
* Fix typo in INTEGER
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 19 |
1 files changed, 17 insertions, 2 deletions
@@ -56,7 +56,7 @@ The main goal of `llama.cpp` is to run the LLaMA model using 4-bit integer quant - Mixed F16 / F32 precision - 4-bit, 5-bit and 8-bit integer quantization support - Runs on the CPU -- OpenBLAS support +- Supports OpenBLAS/Apple BLAS/ARM Performance Lib/ATLAS/BLIS/Intel MKL/NVHPC/ACML/SCSL/SGIMATH and [more](https://cmake.org/cmake/help/latest/module/FindBLAS.html#blas-lapack-vendors) in BLAS - cuBLAS and CLBlast support The original implementation of `llama.cpp` was [hacked in an evening](https://github.com/ggerganov/llama.cpp/issues/33#issuecomment-1465108022). @@ -274,10 +274,25 @@ Building the program with BLAS support may lead to some performance improvements ```bash mkdir build cd build - cmake .. -DLLAMA_OPENBLAS=ON + cmake .. -DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS cmake --build . --config Release ``` +- BLIS + + Check [BLIS.md](BLIS.md) for more information. + +- Intel MKL + + By default, `LLAMA_BLAS_VENDOR` is set to `Generic`, so if you already sourced intel environment script and assign `-DLLAMA_BLAS=ON` in cmake, the mkl version of Blas will automatically been selected. You may also specify it by: + + ```bash + mkdir build + cd build + cmake .. -DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=Intel10_64lp -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx + cmake --build . -config Release + ``` + - cuBLAS This provides BLAS acceleration using the CUDA cores of your Nvidia GPU. Make sure to have the CUDA toolkit installed. You can download it from your Linux distro's package manager or from here: [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads). |