llama_model_load: error loading model: done_getting_tensors: wrong number of tensors; expected 292, got 291

[sunilmathew-mcw]

Error loading "meta-llama/Meta-Llama-3.1-8B-Instruct" that was converted to gguf using llama.cpp convert_hf_to_gguf.py using q_8 quantization

[dwight-foster]

If you update to the newest version of llama-cpp-python it should work.

[sunilmathew-mcw]

Hi @dwight-foster, thank you, that worked.

[asdkazmi]

Hello everyone, I'm using the cuda 11.8 because I need it for some other models. Before, llama cpp was working fine on cuda 11.8 but due to some recent update it has stopped to install on cuda 11.8. So, I found another way to install on cuda 11.8 using this command

LLAMA_CUBLAS="1" FORCE_CMAKE="1" CMAKE_ARGS="-DLLAMA_CUBLAS=on" python -m pip install llama-cpp-python --prefer-binary --extra-index-url=https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/basic/cu118

I can now install with this command but it fails when I try to load the model and give me the same error as described in this thread. I'm using this command to load the model

from llama_cpp import Llama
llama_gguf = Llama( model_path="llama-gguf/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf")

This is the currently installed version I've

llama_cpp_python              0.2.26+cu118