Unsupported Model : Zephyr 'stablelm' GGUF #196

New issue

Open

opened 2023-12-14 04:33:37 -05:00 by Jonathanjordan21 · 0 comments

Jonathanjordan21 commented

2023-12-14 04:33:37 -05:00

(Migrated from github.com)

I wonder why Zephyr is not yet supported in ctransformers.

from ctransformers import AutoModelForCausalLM

m = AutoModelForCausalLM.from_pretrained('TheBloke/stablelm-zephyr-3b-GGUF', model_file='stablelm-zephyr-3b.Q4_K_M.gguf')

The above code actually downloaded the file from huggingface repo, but it output this error

RuntimeError: Failed to create LLM 'stablelm' from '/root/.cache/huggingface/hub/models--TheBloke--stablelm-zephyr-3b-GGUF/blobs/74b2613b6e89d904a2ea38d56d233e4d2ca2fe663844b2e7aa90e769d359061b'.

I wonder why Zephyr is not yet supported in ctransformers. ``` from ctransformers import AutoModelForCausalLM m = AutoModelForCausalLM.from_pretrained('TheBloke/stablelm-zephyr-3b-GGUF', model_file='stablelm-zephyr-3b.Q4_K_M.gguf') ``` The above code actually downloaded the file from huggingface repo, but it output this error ``` RuntimeError: Failed to create LLM 'stablelm' from '/root/.cache/huggingface/hub/models--TheBloke--stablelm-zephyr-3b-GGUF/blobs/74b2613b6e89d904a2ea38d56d233e4d2ca2fe663844b2e7aa90e769d359061b'. ```

👍 3 😕 1

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

marella/ctransformers#196

No description provided.

Rows
Columns