Add support for Microsoft Phi-2 #197

New issue

Open

opened 2023-12-14 13:08:09 -05:00 by niutech · 0 comments

niutech commented

2023-12-14 13:08:09 -05:00

(Migrated from github.com)

When running:

from ctransformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("radames/phi-2-quantized", hf=True)

I get:
RuntimeError: Failed to create LLM 'phi-msft' from '/root/.cache/huggingface/hub/models--radames--phi-2-quantized/blobs/77971e348da4b424832a089f812ba50dec2bd633ae39d26b4f2c89c0ff3dea27'.

However, when I use transformers, it runs fine (see Google Colab).

When running: ``` from ctransformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("radames/phi-2-quantized", hf=True) ``` I get: `RuntimeError: Failed to create LLM 'phi-msft' from '/root/.cache/huggingface/hub/models--radames--phi-2-quantized/blobs/77971e348da4b424832a089f812ba50dec2bd633ae39d26b4f2c89c0ff3dea27'.` However, when I use transformers, it runs fine (see [Google Colab](https://colab.research.google.com/drive/19yqSbRSqrrSHFmkyv1eW04PY20reCpvi)).

👍 6

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

marella/ctransformers#197

No description provided.

Rows
Columns