bigcode-project/starcoder
1
0
Fork 0
Code Issues 93 Pull requests 8 Projects Releases Packages Wiki Activity Actions

Is finetune.py incompatible with older GPUs? #156

New issue
Open
opened 2024-03-19 09:26:59 -04:00 by umm-maybe · 0 comments
umm-maybe commented 2024-03-19 09:26:59 -04:00 (Migrated from github.com)
Copy link

Hi, while running on a Colab A100 instance I noticed that the VRAM consumed by finetune.py was only about 5 GB for starcoderbase-1b so I attempted it on my local machine which has a GTX 1070 card (8 GB VRAM, Pascal architecture). This didn't work, and I got a similar error when attempting again with either starcoderbase-1B or starcoderbase-3B on a larger, but still older GPU (NVIDIA Quadro P6000; 24GB VRAM). Here is the error:

RuntimeError: expected mat1 and mat2 to have the same dtype, but got: c10::Half != float

At first I thought this might be due to some difference in architecture (Pascal vs. Ampere) but this is contradicted by the fact that I have a Kaggle Code notebook which can fine-tune Starcoder with two P100 GPUs, which is also Pascal.

Is there some other explanation for this?

Longer stacktrace attached.
dump.txt

Hi, while running on a Colab A100 instance I noticed that the VRAM consumed by finetune.py was only about 5 GB for starcoderbase-1b so I attempted it on my local machine which has a GTX 1070 card (8 GB VRAM, Pascal architecture). This didn't work, and I got a similar error when attempting again with either starcoderbase-1B or starcoderbase-3B on a larger, but still older GPU (NVIDIA Quadro P6000; 24GB VRAM). Here is the error: RuntimeError: expected mat1 and mat2 to have the same dtype, but got: c10::Half != float At first I thought this might be due to some difference in architecture (Pascal vs. Ampere) but this is contradicted by the fact that I have a Kaggle Code notebook which can fine-tune Starcoder with two P100 GPUs, which is also Pascal. Is there some other explanation for this? Longer stacktrace attached. [dump.txt](https://github.com/bigcode-project/starcoder/files/14651127/dump.txt)
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
loubnabnl-patch-2
lewtun-patch-1
loubnabnl-patch-1
arjunguha-readme
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
bigcode-project/starcoder#156
No description provided.