bigcode-project/starcoder
1
0
Fork 0
Code Issues 93 Pull requests 8 Projects Releases Packages Wiki Activity Actions

zero3 DPO starcoder OOM #161

New issue
Open
opened 2024-06-25 23:37:52 -04:00 by oo0-0-0oo · 0 comments
oo0-0-0oo commented 2024-06-25 23:37:52 -04:00 (Migrated from github.com)
Copy link

when I use DPO to train 7B starcoder, OOM happened, i used 16 A100 ,zero3
used TRL and transformers, When the code runs to AutoModelForCausalLM.from_pretrained , OOM happened. but qwencoder don't have this trouble. Are there any special settings in the model's structure that are not suitable for DPO (Direct Policy Optimization)?
code is
`parser = HfArgumentParser(DPOTrainingArguments)
args = parser.parse_args_into_dataclasses()[0]
down_file(args.model_path, args.pretrained_model)

model = AutoModelForCausalLM.from_pretrained(args.model_path)
model_ref = AutoModelForCausalLM.from_pretrained(args.model_path)
tokenizer = AutoTokenizer.from_pretrained(args.model_path)

dpo_trainer = MyDPOTrainer(
model,
model_ref,
args=args,
....
)

dpo_trainer.train()

`

when I use DPO to train 7B starcoder, OOM happened, i used 16 A100 ,zero3 used TRL and transformers, When the code runs to `AutoModelForCausalLM.from_pretrained` , OOM happened. but qwencoder don't have this trouble. Are there any special settings in the model's structure that are not suitable for DPO (Direct Policy Optimization)? code is `parser = HfArgumentParser(DPOTrainingArguments) args = parser.parse_args_into_dataclasses()[0] down_file(args.model_path, args.pretrained_model) model = AutoModelForCausalLM.from_pretrained(args.model_path) model_ref = AutoModelForCausalLM.from_pretrained(args.model_path) tokenizer = AutoTokenizer.from_pretrained(args.model_path) dpo_trainer = MyDPOTrainer( model, model_ref, args=args, .... ) dpo_trainer.train() `
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
loubnabnl-patch-2
lewtun-patch-1
loubnabnl-patch-1
arjunguha-readme
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
bigcode-project/starcoder#161
No description provided.