Add example to fine-tune StarCoder for chat-based applications #17
No reviewers
Labels
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
bigcode-project/starcoder!17
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "starchat"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
TODO
Looks good overall. Left only a few minor comments.
I am mostly wondering if we could have used the
SFTTrainerto reduce a bit the amount of code needed fortrain.pyor run a simple preprocessing script that pushes the formated dataset to the hub for the script to reduce a bit the amount of code. Don't think we need to change anything now but I think we should provide a bit more utility to be able to make such examples more concise.@ -0,0 +1,111 @@# Fine-tuning StarCoder for chat-based applicationsThis is a fully-working example to fine-tune `StarCoder` on a corpus of multi-turn dialogues and thus create a coding assistant that is chatty and helpful. The resulting model is quite good at generating code for plots and other programming tasks. For example, given the prompt:maybe some examples of what the model can do after tuning would be cool to show here.
@ -0,0 +108,4 @@```shellpython generate.py --model_id path/to/your/model```I guess something is missing here :)
Yes I agree this would be nice to do & I'll fix this in a follow-up PR (I ran out of time to test everything with the new API)
@ -0,0 +108,4 @@```shellpython generate.py --model_id path/to/your/model```Good catch!