Adding columns to understand Model Performance #24
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Can we add some metrics or a relative scoring mechanism or something like that to understand how good the models are?
@Pendrokar is doing that here: https://github.com/Vaibhavs10/open-tts-tracker/pull/19
With "Real-time factor below threshold" being that relative scoring for CPU. For GPU acceleration it would be too hard to tell. Best to just visit the Repo of the specific TTS or test the 🤗 Space and see if they provide the data.
@trivikramak The pull request for adding the capability table has been accepted. Feel free to correct information within it with a PR. I judged the capability from HF Spaces. If a processor column is empty, then that means the RTF was quick with CUDA, but I could not determine if it would have been as fast with just the CPU.
https://github.com/Vaibhavs10/open-tts-tracker/blob/main/README.md#capability-specifics
@fakerybakery I noticed you've added CPU as processor for StyleTTS under the capabilities table, but when I cloned the space of it on HF, I got a RTF of around 7.0. 🤔 Now I did choose "RTF below 2.0" condition arbitrarily as any TTS is capable of processing the audio with CPU. Just the question arises if there is a point to including those whose RTF is 5, 10, 20 with CPU. If so, all cells would have CPU as the processor... @Vaibhavs10 thoughts?
I see an RTF of 2.0 as a good target as a TTS with streaming support would be able to playback the audio once it gets passed half-way of processing the audio.
@Pendrokar I think it also depends on which CPU your using. On a MacBook I get much higher speeds, free HF Spaces have a quite basic CPU