Autosub support (automatically generate subtitles by analyzing audio file)

221 votes

Would it be possible to add an option to 'analyse' the audio using a tool such as 'autosub' (see here, and here) if there's no results from any of the indexes...

I have several locally produced series that do not have subtitles available in the usual places around the web, and figure that something like this might be a good 'last resort' option to fallback to if there's no subs available.

Above docker link is a quick and easy way to test it locally if you don't want to mess around with python libs etc...

Cheers,

https://github.com/morpheus65535/bazarr/issues/507

Under consideration Suggested by: morpheus65535 (27 Aug, '19) • Upvoted: 10 Apr • Comments: 12

Comments: 12

03 Nov, '19
a-nunes
This is an more updated version:

https://github.com/BingLingGroup/autosub
29 Nov, '21
whydie
This seems amazingly useful
17 Feb, '22
Jonas Friedmann
There's also https://github.com/sc0ty/subsync (used by online service https://subsync.online/)
1
17 Feb, '22
Topper Admin
@Jonas Firedmann this is unrelated to this FR. The OP was requesting a way to generate subtitles, not sync them.
23 Sep, '22
Alex Yancey
Hello,

With the release of openai's Whisper project, I believe this is absolutely possible now. https://github.com/openai/whisper

I would like to play around with whisper as a fall-back subtitle provider in Bazarr. It helps a lot to have a powerful GPU in order to accelerate this process. It may be unbearably slow without gpu acceleration.
01 Nov, '22
Jake
Looks pretty helpful. Already has subtitle endpoints implemented: https://github.com/ahmetoner/whisper-asr-webservice
25 Dec, '22
Roberto Michelena
+1 for using Whisper, it's performance is superb in multiple languages and even has support for translation
it's open source, python-based... it's being used already in video workflows such as here:
https://avflow.io/flows/use-openais-whisper-to-generate-srt-sidecar-files-in-your-flow-for-mux
29 Dec, '22
Roberto Michelena
here there's a version with improved timestamps alignment:
https://github.com/m-bain/whisperX
26 Jan, '23
Maxim
just use live captures form google chrome for now =)
https://support.google.com/chrome/answer/10538231?hl=en
07 Feb, '23
Garret
I would also now go with the 'whisper.cpp' route. It is astonishgly powerful and it can even run on a raspberry pi 4 (although I would not go over 'base' library).
There is already a wrapper for Tautulli/Plex being developed. It has been suggested to the developer to make it as a provider for bazarr but developer is not interested and just shared its code so that someone else could fork it from his idea.

I leave some links for more info:
- https://github.com/McCloudS/subgen
- https://www.reddit.com/r/selfhosted/comments/10nlc4h/subgen_autogen_plex_subtitles_using_tautulli_and/
27 Feb, '23
Alex Yancey
I'm moving forward with an approach to add this feature with suggestions from this discussion. https://github.com/morpheus65535/bazarr/pull/2073

Please contact me on Discord if you would like to contribute or pitch ideas as I'm not an expert developer by any means.
24 Mar, '23
Amichai Hadad
Is there any reason why we can't have like a premade API connector to the public whisper API?
or to some cloud implementation of it?

I feel like it would be more convenient for the end-user to opt to use STT to generate subtitles via Bazarr.

Allow me as a user to fill in my own API credentials with OpenAI billing.
Send the audio track to the OpenAI endpoint, and, handle the result srt/vtt file by Bazarr / etc.

i don't know if they moderate that stuff or not and if they will block my account if I try to transcribe certain content via whispers public infrastructure but well worth a try.
the difference in performance and user experience will be unparalleled compared to running whisper locally (even on GPU) on the end-user computer.