202 Commits

Author SHA1 Message Date
Erik Thuning
727d6da1af Removed the ability for any handler to use multiple instances by default
Now each handler must implement support for multiple instances in its own
implementation of the instantiate() method
2025-02-06 15:43:19 +01:00
Erik Thuning
ae3ea546a2 Lowered the max number of lines per subtitle segment from 3 to 2 2025-02-06 15:41:49 +01:00
Erik Thuning
365a66bdfa Added a limit to the number of characters per subtitle line 2025-02-06 10:24:48 +01:00
Erik Thuning
f455f95ce0 Stop seeking into the video for poster generation, just use the first frame.
This guarantees that a poster can be generated for any valid stream instead of
breaking on extremely short streams (<1s).
2024-12-06 15:27:34 +01:00
Erik Thuning
cbf29c4962 Implementing a unified job pool for transcodes and subtitles never panned out,
so the code was just adding unnecessary complexity. The pipeline now uses
mp.pool to manage ffmpeg jobs as before.

This reverts commit f91109fb3e4795ce652b966ba0563ba4e100b3d1 and deletes the
WorkThread class and its associated tests.
2024-10-17 11:34:00 +02:00
Erik Thuning
3da736422d Added a debug log line when the model has finished loading.
Also removed some redundant info from the startup debug line.
2024-10-16 14:03:20 +02:00
Erik Thuning
dd0af2221f Removed GPU code from transcode.py, since it never worked properly.
Also cleaned out irrelevant options from config.ini.example.

 - Removed the encoder setting for transcodes i config.ini.example since only
   software is supported now.
 - Since TranscodeWhisperHandler no longer uses the job pool, removed the
   jobsize setting in config.ini.example.
 - Changed default pool and job size to saner values.

The entire WorkThread class should probably be dropped.
2024-10-16 13:56:17 +02:00
Erik Thuning
27598d4401 Promoted subtitles_whisper_hack from hack status.
- Overwrote subtitles_whisper with subtitles_whisper_hack
 - Moved the comment about it being a hack to the more specific hack spot
 - Updated __init__.py to import from subtitles_whisper

The work invested in _hack is now significant enough that if automatic gpu
detection becomes viable again, the only meaningful starting point is
the _hack implementation. It's probably a good idea to remove _serial as well,
but leaving it around for now.
2024-10-14 16:34:46 +02:00
Erik Thuning
d6206f1e6b Changed instantiation logic for whisper handler:
- Separated GPU and CPU config keywords
 - GPUs are now chosen based on GPU index
 - Configurable number of threads per GPU
2024-10-14 16:26:31 +02:00
Erik Thuning
f1de6a19c4 Whitespace 2024-10-10 12:59:05 +02:00
Erik Thuning
f8ab46bac8 Added support for uploads with name prefix 2024-09-06 11:40:59 +02:00
Erik Thuning
11917608bb Made raise_for_structure provide better feedback on errors
The function now takes an argument corresponding to the key that is
being worked on. Also made the handling of dict validation more explicit
about keys and the reasoning around why it does what it does.
2024-06-18 11:26:20 +02:00
Erik Thuning
8de039a3f3 Fixed some bugs so the preprocessor works:
- start time is now recorded as a timestamp
 - typo fix: match.groups() -> match.group()
 - Subtitle generation is only requested if the presentation has video on channel 1
2024-06-18 11:23:43 +02:00
Erik Thuning
4842ed56f1 Fixed several errors in the arec preprocessor
* renamed the 'data' dict to 'arec_data' and made sure all references are consistent
* switched the recorder identifier from hostname to description, because there was a wild \n in the hostname for unknown reasons
* removed timezone awareness from starttime and endtime to make them compatible with times loaded from daisy
2024-06-13 15:37:21 +02:00
Erik Thuning
b7a3d7dec2 Activated the arec preprocessor 2024-06-13 15:36:25 +02:00
Erik Thuning
dec2da4310 Added a preprocessor for arec recorders.
Also moved the code for creating a basic jobspec and pulling information from
the relevant daisy booking into the preprocessor superclass so it can be
called by both the cattura and arec preprocessors.
2024-06-04 15:53:46 +02:00
Erik Thuning
49d1da804b Started seting the running flag immediately on the call to start()
instead of after all the startup tsks are done. This alleviates a potential
problem where start() could be called more than once in quick succession
with unpredictable results.
2024-06-04 15:48:17 +02:00
30361abdd9 VisibilityHandler (#3)
Create a VisibilityHandler

Co-authored-by: nenzen <stefan@nenzen.com>
Reviewed-on: #3
Reviewed-by: erth9960 <thuning@dsv.su.se>
Co-authored-by: Stefan Nenzén <nenzen@dsv.su.se>
Co-committed-by: Stefan Nenzén <nenzen@dsv.su.se>
2024-03-27 14:54:33 +01:00
Erik Thuning
37c2dbaec6 Merge branch 'master' of gitea.dsv.su.se:DMC/play-daemon-threaded 2024-03-27 10:50:33 +01:00
Erik Thuning
3d9075a5e0 Starting on a visibility handler 2024-03-27 10:50:09 +01:00
Erik Thuning
bfe68b9250 Fixed a typo 2024-03-27 10:40:31 +01:00
Erik Thuning
6c11d5578f Added optional support for user-specified notification endpoints in packages 2024-03-26 15:47:39 +01:00
Erik Thuning
19498e64f9 Added logging of contents of queue files from disk and preprocessed jobs 2024-02-16 11:58:05 +01:00
Erik Thuning
02341f0cd8 The original check was a bit hasty, this one should actually work properly. 2024-02-06 14:17:03 +01:00
Erik Thuning
8a0a9fbc33 Set the cattura preprocessor to automatically request subtitling
based on the camera source.
2024-02-06 13:31:11 +01:00
Erik Thuning
9874390a78 Python complains if a dictionary is modified while using an iterator on it, so changing iterator to a list 2024-01-18 10:44:15 +01:00
Erik Thuning
35d7a057e3 Missed another import 2024-01-18 10:37:53 +01:00
Erik Thuning
2cbb501361 Forgot an import 2024-01-18 10:34:18 +01:00
Erik Thuning
bb76807921 Added logging when deleting job files 2024-01-18 10:32:13 +01:00
Erik Thuning
d28be73e2c No longer setting explicit width, so that ffmpeg can take care of odd number oddness 2024-01-15 20:25:51 +01:00
Erik Thuning
7a2736300b Made logging statements use consistent string formats and some minor related fixes 2023-12-21 14:41:11 +01:00
Erik Thuning
18841b44d7 Switched to the hack version of SubtitlesWhisperHandler 2023-12-20 16:19:07 +01:00
Erik Thuning
b4f6ff8bde Formatting 2023-12-20 16:18:43 +01:00
Erik Thuning
e19fb3c9a4 Added a hack version of parallel transcription until triton is fixed 2023-12-20 16:18:03 +01:00
Erik Thuning
a83efc62a5 Logging process information if log level is DEBUG or higher 2023-12-14 16:40:24 +01:00
Erik Thuning
4283badaef Saving intermediate subtitle files under the track name instead of the source name 2023-12-14 16:29:30 +01:00
Erik Thuning
784b365f1e Switching to the serial whisper handler 2023-12-14 14:59:48 +01:00
Erik Thuning
51991d327b Testing out a new approach to dispatching subtitle generation jobs.
Hope to solve gpu assignment issues
2023-12-14 14:58:35 +01:00
Erik Thuning
a5d9eef356 Switched the directory observer in QueueReader from inotify to polling,
in order to properly handle NFS-mounted queue directories.

As a consequence, the default timeout for awaiting job results in the tests
was upped to 10 seconds due to the slightly slower queue pickup.
2023-12-08 11:32:16 +01:00
Erik Thuning
10c126a37c Split subtitles handling into two handlers.
One for generation and one for import.
2023-11-30 16:37:13 +01:00
Erik Thuning
2da06f84a7 Attempting to make sure all log lines get flushed, and fixed a variable name 2023-11-30 16:08:07 +01:00
Erik Thuning
f634fd4b1e Documented the ability to force the language when generating subtitles 2023-11-30 15:51:14 +01:00
Erik Thuning
881de4f699 Merged two log lines for better emails 2023-11-30 12:35:11 +01:00
Erik Thuning
a6ccb053d8 Removed an unnecessary try/except.
Handler._process() already catches the exception in a sane way and hence no
such handling is needed in SubtitlesHandler._handle()
2023-11-30 11:53:14 +01:00
Erik Thuning
b5eda4695f Updated logging calls 2023-11-30 09:53:09 +01:00
Erik Thuning
a0bba02840 Major updates to the subtitles handler with some knock-on effects
- More than one subtitles handler can be run in parallel
 - Language can be specified in generation tasks
 - Switched to pathlib for path handling in subtitles handler (simpler)
 - Handlers are now instantiated via factory function
2023-11-24 16:17:51 +01:00
Erik Thuning
2ce11af099 Formatting 2023-11-23 15:43:38 +01:00
Erik Thuning
b81bd64ffd Made it possible to have more than one handler of the same type
by making the handler's queue an init parameter.
2023-11-23 15:41:42 +01:00
Erik Thuning
c5bfc9f8d5 Made queue items be processed chronologically on start 2023-11-22 10:03:54 +01:00
Erik Thuning
5641aef235 Fixed a crash caused by empty sequence 2023-11-22 09:44:00 +01:00