WIP: Submit transcoding jobs via a HTTP API #6

Draft
ansv7779 wants to merge 22 commits from api-submission into master
Owner

Create a HTTP API for submitting transcoding jobs. Implemented as a three-step process with step one being creating and configuring the transcription job, step two attaching the files to be transcribed, and the final step three submitting the job for processing.

Each job can have multiple files attached to it but they are submitted as individual jobs to the Whisper engine and the system will deal with grouping them together. The user of the API will only receive one final callback once all the individual jobs have been completed, successful or not. This means the user has to deal with potentially partially successful jobs, some files may have been transcoded correctly while others not.

Fixes #2, #3, #4

Create a HTTP API for submitting transcoding jobs. Implemented as a three-step process with step one being creating and configuring the transcription job, step two attaching the files to be transcribed, and the final step three submitting the job for processing. Each job can have multiple files attached to it but they are submitted as individual jobs to the Whisper engine and the system will deal with grouping them together. The user of the API will only receive one final callback once all the individual jobs have been completed, successful or not. This means the user has to deal with potentially partially successful jobs, some files may have been transcoded correctly while others not. Fixes #2, #3, #4
ansv7779 added 3 commits 2024-06-17 15:59:16 +02:00
To get something transcribed is a three-step process;
1. Create and configure the transcription job
2. Attach files to the job to be transcribed
3. Submit the job for processing

This is step one of the process.
To get something transcribed is a three-step process;
1. Create and configure the transcription job
2. Attach files to the job to be transcribed
3. Submit the job for processing

This is step two of the process.
To get something transcribed is a three-step process;
1. Create and configure the transcription job
2. Attach files to the job to be transcribed
3. Submit the job for processing

This is step three of the process.
erth9960 reviewed 2024-06-19 13:19:39 +02:00
@ -0,0 +48,4 @@
OutputFormat outputFormat = parseOutputFormat(createTranscriptionRequest.outputFormat());
CreateTranscription createTranscription = new CreateTranscription(owner, callbackUri, outputFormat);
Transcription transcription = transcriptionService.createTranscription(createTranscription);
return new TranscriptionCreatedResponse(transcription.id());
Owner

This response should contain a reference to the upload URL for this job.

This response should contain a reference to the upload URL for this job.
Author
Owner

Fixed in aa49f418d2

Fixed in https://gitea.dsv.su.se/DMC/whisper-frontend/commit/aa49f418d29b550f9340cb953fb4648c971f7eb7
erth9960 reviewed 2024-06-19 13:20:24 +02:00
@ -0,0 +84,4 @@
Transcription transcription = transcriptionService.getTranscription(owner, uuid)
.orElseThrow(() -> new TranscriptionNotFound(id));
transcriptionService.addFileToBeTranscribed(transcription, new TranscriptionFile(filename, fileData));
return ResponseEntity.accepted().build();
Owner

This response should contain two references to URLs:

  • submit another file
  • start job
This response should contain two references to URLs: - submit another file - start job
Author
Owner

Fixed in 8d13dc31c6

Fixed in https://gitea.dsv.su.se/DMC/whisper-frontend/commit/8d13dc31c6666ab0a2a73a88974deec3a7ad6f0e
erth9960 reviewed 2024-06-19 13:26:15 +02:00
@ -0,0 +45,4 @@
public void addFileToBeTranscribed(Transcription transcription, TranscriptionFile file)
throws IOException
{
Path fileToBeTranscribed = fileDirectory.resolve(transcription.id().toString()).resolve(file.filename());
Owner

Avoid using the filename provided by the user as a security measure.

Avoid using the filename provided by the user as a security measure.
Author
Owner

Fixed in 515b2aa642

Fixed in https://gitea.dsv.su.se/DMC/whisper-frontend/commit/515b2aa642d3bdce23a27f96ae532f2a1ea29543
ansv7779 marked this conversation as resolved
ansv7779 added 4 commits 2024-06-19 14:34:15 +02:00
ansv7779 added 2 commits 2024-07-08 16:10:37 +02:00
Callbacks are sent out from an asynchronous job with no web context. Therefore, it is impossible, at that time, to generate absolute URI:s. Pre-generate them at upload time to maximize the chance that the correct web context is used to generate the URI:s.
ansv7779 added 2 commits 2024-07-10 11:56:57 +02:00
ansv7779 added 6 commits 2024-07-11 14:19:00 +02:00
ansv7779 added 3 commits 2024-07-11 15:01:39 +02:00
Author
Owner

0064149d79 also fixes #7

https://gitea.dsv.su.se/DMC/whisper-frontend/commit/0064149d799c3cde935fbe9c33f1ecab778f504d also fixes #7
Author
Owner

Closing and re-opening to deploy to test server.

Closing and re-opening to deploy to test server.
ansv7779 closed this pull request 2024-07-11 15:05:33 +02:00
ansv7779 reopened this pull request 2024-07-11 15:06:01 +02:00
ansv7779 added 1 commit 2024-07-11 15:08:10 +02:00
Merge branch 'master' into api-submission
Some checks failed
Clean up branch.dsv.su.se / Cleanup-branch (pull_request) Failing after 3s
Deploy to branch.dsv.su.se / Deploy-branch (pull_request) Successful in 1m13s
f2eaed3d00
# Conflicts:
#	src/main/java/se/su/dsv/whisperapi/WhisperApiApplication.java
Author
Owner

Have to merge the workflows first

Have to merge the workflows first
ansv7779 closed this pull request 2024-07-11 15:08:45 +02:00
ansv7779 reopened this pull request 2024-07-11 15:09:18 +02:00
ansv7779 added 1 commit 2024-07-11 16:39:28 +02:00
First-time contributor
[https://whisper-frontend-api-submission.branch.dsv.su.se](https://whisper-frontend-api-submission.branch.dsv.su.se/swagger-ui/index.html)
This pull request is marked as a work in progress.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin api-submission:api-submission
git checkout api-submission
Sign in to join this conversation.
No Reviewers
No Label
No Milestone
No project
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DMC/whisper-frontend#6
No description provided.