From d039f0e3b7c071553e71486a29b92ccb5c05794c Mon Sep 17 00:00:00 2001 From: Stephen G Pope Date: Mon, 16 Dec 2024 00:20:36 -0800 Subject: [PATCH] bunch of minor fixes --- docs/code/execute/execute_python.md | 124 +++++++++-------- docs/ffmpeg/compose.md | 144 +++++++++----------- docs/image/transform/image_to_video.md | 143 ++++++++++++++------ docs/media/transcribe.md | 131 +----------------- docs/media/transform/media_to_mp3.md | 142 +++++++++----------- docs/toolkit/authenticate.md | 61 +++++---- docs/toolkit/test.md | 57 ++++---- docs/video/caption_video.md | 172 +++++++++++++----------- docs/video/concatenate.md | 178 +++++++++++++------------ generate_docs.py | 15 ++- 10 files changed, 574 insertions(+), 593 deletions(-) mode change 100644 => 100755 generate_docs.py diff --git a/docs/code/execute/execute_python.md b/docs/code/execute/execute_python.md index 2f577f5..280695b 100644 --- a/docs/code/execute/execute_python.md +++ b/docs/code/execute/execute_python.md @@ -2,7 +2,7 @@ ## 1. Overview -The `/v1/code/execute/python` endpoint allows users to execute Python code on the server. This endpoint is part of the version 1 API and is registered in the main `app.py` file. It is designed to provide a secure and controlled environment for executing Python code, with features like input validation, timeout handling, and output capturing. +The `/v1/code/execute/python` endpoint allows users to execute Python code on the server. This endpoint is part of the `v1` API and is registered in the `app.py` file. It is designed to provide a secure and controlled environment for executing Python code, with features like input validation, timeout handling, and output capturing. ## 2. Endpoint @@ -17,19 +17,37 @@ The `/v1/code/execute/python` endpoint allows users to execute Python code on th ### Body Parameters -- `code` (required, string): The Python code to be executed. -- `timeout` (optional, integer): The maximum execution time in seconds, between 1 and 300. Default is 30 seconds. -- `webhook_url` (optional, string): The URL to send the execution results to as a webhook. -- `id` (optional, string): An identifier for the request. +The request body must be a JSON object with the following properties: -#### Example Request +- `code` (string, required): The Python code to be executed. +- `timeout` (integer, optional): The maximum execution time in seconds, between 1 and 300. Default is 30 seconds. +- `webhook_url` (string, optional): The URL to receive the execution result via a webhook. +- `id` (string, optional): A unique identifier for the request. + +The `validate_payload` decorator in the routes file enforces the following JSON schema for the request body: + +```json +{ + "type": "object", + "properties": { + "code": {"type": "string"}, + "timeout": {"type": "integer", "minimum": 1, "maximum": 300}, + "webhook_url": {"type": "string", "format": "uri"}, + "id": {"type": "string"} + }, + "required": ["code"], + "additionalProperties": False +} +``` + +### Example Request ```json { "code": "print('Hello, World!')", "timeout": 10, "webhook_url": "https://example.com/webhook", - "id": "request-123" + "id": "unique-request-id" } ``` @@ -37,77 +55,79 @@ The `/v1/code/execute/python` endpoint allows users to execute Python code on th curl -X POST \ -H "x-api-key: YOUR_API_KEY" \ -H "Content-Type: application/json" \ - -d '{"code": "print('Hello, World!')", "timeout": 10, "webhook_url": "https://example.com/webhook", "id": "request-123"}' \ - http://localhost:8080/v1/code/execute/python + -d '{"code": "print('Hello, World!')", "timeout": 10, "webhook_url": "https://example.com/webhook", "id": "unique-request-id"}' \ + http://your-api-endpoint/v1/code/execute/python ``` ## 4. Response ### Success Response -**Status Code:** `200 OK` +The success response follows the general response format defined in `app.py`. Here's an example: ```json { - "result": null, - "stdout": "Hello, World!\n", - "stderr": "", - "exit_code": 0 + "code": 200, + "id": "unique-request-id", + "job_id": "generated-job-id", + "response": { + "result": null, + "stdout": "Hello, World!\n", + "stderr": "", + "exit_code": 0 + }, + "message": "success", + "pid": 1234, + "queue_id": 12345678, + "run_time": 0.123, + "queue_time": 0.0, + "total_time": 0.123, + "queue_length": 0, + "build_number": "1.0.0" } ``` ### Error Responses -**Status Code:** `400 Bad Request` - -```json -{ - "error": "Error message", - "stdout": "Output from the executed code", - "exit_code": 1 -} -``` +- **400 Bad Request**: + - Example: `{"error": "Execution failed", "stdout": "", "exit_code": 1}` + - Reason: The Python code execution failed, or there was an error in the user's code. -**Status Code:** `408 Request Timeout` +- **408 Request Timeout**: + - Example: `{"error": "Execution timed out after 10 seconds"}` + - Reason: The Python code execution exceeded the specified timeout. -```json -{ - "error": "Execution timed out after 10 seconds" -} -``` +- **500 Internal Server Error**: + - Example: `{"error": "Failed to parse execution result"}` + - Reason: An internal error occurred while parsing the execution result. -**Status Code:** `500 Internal Server Error` +## 5. Error Handling -```json -{ - "error": "Failed to parse execution result", - "stdout": "Output from the executed code", - "stderr": "Error output from the executed code", - "exit_code": 1 -} -``` +The endpoint handles the following common errors: -## 5. Error Handling +- **Missing or invalid parameters**: If the request body is missing or contains invalid parameters, a `400 Bad Request` error is returned. +- **Execution timeout**: If the Python code execution exceeds the specified timeout, a `408 Request Timeout` error is returned. +- **Internal errors**: If an internal error occurs during code execution or result parsing, a `500 Internal Server Error` is returned. -- Missing or invalid parameters will result in a `400 Bad Request` error. -- If the execution times out, a `408 Request Timeout` error is returned. -- Any other execution errors or server-side errors will result in a `500 Internal Server Error`. -- The main application context handles errors related to the task queue, such as reaching the maximum queue length (`429 Too Many Requests`). +Additionally, the main application context (`app.py`) includes error handling for queue overload. If the maximum queue length is reached, a `429 Too Many Requests` error is returned. ## 6. Usage Notes -- The executed code runs in a sandboxed environment, with limited access to system resources. -- The code's output (both stdout and stderr) is captured and returned in the response. -- If a `webhook_url` is provided, the execution results will be sent to that URL as a webhook. +- The Python code is executed in a sandboxed environment, with limited access to system resources and libraries. +- The execution output (stdout and stderr) is captured and included in the response. +- The return value of the executed code is included in the response if the execution is successful. +- The `webhook_url` parameter is optional and can be used to receive the execution result via a webhook. ## 7. Common Issues -- Executing code that takes too long or consumes too many resources may result in a timeout or server error. -- Code that attempts to access restricted system resources or perform unauthorized operations may fail or produce unexpected results. +- **Syntax errors or exceptions in the user's code**: These will result in a `400 Bad Request` error, with the error message and traceback included in the response. +- **Infinite loops or long-running code**: The execution will be terminated after the specified timeout, resulting in a `408 Request Timeout` error. +- **Excessive resource usage**: The sandboxed environment may impose limits on memory, CPU, or other resources, which could cause the execution to fail. ## 8. Best Practices -- Always validate and sanitize user input before executing code. -- Set an appropriate timeout value based on the expected execution time of the code. -- Monitor the server's resource usage and adjust the maximum queue length or other limits as needed. -- Implement additional security measures, such as code sandboxing or whitelisting, to prevent malicious code execution. \ No newline at end of file +- **Validate user input**: Always validate and sanitize user input to prevent code injection or other security vulnerabilities. +- **Set appropriate timeouts**: Choose a reasonable timeout value based on the expected complexity of the code to be executed. +- **Monitor resource usage**: Keep an eye on the resource usage of the execution environment to prevent excessive consumption or denial of service attacks. +- **Implement rate limiting**: Consider implementing rate limiting to prevent abuse or overload of the endpoint. +- **Log and monitor errors**: Ensure that errors are properly logged and monitored for debugging and security purposes. \ No newline at end of file diff --git a/docs/ffmpeg/compose.md b/docs/ffmpeg/compose.md index c5ed3bd..e82e90e 100644 --- a/docs/ffmpeg/compose.md +++ b/docs/ffmpeg/compose.md @@ -1,13 +1,13 @@ -# FFmpeg Compose API +# FFmpeg Compose API Endpoint ## 1. Overview -The FFmpeg Compose API endpoint allows users to perform complex video and audio processing tasks using the powerful FFmpeg library. It provides a flexible way to combine multiple input files, apply various filters and options, and generate one or more output files. This endpoint fits into the overall API structure as part of the version 1 (`v1`) routes, specifically under the `/v1/ffmpeg/compose` path. +The `/v1/ffmpeg/compose` endpoint is a part of the Flask API application and is designed to provide a flexible and powerful way to compose and manipulate media files using FFmpeg. This endpoint allows users to specify input files, filters, and output options, enabling a wide range of media processing tasks such as transcoding, concatenation, and video editing. ## 2. Endpoint -**URL Path:** `/v1/ffmpeg/compose` -**HTTP Method:** `POST` +- **URL Path**: `/v1/ffmpeg/compose` +- **HTTP Method**: `POST` ## 3. Request @@ -21,20 +21,25 @@ The request body should be a JSON object with the following properties: - `inputs` (required, array): An array of input file objects, each containing: - `file_url` (required, string): The URL of the input file. - - `options` (optional, array): An array of option objects, each containing: + - `options` (optional, array): An array of input file options, each containing: - `option` (required, string): The FFmpeg option to apply. - `argument` (optional, string, number, or null): The argument for the option. - `filters` (optional, array): An array of filter objects, each containing: - `filter` (required, string): The FFmpeg filter to apply. - `outputs` (required, array): An array of output file objects, each containing: - - `options` (required, array): An array of option objects, each containing: + - `options` (required, array): An array of output file options, each containing: - `option` (required, string): The FFmpeg option to apply. - `argument` (optional, string, number, or null): The argument for the option. - `global_options` (optional, array): An array of global option objects, each containing: - `option` (required, string): The FFmpeg global option to apply. - `argument` (optional, string, number, or null): The argument for the option. -- `metadata` (optional, object): An object specifying which metadata to include in the response, with boolean properties for `thumbnail`, `filesize`, `duration`, `bitrate`, and `encoder`. -- `webhook_url` (required, string): The URL to send the response webhook. +- `metadata` (optional, object): An object specifying which metadata to include in the response, with the following properties: + - `thumbnail` (optional, boolean): Whether to include a thumbnail for the output file. + - `filesize` (optional, boolean): Whether to include the file size of the output file. + - `duration` (optional, boolean): Whether to include the duration of the output file. + - `bitrate` (optional, boolean): Whether to include the bitrate of the output file. + - `encoder` (optional, boolean): Whether to include the encoder used for the output file. +- `webhook_url` (required, string): The URL to send the response to. - `id` (required, string): A unique identifier for the request. ### Example Request @@ -61,7 +66,7 @@ The request body should be a JSON object with the following properties: ], "filters": [ { - "filter": "hflip" + "filter": "scale=640:480" } ], "outputs": [ @@ -121,7 +126,7 @@ curl -X POST \ ], "filters": [ { - "filter": "hflip" + "filter": "scale=640:480" } ], "outputs": [ @@ -159,94 +164,77 @@ curl -X POST \ ### Success Response -**Status Code:** `200 OK` - -The response body will be a JSON array containing one or more objects, each representing an output file. Each object will have the following properties: - -- `file_url` (string): The URL of the uploaded output file. -- `thumbnail_url` (string, optional): The URL of the uploaded thumbnail image, if requested. -- `filesize` (number, optional): The size of the output file in bytes, if requested. -- `duration` (number, optional): The duration of the output file in seconds, if requested. -- `bitrate` (number, optional): The bitrate of the output file in bits per second, if requested. -- `encoder` (string, optional): The encoder used for the output file, if requested. - -```json -[ - { - "file_url": "https://storage.googleapis.com/bucket/output1.mp4", - "thumbnail_url": "https://storage.googleapis.com/bucket/output1_thumbnail.jpg", - "filesize": 12345678, - "duration": 42.5, - "bitrate": 1234567, - "encoder": "libx264" - }, - { - "file_url": "https://storage.googleapis.com/bucket/output2.mp4", - "filesize": 87654321, - "duration": 120.0, - "bitrate": 7654321, - "encoder": "libx264" - } -] -``` +The response will be sent to the specified `webhook_url` as a JSON object with the following properties: + +- `endpoint` (string): The endpoint that processed the request (`/v1/ffmpeg/compose`). +- `code` (number): The HTTP status code (200 for success). +- `id` (string): The unique identifier for the request. +- `job_id` (string): The unique job ID assigned to the request. +- `response` (array): An array of output file objects, each containing: + - `file_url` (string): The URL of the uploaded output file. + - `thumbnail_url` (string, optional): The URL of the uploaded thumbnail, if requested. + - `filesize` (number, optional): The size of the output file in bytes, if requested. + - `duration` (number, optional): The duration of the output file in seconds, if requested. + - `bitrate` (number, optional): The bitrate of the output file in bits per second, if requested. + - `encoder` (string, optional): The encoder used for the output file, if requested. +- `message` (string): A success message (`"success"`). +- `pid` (number): The process ID of the worker that processed the request. +- `queue_id` (number): The ID of the queue used for processing the request. +- `run_time` (number): The time taken to process the request, in seconds. +- `queue_time` (number): The time the request spent in the queue, in seconds. +- `total_time` (number): The total time taken to process the request, including queue time, in seconds. +- `queue_length` (number): The current length of the processing queue. +- `build_number` (string): The build number of the application. ### Error Responses -**Status Code:** `400 Bad Request` - -```json -{ - "message": "Invalid request payload" -} -``` - -**Status Code:** `401 Unauthorized` +- **400 Bad Request**: The request payload is invalid or missing required parameters. +- **401 Unauthorized**: The provided API key is invalid or missing. +- **429 Too Many Requests**: The maximum queue length has been reached. +- **500 Internal Server Error**: An unexpected error occurred while processing the request. -```json -{ - "message": "Missing or invalid API key" -} -``` - -**Status Code:** `500 Internal Server Error` +Example error response (400 Bad Request): ```json { - "message": "An error occurred while processing the request" + "code": 400, + "message": "Invalid request payload: 'inputs' is a required property" } ``` ## 5. Error Handling -The endpoint performs the following error handling: - -- **Missing or invalid API key**: Returns a `401 Unauthorized` status code with an error message. -- **Invalid request payload**: Returns a `400 Bad Request` status code with an error message. -- **FFmpeg processing error**: Returns a `500 Internal Server Error` status code with the error message from FFmpeg. -- **Output file not found**: Returns a `500 Internal Server Error` status code with an error message indicating that the expected output file was not found. +The API handles various types of errors, including: -Additionally, the main application context includes error handling for the following cases: +- **Missing or invalid parameters**: If the request payload is missing required parameters or contains invalid data types, a 400 Bad Request error will be returned. +- **Authentication failure**: If the provided API key is invalid or missing, a 401 Unauthorized error will be returned. +- **Queue limit reached**: If the maximum queue length is reached, a 429 Too Many Requests error will be returned. +- **Unexpected errors**: If an unexpected error occurs during request processing, a 500 Internal Server Error will be returned, along with an error message. -- **Queue length exceeded**: If the maximum queue length is set and the queue size reaches that limit, the endpoint returns a `429 Too Many Requests` status code with an error message. -- **Missing webhook URL**: If the request does not include a `webhook_url`, the task is processed immediately without being queued, and the response is returned directly. +The main application context (`app.py`) includes error handling for the processing queue. If the maximum queue length is set and the queue size reaches that limit, new requests will be rejected with a 429 Too Many Requests error. ## 6. Usage Notes -- The FFmpeg Compose API provides a flexible way to perform complex video and audio processing tasks by combining multiple input files, applying filters and options, and generating one or more output files. -- The request payload allows for a wide range of customization, including specifying input file options, filters, output file options, and global options. -- The `metadata` object in the request payload allows users to request specific metadata to be included in the response, such as thumbnail images, file size, duration, bitrate, and encoder information. -- The response includes the URLs of the uploaded output files, as well as any requested metadata. +- The `inputs` array can contain multiple input files, allowing for operations like concatenation or merging. +- The `filters` array allows applying FFmpeg filters to the input files. +- The `outputs` array specifies the output file options, such as codec, bitrate, and resolution. +- The `global_options` array allows setting global FFmpeg options that apply to the entire operation. +- The `metadata` object allows requesting specific metadata to be included in the response, such as thumbnail, file size, duration, bitrate, and encoder information. +- The `webhook_url` parameter is required, and the response will be sent to this URL as a POST request. +- The `id` parameter is a unique identifier for the request, which can be used for tracking and logging purposes. ## 7. Common Issues -- **Invalid input file URLs**: Ensure that the provided input file URLs are valid and accessible. -- **Unsupported FFmpeg options or filters**: Double-check that the specified FFmpeg options and filters are valid and supported by the version of FFmpeg used by the API. -- **Large file sizes**: Processing large video or audio files may take longer and consume more resources. Consider breaking down large files into smaller chunks or adjusting the processing options to reduce the output file size. +- Providing invalid or malformed input file URLs. +- Specifying invalid or unsupported FFmpeg options or filters. +- Attempting to process large or high-resolution files, which may result in long processing times or queue delays. +- Reaching the maximum queue length, which will cause new requests to be rejected. ## 8. Best Practices -- **Validate input file URLs**: Before submitting a request, ensure that the input file URLs are valid and accessible to avoid errors during processing. -- **Test with sample files**: Before processing large or important files, test the desired FFmpeg options and filters with sample files to ensure the expected output. -- **Monitor response webhooks**: Set up a webhook URL to receive notifications and monitor the progress and results of your FFmpeg processing tasks. -- **Optimize processing options**: Experiment with different FFmpeg options and filters to find the optimal balance between output quality and file size for your use case. -- **Handle errors gracefully**: Implement error handling in your application to gracefully handle any errors returned by the API and provide appropriate feedback to users. \ No newline at end of file +- Validate input file URLs and ensure they are accessible before submitting the request. +- Test FFmpeg options and filters with sample files before using them in production. +- Consider transcoding or downscaling large or high-resolution files to reduce processing time and queue delays. +- Monitor the queue length and adjust the maximum queue length as needed to prevent excessive queuing or request rejections. +- Implement retry mechanisms for failed requests or requests that encounter queue limits. +- Use the `id` parameter to track and correlate requests with responses for better logging and debugging. \ No newline at end of file diff --git a/docs/image/transform/image_to_video.md b/docs/image/transform/image_to_video.md index decad71..45b1a16 100644 --- a/docs/image/transform/image_to_video.md +++ b/docs/image/transform/image_to_video.md @@ -1,8 +1,8 @@ -# Image to Video Endpoint +# Image to Video Endpoint Documentation ## 1. Overview -The `/v1/image/transform/video` endpoint is a part of the Flask API application and is responsible for converting an image into a video file. This endpoint is registered in the `app.py` file under the `v1_image_transform_video_bp` blueprint. +The `/v1/image/transform/video` endpoint is part of the Flask API application and is responsible for converting an image into a video file. This endpoint is registered in the `app.py` file under the `v1_image_transform_video_bp` blueprint, which is imported from the `routes.v1.image.transform.image_to_video` module. ## 2. Endpoint @@ -19,12 +19,32 @@ The `/v1/image/transform/video` endpoint is a part of the Flask API application The request body must be a JSON object with the following properties: -- `image_url` (required, string): The URL of the image to be converted into a video. -- `length` (optional, number): The desired length of the video in seconds. Default is 5 seconds. Minimum value is 1, and maximum value is 60. -- `frame_rate` (optional, integer): The frame rate of the output video. Default is 30 frames per second. Minimum value is 15, and maximum value is 60. -- `zoom_speed` (optional, number): The speed at which the image should zoom in or out during the video. Default is 3%. Minimum value is 0, and maximum value is 100. -- `webhook_url` (optional, string): The URL to which a webhook notification should be sent upon completion of the video conversion. -- `id` (optional, string): A unique identifier for the request. +| Parameter | Type | Required | Description | +|-------------|--------|----------|--------------------------------------------------------------| +| `image_url` | string | Yes | The URL of the image to be converted into a video. | +| `length` | number | No | The desired length of the video in seconds (default: 5). | +| `frame_rate`| integer| No | The frame rate of the output video (default: 30). | +| `zoom_speed`| number | No | The speed of the zoom effect (0-100, default: 3). | +| `webhook_url`| string| No | The URL to receive a webhook notification upon completion. | +| `id` | string | No | An optional ID to associate with the request. | + +The `validate_payload` decorator in the `routes` file enforces the following JSON schema for the request body: + +```json +{ + "type": "object", + "properties": { + "image_url": {"type": "string", "format": "uri"}, + "length": {"type": "number", "minimum": 1, "maximum": 60}, + "frame_rate": {"type": "integer", "minimum": 15, "maximum": 60}, + "zoom_speed": {"type": "number", "minimum": 0, "maximum": 100}, + "webhook_url": {"type": "string", "format": "uri"}, + "id": {"type": "string"} + }, + "required": ["image_url"], + "additionalProperties": False +} +``` ### Example Request @@ -35,7 +55,7 @@ The request body must be a JSON object with the following properties: "frame_rate": 24, "zoom_speed": 5, "webhook_url": "https://example.com/webhook", - "id": "unique-request-id" + "id": "request-123" } ``` @@ -43,7 +63,7 @@ The request body must be a JSON object with the following properties: curl -X POST \ -H "x-api-key: YOUR_API_KEY" \ -H "Content-Type: application/json" \ - -d '{"image_url": "https://example.com/image.jpg", "length": 10, "frame_rate": 24, "zoom_speed": 5, "webhook_url": "https://example.com/webhook", "id": "unique-request-id"}' \ + -d '{"image_url": "https://example.com/image.jpg", "length": 10, "frame_rate": 24, "zoom_speed": 5, "webhook_url": "https://example.com/webhook", "id": "request-123"}' \ http://your-api-url/v1/image/transform/video ``` @@ -51,76 +71,117 @@ curl -X POST \ ### Success Response -**Status Code:** `200 OK` +Upon successful conversion, the endpoint returns a JSON response with the following structure: ```json { + "code": 200, + "id": "request-123", + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", "response": "https://cloud-storage.example.com/converted-video.mp4", - "endpoint": "/v1/image/transform/video", - "code": 200 + "message": "success", + "run_time": 5.123, + "queue_time": 0.456, + "total_time": 5.579, + "pid": 12345, + "queue_id": 1234567890, + "queue_length": 0, + "build_number": "1.0.0" } ``` +The `response` field contains the URL of the converted video file uploaded to cloud storage. + ### Error Responses -**Status Code:** `400 Bad Request` +- **400 Bad Request** ```json { + "code": 400, + "id": "request-123", + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", "message": "Invalid request payload", - "endpoint": "/v1/image/transform/video", - "code": 400 + "pid": 12345, + "queue_id": 1234567890, + "queue_length": 0, + "build_number": "1.0.0" } ``` -**Status Code:** `401 Unauthorized` +- **401 Unauthorized** ```json { - "message": "Missing or invalid API key", - "endpoint": "/v1/image/transform/video", - "code": 401 + "code": 401, + "id": "request-123", + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "message": "Invalid API key", + "pid": 12345, + "queue_id": 1234567890, + "queue_length": 0, + "build_number": "1.0.0" } ``` -**Status Code:** `500 Internal Server Error` +- **429 Too Many Requests** ```json { - "message": "An error occurred while processing the request", - "endpoint": "/v1/image/transform/video", - "code": 500 + "code": 429, + "id": "request-123", + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "message": "MAX_QUEUE_LENGTH (100) reached", + "pid": 12345, + "queue_id": 1234567890, + "queue_length": 100, + "build_number": "1.0.0" } ``` -## 5. Error Handling - -The endpoint uses the `validate_payload` decorator to validate the request payload against a JSON schema. If the payload is invalid, a `400 Bad Request` error is returned. +- **500 Internal Server Error** -The `authenticate` decorator is used to verify the `x-api-key` header. If the API key is missing or invalid, a `401 Unauthorized` error is returned. +```json +{ + "code": 500, + "id": "request-123", + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "message": "Error processing image to video: ", + "pid": 12345, + "queue_id": 1234567890, + "queue_length": 0, + "build_number": "1.0.0" +} +``` -If an exception occurs during the image-to-video conversion process, a `500 Internal Server Error` is returned, and the error is logged with the `logger.error` function. +## 5. Error Handling -The main application context (`app.py`) includes error handling for the task queue. If the maximum queue length is reached, a `429 Too Many Requests` error is returned. +- **Missing or invalid parameters**: If the request payload is missing required parameters or contains invalid values, the endpoint returns a 400 Bad Request error. +- **Authentication error**: If the provided `x-api-key` header is missing or invalid, the endpoint returns a 401 Unauthorized error. +- **Queue limit reached**: If the maximum queue length (`MAX_QUEUE_LENGTH`) is set and the queue size reaches that limit, the endpoint returns a 429 Too Many Requests error. +- **Other errors**: Any other exceptions raised during the image-to-video conversion process will result in a 500 Internal Server Error response, with the error message included in the response body. ## 6. Usage Notes - The `image_url` parameter must be a valid URL pointing to an image file. -- The `length` parameter specifies the duration of the output video in seconds. -- The `frame_rate` parameter determines the number of frames per second in the output video. -- The `zoom_speed` parameter controls the speed at which the image zooms in or out during the video. A value of 0 means no zooming, and a value of 100 means the image will zoom in or out at the maximum speed. -- The `webhook_url` parameter is optional and can be used to receive a notification when the video conversion is complete. -- The `id` parameter is optional and can be used to uniquely identify the request. +- The `length` parameter specifies the duration of the output video in seconds and must be between 1 and 60. +- The `frame_rate` parameter determines the frame rate of the output video and must be between 15 and 60 frames per second. +- The `zoom_speed` parameter controls the speed of the zoom effect applied to the image during the video conversion. It is a value between 0 and 100, where 0 means no zoom, and 100 is the maximum zoom speed. +- If the `webhook_url` parameter is provided, a webhook notification will be sent to the specified URL upon completion of the conversion process. +- The `id` parameter is an optional identifier that can be associated with the request for tracking purposes. ## 7. Common Issues - Providing an invalid or inaccessible `image_url`. -- Specifying invalid or out-of-range values for `length`, `frame_rate`, or `zoom_speed`. -- Reaching the maximum queue length, which will result in a `429 Too Many Requests` error. +- Setting the `length` parameter to an extremely high value, which may result in long processing times or resource exhaustion. +- Specifying an unsupported image format. +- Network or connectivity issues that may cause the image download or video upload to fail. ## 8. Best Practices -- Validate the `image_url` parameter before sending the request to ensure it points to a valid and accessible image file. -- Use appropriate values for `length`, `frame_rate`, and `zoom_speed` based on your requirements and the capabilities of your system. -- Consider implementing rate limiting or queue management strategies to prevent overloading the server with too many requests. -- Monitor the application logs for any errors or issues that may occur during the image-to-video conversion process. \ No newline at end of file +- Validate the `image_url` parameter before submitting the request to ensure it points to a valid and accessible image file. +- Set reasonable values for the `length` and `frame_rate` parameters based on your requirements and available resources. +- Consider using the `webhook_url` parameter to receive notifications about the conversion process, especially for long-running or asynchronous requests. +- Implement proper error handling and retry mechanisms in your client application to handle potential failures or network issues. +- Monitor the API logs for any errors or warnings related to the image-to-video conversion process. +- Use the `id` parameter to associate requests with specific users, sessions, or other identifiers for better tracking and debugging. \ No newline at end of file diff --git a/docs/media/transcribe.md b/docs/media/transcribe.md index 2aa748a..6f61b5e 100644 --- a/docs/media/transcribe.md +++ b/docs/media/transcribe.md @@ -2,13 +2,12 @@ ## 1. Overview -The `/v1/media/transcribe` endpoint is part of the version 1 API and is responsible for transcribing or translating media files (audio or video) into text format. It supports various output formats, including plain text, SRT subtitles, and segmented transcripts with word-level timestamps. The endpoint integrates with the main application context by utilizing the `queue_task_wrapper` decorator, which allows for efficient task queueing and processing. +The `/v1/media/transcribe` endpoint is part of the Flask API application and is responsible for transcribing media files (audio or video) into text format. It supports various options such as generating SRT subtitles, including word-level timestamps, and translating the transcription into different languages. The endpoint can return the transcription results directly or upload them to cloud storage and provide the URLs. ## 2. Endpoint -``` -POST /v1/media/transcribe -``` +- **URL Path**: `/v1/media/transcribe` +- **HTTP Method**: `POST` ## 3. Request @@ -20,126 +19,4 @@ POST /v1/media/transcribe The request body should be a JSON object with the following parameters: -| Parameter | Type | Required | Description | -|-----------------|---------|----------|--------------------------------------------------------------| -| `media_url` | string | Yes | The URL of the media file to be transcribed or translated. | -| `task` | string | No | The task to perform, either "transcribe" or "translate". Default is "transcribe". | -| `include_text` | boolean | No | Whether to include the plain text transcript in the response. Default is `true`. | -| `include_srt` | boolean | No | Whether to include the SRT subtitle file in the response. Default is `false`. | -| `include_segments` | boolean | No | Whether to include the segmented transcript with word-level timestamps in the response. Default is `false`. | -| `word_timestamps` | boolean | No | Whether to include word-level timestamps in the segmented transcript. Default is `false`. | -| `response_type` | string | No | The type of response, either "direct" (return data directly) or "cloud" (return cloud storage URLs). Default is "direct". | -| `language` | string | No | The language code for translation (if `task` is "translate"). | -| `webhook_url` | string | No | The URL to receive a webhook notification upon completion. | -| `id` | string | No | A unique identifier for the request. | - -### Example Request - -```json -{ - "media_url": "https://example.com/media/audio.mp3", - "task": "transcribe", - "include_text": true, - "include_srt": true, - "include_segments": false, - "word_timestamps": false, - "response_type": "direct", - "webhook_url": "https://example.com/webhook" -} -``` - -```bash -curl -X POST \ - https://api.example.com/v1/media/transcribe \ - -H 'x-api-key: YOUR_API_KEY' \ - -H 'Content-Type: application/json' \ - -d '{ - "media_url": "https://example.com/media/audio.mp3", - "task": "transcribe", - "include_text": true, - "include_srt": true, - "include_segments": false, - "word_timestamps": false, - "response_type": "direct", - "webhook_url": "https://example.com/webhook" - }' -``` - -## 4. Response - -### Success Response - -**Status Code:** 200 OK - -**Response Body (direct):** - -```json -{ - "text": "This is the transcribed text.", - "srt": "This is the SRT subtitle file content.", - "segments": null -} -``` - -**Response Body (cloud):** - -```json -{ - "text": "https://cloud.example.com/transcripts/text.txt", - "srt": "https://cloud.example.com/transcripts/subtitles.srt", - "segments": null -} -``` - -### Error Responses - -**Status Code:** 400 Bad Request - -```json -{ - "message": "Invalid request payload" -} -``` - -**Status Code:** 401 Unauthorized - -```json -{ - "message": "Missing or invalid API key" -} -``` - -**Status Code:** 500 Internal Server Error - -```json -{ - "message": "An error occurred during transcription process" -} -``` - -## 5. Error Handling - -- **Missing or invalid parameters**: If any required parameters are missing or invalid, the endpoint will return a 400 Bad Request error with an appropriate error message. -- **Authentication error**: If the provided API key is missing or invalid, the endpoint will return a 401 Unauthorized error. -- **Processing error**: If an error occurs during the transcription or translation process, the endpoint will return a 500 Internal Server Error with the error message. -- **Queue overflow**: If the task queue reaches the maximum length specified by the `MAX_QUEUE_LENGTH` environment variable, the endpoint will return a 429 Too Many Requests error. - -## 6. Usage Notes - -- The `media_url` parameter should point to a valid media file (audio or video) that can be accessed by the server. -- If the `response_type` is set to "cloud", the endpoint will return URLs to the transcribed files stored in cloud storage, instead of the actual file contents. -- The `webhook_url` parameter is optional and can be used to receive a notification when the transcription process is complete. -- The `id` parameter is optional and can be used to uniquely identify the request. - -## 7. Common Issues - -- Providing an invalid or inaccessible `media_url`. -- Attempting to transcribe or translate unsupported media formats. -- Exceeding the maximum queue length, resulting in a 429 Too Many Requests error. - -## 8. Best Practices - -- Validate the `media_url` parameter before submitting the request to ensure it points to a valid and accessible media file. -- Use the `response_type` parameter wisely, considering the trade-off between response size and cloud storage costs. -- Implement proper error handling and retry mechanisms in your client application to handle potential errors and retries. -- Monitor the queue length and adjust the `MAX_QUEUE_LENGTH` environment variable as needed to prevent queue overflow. \ No newline at end of file +| Parameter | Type | Required | Description \ No newline at end of file diff --git a/docs/media/transform/media_to_mp3.md b/docs/media/transform/media_to_mp3.md index 73a2a32..23ebffd 100644 --- a/docs/media/transform/media_to_mp3.md +++ b/docs/media/transform/media_to_mp3.md @@ -2,7 +2,7 @@ ## 1. Overview -The `/v1/media/transform/mp3` endpoint is a part of the API's media transformation functionality. It allows users to convert various media files (audio or video) to MP3 format. This endpoint fits into the overall API structure as a part of the `v1` namespace, specifically under the `media/transform` category. +The `/v1/media/transform/mp3` endpoint is a part of the API's media transformation functionality. It allows users to convert various media files (audio or video) to MP3 format. This endpoint fits into the overall API structure as a part of the `v1` namespace, which is dedicated to the first version of the API. ## 2. Endpoint @@ -18,57 +18,67 @@ POST /v1/media/transform/mp3 ### Body Parameters -The request body must be in JSON format and should include the following parameters: - - `media_url` (required, string): The URL of the media file to be converted. - `webhook_url` (optional, string): The URL to receive a webhook notification upon completion. - `id` (optional, string): A unique identifier for the request. -- `bitrate` (optional, string): The desired bitrate for the output MP3 file, specified in the format `k` (e.g., `128k`). If not provided, the default bitrate of `128k` will be used. +- `bitrate` (optional, string): The desired bitrate for the MP3 output, specified in the format `k` (e.g., `128k`). If not provided, defaults to `128k`. -### Example Request +The `validate_payload` directive in the routes file enforces the following JSON schema for the request body: ```json { - "media_url": "https://example.com/video.mp4", - "webhook_url": "https://example.com/webhook", - "id": "unique-request-id", - "bitrate": "192k" + "type": "object", + "properties": { + "media_url": {"type": "string", "format": "uri"}, + "webhook_url": {"type": "string", "format": "uri"}, + "id": {"type": "string"}, + "bitrate": {"type": "string", "pattern": "^[0-9]+k$"} + }, + "required": ["media_url"], + "additionalProperties": False } ``` -```bash -curl -X POST \ - https://api.example.com/v1/media/transform/mp3 \ - -H 'x-api-key: YOUR_API_KEY' \ - -H 'Content-Type: application/json' \ - -d '{ +### Example Request + +```json +{ "media_url": "https://example.com/video.mp4", "webhook_url": "https://example.com/webhook", "id": "unique-request-id", "bitrate": "192k" - }' +} +``` + +```bash +curl -X POST \ + -H "x-api-key: YOUR_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{"media_url": "https://example.com/video.mp4", "webhook_url": "https://example.com/webhook", "id": "unique-request-id", "bitrate": "192k"}' \ + https://your-api-endpoint.com/v1/media/transform/mp3 ``` ## 4. Response ### Success Response -**Status Code:** 200 OK +Upon successful conversion, the endpoint returns a JSON response with the following structure: ```json { - "code": 200, - "id": "unique-request-id", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "response": "https://cloud.example.com/converted.mp3", - "message": "success", - "run_time": 12.345, - "queue_time": 0.123, - "total_time": 12.468, - "pid": 12345, - "queue_id": 1234567890, - "queue_length": 0, - "build_number": "1.0.0" + "endpoint": "/v1/media/transform/mp3", + "code": 200, + "id": "unique-request-id", + "job_id": "generated-job-id", + "response": "https://cloud-storage.com/converted-file.mp3", + "message": "success", + "pid": 12345, + "queue_id": 1234567890, + "run_time": 5.123, + "queue_time": 0.456, + "total_time": 5.579, + "queue_length": 0, + "build_number": "1.0.0" } ``` @@ -76,74 +86,46 @@ The `response` field contains the URL of the converted MP3 file uploaded to clou ### Error Responses -**Status Code:** 400 Bad Request - -```json -{ - "code": 400, - "id": "unique-request-id", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "Invalid request payload", - "pid": 12345, - "queue_id": 1234567890, - "queue_length": 0, - "build_number": "1.0.0" -} -``` +- **400 Bad Request**: Returned when the request payload is invalid or missing required parameters. +- **401 Unauthorized**: Returned when the `x-api-key` header is missing or invalid. +- **500 Internal Server Error**: Returned when an unexpected error occurs during the conversion process. -**Status Code:** 401 Unauthorized +Example error response: ```json { - "code": 401, - "id": "unique-request-id", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "Missing or invalid API key", - "pid": 12345, - "queue_id": 1234567890, - "queue_length": 0, - "build_number": "1.0.0" + "code": 400, + "message": "Invalid request payload: 'media_url' is a required property" } ``` -**Status Code:** 500 Internal Server Error +## 5. Error Handling -```json -{ - "code": 500, - "id": "unique-request-id", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "An error occurred during media conversion", - "pid": 12345, - "queue_id": 1234567890, - "queue_length": 0, - "build_number": "1.0.0" -} -``` +The endpoint handles the following common errors: -## 5. Error Handling +- Missing or invalid `media_url` parameter: Returns a 400 Bad Request error. +- Invalid `bitrate` parameter: Returns a 400 Bad Request error. +- Authentication failure: Returns a 401 Unauthorized error. +- Unexpected exceptions during the conversion process: Returns a 500 Internal Server Error error. -- **Missing or Invalid Parameters**: If the required `media_url` parameter is missing or invalid, or if any other provided parameters are invalid, the endpoint will return a 400 Bad Request error. -- **Authentication Error**: If the `x-api-key` header is missing or invalid, the endpoint will return a 401 Unauthorized error. -- **Internal Server Error**: If an unexpected error occurs during the media conversion process, the endpoint will return a 500 Internal Server Error. -- **Queue Limit Reached**: If the maximum queue length (`MAX_QUEUE_LENGTH`) is set and the task queue has reached its limit, the endpoint will return a 429 Too Many Requests error. +Additionally, the main application context (`app.py`) includes error handling for queue overload. If the maximum queue length is reached, the endpoint returns a 429 Too Many Requests error. ## 6. Usage Notes -- The `media_url` parameter should point to a valid media file (audio or video) that can be converted to MP3 format. -- If the `webhook_url` parameter is provided, a webhook notification will be sent to the specified URL upon completion of the conversion process. -- The `id` parameter can be used to uniquely identify the request, which can be helpful for tracking and logging purposes. -- The `bitrate` parameter allows you to specify the desired bitrate for the output MP3 file. If not provided, the default bitrate of `128k` will be used. +- The `media_url` parameter should point to a valid media file (audio or video) that can be processed by the conversion service. +- The `webhook_url` parameter is optional and can be used to receive a notification when the conversion is complete. +- The `id` parameter is optional and can be used to associate the request with a specific identifier. +- The `bitrate` parameter is optional and defaults to `128k` if not provided. ## 7. Common Issues - Providing an invalid or inaccessible `media_url`. -- Attempting to convert unsupported media formats. -- Providing an invalid `bitrate` value (e.g., not following the `k` format). +- Specifying an invalid `bitrate` value (e.g., not following the `k` format). +- Reaching the maximum queue length, resulting in a 429 Too Many Requests error. ## 8. Best Practices -- Always provide a valid `x-api-key` header for authentication. -- Consider using the `webhook_url` parameter to receive notifications about the conversion process completion. -- Ensure that the provided `media_url` is accessible and points to a valid media file. -- If you need to convert multiple media files, consider using a queue or batch processing approach to avoid overwhelming the server with too many concurrent requests. \ No newline at end of file +- Validate the `media_url` parameter before sending the request to ensure it points to a valid and accessible media file. +- Use the `id` parameter to associate requests with specific identifiers for better tracking and debugging. +- Consider providing a `webhook_url` to receive notifications about the conversion status and the resulting file URL. +- Monitor the queue length and adjust the `MAX_QUEUE_LENGTH` environment variable accordingly to prevent overloading the system. \ No newline at end of file diff --git a/docs/toolkit/authenticate.md b/docs/toolkit/authenticate.md index 68b6deb..dec3e3b 100644 --- a/docs/toolkit/authenticate.md +++ b/docs/toolkit/authenticate.md @@ -1,20 +1,19 @@ -# Authenticate Endpoint +# Authenticate Endpoint Documentation ## 1. Overview -The `/v1/toolkit/authenticate` endpoint is a part of the `v1_toolkit_auth` blueprint and is responsible for authenticating requests to the API. It checks if the provided API key matches the expected value, and if so, grants access to the API. This endpoint is essential for securing the API and ensuring that only authorized clients can access the available resources. +The `/v1/toolkit/authenticate` endpoint is a part of the `v1_toolkit_auth` blueprint in the Flask application. Its purpose is to authenticate requests by verifying the provided API key against a predefined value. This endpoint serves as a gatekeeper for accessing other protected endpoints within the API. ## 2. Endpoint -``` -GET /v1/toolkit/authenticate -``` +- **URL Path**: `/v1/toolkit/authenticate` +- **HTTP Method**: `GET` ## 3. Request ### Headers -- `X-API-Key` (required): The API key used for authentication. +- `X-API-Key` (required): The API key to be verified for authentication. ### Body Parameters @@ -30,49 +29,63 @@ curl -X GET -H "X-API-Key: YOUR_API_KEY" http://localhost:8080/v1/toolkit/authen ### Success Response -**Status Code:** 200 OK - -**Response Body:** +If the provided API key matches the predefined value (`API_KEY` environment variable), the endpoint will return a 200 OK status code with the following response: ```json { - "message": "Authorized", + "code": 200, "endpoint": "/authenticate", - "code": 200 + "id": null, + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "message": "success", + "pid": 12345, + "queue_id": 1234567890, + "queue_length": 0, + "response": "Authorized", + "run_time": 0.001, + "total_time": 0.001, + "build_number": "1.0.0" } ``` ### Error Responses -**Status Code:** 401 Unauthorized - -**Response Body:** +If the provided API key does not match the predefined value, the endpoint will return a 401 Unauthorized status code with the following response: ```json { - "message": "Unauthorized", + "code": 401, "endpoint": "/authenticate", - "code": 401 + "id": null, + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "message": "Unauthorized", + "pid": 12345, + "queue_id": 1234567890, + "queue_length": 0, + "response": null, + "run_time": 0.001, + "total_time": 0.001, + "build_number": "1.0.0" } ``` ## 5. Error Handling -- If the provided `X-API-Key` header does not match the expected `API_KEY` value, the endpoint will return a 401 Unauthorized response. -- The main application context (`app.py`) includes error handling for the queue length. If the maximum queue length (`MAX_QUEUE_LENGTH`) is reached, the endpoint will return a 429 Too Many Requests response. +The main error that can occur with this endpoint is providing an invalid or missing API key. In such cases, the endpoint will return a 401 Unauthorized status code with an appropriate error message. ## 6. Usage Notes -- This endpoint should be called before making any other requests to the API to ensure that the client is authenticated. -- The `X-API-Key` header must be included in all subsequent requests to the API. +- This endpoint is designed to be used as a gatekeeper for other protected endpoints within the API. +- The API key must be provided in the `X-API-Key` header for every request to this endpoint. +- The API key is typically a secret value that should be kept secure and not shared publicly. ## 7. Common Issues - Forgetting to include the `X-API-Key` header in the request. -- Using an incorrect or expired API key. +- Providing an incorrect or invalid API key. ## 8. Best Practices -- Keep the API key secure and avoid exposing it in client-side code or publicly accessible repositories. -- Consider implementing additional security measures, such as rate limiting or IP whitelisting, to further secure the API. -- Regularly rotate the API key to mitigate the risk of unauthorized access. \ No newline at end of file +- Keep the API key secure and avoid storing it in plaintext or committing it to version control systems. +- Consider implementing additional security measures, such as rate limiting or IP whitelisting, to further protect the authentication endpoint. +- Regularly rotate or update the API key to enhance security. \ No newline at end of file diff --git a/docs/toolkit/test.md b/docs/toolkit/test.md index 7f3ecd2..b661a72 100644 --- a/docs/toolkit/test.md +++ b/docs/toolkit/test.md @@ -2,7 +2,7 @@ ## 1. Overview -The `/v1/toolkit/test` endpoint is a part of the NCA Toolkit API and is designed to test the API setup. It creates a temporary file, uploads it to cloud storage, and then returns the upload URL. This endpoint serves as a simple test to ensure that the API is correctly installed and configured. +The `/v1/toolkit/test` endpoint is a part of the NCA Toolkit API and is designed to test the API setup. It creates a temporary file, uploads it to cloud storage, and then returns the upload URL. This endpoint serves as a simple test to ensure that the API is properly configured and functioning correctly. ## 2. Endpoint @@ -31,57 +31,66 @@ curl -X GET \ ### Success Response -**Status Code:** `200 OK` - -**Example JSON Response:** - ```json { - "response": "https://cloud.example.com/success.txt", "endpoint": "/v1/toolkit/test", - "code": 200 + "code": 200, + "id": null, + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "response": "https://storage.example.com/success.txt", + "message": "success", + "pid": 12345, + "queue_id": 1234567890, + "run_time": 0.123, + "queue_time": 0.0, + "total_time": 0.123, + "queue_length": 0, + "build_number": "1.0.0" } ``` ### Error Responses -**Status Code:** `401 Unauthorized` - -**Example JSON Response:** +**Status Code: 401 Unauthorized** ```json { - "message": "Invalid or missing API key", - "code": 401 + "code": 401, + "message": "Unauthorized" } ``` -**Status Code:** `500 Internal Server Error` - -**Example JSON Response:** +**Status Code: 500 Internal Server Error** ```json { - "message": "An error occurred while testing the API setup", - "code": 500 + "code": 500, + "id": null, + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "message": "Error message", + "pid": 12345, + "queue_id": 1234567890, + "queue_length": 0, + "build_number": "1.0.0" } ``` ## 5. Error Handling -- **Missing or Invalid API Key:** If the `x-api-key` header is missing or invalid, the endpoint will return a `401 Unauthorized` status code with an appropriate error message. -- **Internal Server Error:** If an unexpected error occurs during the execution of the endpoint, it will return a `500 Internal Server Error` status code with an error message. +- **401 Unauthorized**: This error occurs when the provided `x-api-key` is missing or invalid. +- **500 Internal Server Error**: This error occurs when an unexpected exception is raised during the execution of the endpoint. ## 6. Usage Notes -This endpoint is primarily used for testing purposes and does not require any specific input parameters. It can be called to verify that the API is correctly installed and configured. +This endpoint is primarily used for testing purposes and does not require any specific input parameters. It can be called to verify that the API is set up correctly and can successfully upload files to cloud storage. ## 7. Common Issues -- Incorrect API key: Ensure that the provided `x-api-key` header value is valid and has the necessary permissions. -- Network connectivity issues: Make sure that the API server is accessible and that there are no network connectivity problems. +- Incorrect or missing `x-api-key` header. +- Temporary file creation or upload failures due to permissions or storage issues. ## 8. Best Practices -- Use this endpoint as a part of your API testing and monitoring processes to ensure that the API is functioning correctly. -- Regularly rotate and update your API keys to maintain security. \ No newline at end of file +- Use this endpoint as a simple health check to ensure the API is functioning correctly before attempting more complex operations. +- Regularly test the API setup, especially after updates or changes to the environment. +- Monitor the API logs for any errors or exceptions that may occur during the test. \ No newline at end of file diff --git a/docs/video/caption_video.md b/docs/video/caption_video.md index 70f3f64..43f1431 100644 --- a/docs/video/caption_video.md +++ b/docs/video/caption_video.md @@ -2,33 +2,32 @@ ## 1. Overview -The `/v1/video/caption` endpoint is part of the Video API in version 1 of the application. It allows users to add captions to a video file, with various customization options for the caption appearance and behavior. This endpoint utilizes the enhanced `process_captioning_v1` service, which provides more advanced captioning capabilities compared to the previous version. +The `/v1/video/caption` endpoint is part of the Video API and is responsible for adding captions to a video file. It accepts a video URL, caption text, and various styling options for the captions. The endpoint utilizes the `process_captioning_v1` service to generate a captioned video file, which is then uploaded to cloud storage, and the cloud URL is returned in the response. ## 2. Endpoint -``` -POST /v1/video/caption -``` +**URL:** `/v1/video/caption` +**Method:** `POST` ## 3. Request ### Headers -- `x-api-key` (required): The API key for authentication. +- `x-api-key`: Required. The API key for authentication. ### Body Parameters The request body should be a JSON object with the following properties: -- `video_url` (required, string): The URL of the video file to be captioned. -- `captions` (optional, string): The caption text to be added to the video. -- `settings` (optional, object): An object containing various settings for customizing the caption appearance and behavior. See the schema below for available options. -- `replace` (optional, array): An array of objects with `find` and `replace` properties, specifying text replacements to be made in the captions. -- `webhook_url` (optional, string): The URL to receive a webhook notification when the captioning process is complete. -- `id` (optional, string): A unique identifier for the request. -- `language` (optional, string): The language code for the captions (e.g., 'en', 'fr'). Defaults to 'auto'. +- `video_url` (string, required): The URL of the video file to be captioned. +- `captions` (string, optional): The caption text to be added to the video. +- `settings` (object, optional): An object containing various styling options for the captions. See the schema below for available options. +- `replace` (array, optional): An array of objects with `find` and `replace` properties, specifying text replacements to be made in the captions. +- `webhook_url` (string, optional): A URL to receive a webhook notification when the captioning process is complete. +- `id` (string, optional): An identifier for the request. +- `language` (string, optional): The language code for the captions (e.g., "en", "fr"). Defaults to "auto". -The `settings` object has the following schema: +#### Settings Schema ```json { @@ -90,7 +89,7 @@ The `settings` object has the following schema: "alignment": "left", "font_family": "Arial", "font_size": 24, - "bold": true, + "bold": false, "italic": false, "underline": false, "strikeout": false, @@ -107,7 +106,7 @@ The `settings` object has the following schema: } ], "webhook_url": "https://example.com/webhook", - "id": "unique-request-id", + "id": "request-123", "language": "en" } ``` @@ -132,7 +131,7 @@ curl -X POST \ "alignment": "left", "font_family": "Arial", "font_size": 24, - "bold": true, + "bold": false, "italic": false, "underline": false, "strikeout": false, @@ -149,7 +148,7 @@ curl -X POST \ } ], "webhook_url": "https://example.com/webhook", - "id": "unique-request-id", + "id": "request-123", "language": "en" }' ``` @@ -158,86 +157,104 @@ curl -X POST \ ### Success Response -**Status Code:** 200 OK +The response will be a JSON object with the following properties: + +- `code` (integer): The HTTP status code (200 for success). +- `id` (string): The request identifier, if provided in the request. +- `job_id` (string): A unique identifier for the job. +- `response` (string): The cloud URL of the captioned video file. +- `message` (string): A success message. +- `pid` (integer): The process ID of the worker that processed the request. +- `queue_id` (integer): The ID of the queue used for processing the request. +- `run_time` (float): The time taken to process the request (in seconds). +- `queue_time` (float): The time the request spent in the queue (in seconds). +- `total_time` (float): The total time taken to process the request, including queue time (in seconds). +- `queue_length` (integer): The current length of the processing queue. +- `build_number` (string): The build number of the application. ```json { - "response": "https://cloud.example.com/captioned-video.mp4", - "endpoint": "/v1/video/caption", "code": 200, - "id": "unique-request-id", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "id": "request-123", + "job_id": "d290f1ee-6c54-4b01-90e6-d701748f0851", + "response": "https://storage.example.com/captioned-video.mp4", "message": "success", - "pid": 12345, - "queue_id": 6789, + "pid": 123, + "queue_id": 140567495862272, "run_time": 5.234, - "queue_time": 0.123, - "total_time": 5.357, + "queue_time": 0.012, + "total_time": 5.246, "queue_length": 0, "build_number": "1.0.0" } ``` -The response contains the URL of the captioned video file uploaded to cloud storage. - ### Error Responses -**Status Code:** 400 Bad Request +#### 400 Bad Request + +Returned when there is an issue with the request payload, such as missing or invalid parameters. ```json { - "error": "Invalid video URL", - "endpoint": "/v1/video/caption", "code": 400, - "id": "unique-request-id", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "Invalid video URL", - "pid": 12345, - "queue_id": 6789, - "run_time": 0.001, - "queue_time": 0.0, - "total_time": 0.001, + "id": "request-123", + "job_id": "d290f1ee-6c54-4b01-90e6-d701748f0851", + "message": "Invalid request payload", + "pid": 123, + "queue_id": 140567495862272, "queue_length": 0, "build_number": "1.0.0" } ``` -**Status Code:** 400 Bad Request (Font Error) +#### 400 Font Error + +Returned when there is an issue with the requested font family. ```json { - "error": "Font 'CustomFont' is not available. Please choose from the available fonts.", + "error": "The requested font family 'InvalidFont' is not available. Please choose from the available fonts.", "available_fonts": ["Arial", "Times New Roman", "Courier New", ...], - "endpoint": "/v1/video/caption", "code": 400, - "id": "unique-request-id", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "Font 'CustomFont' is not available. Please choose from the available fonts.", - "pid": 12345, - "queue_id": 6789, - "run_time": 0.001, - "queue_time": 0.0, - "total_time": 0.001, + "id": "request-123", + "job_id": "d290f1ee-6c54-4b01-90e6-d701748f0851", + "pid": 123, + "queue_id": 140567495862272, "queue_length": 0, "build_number": "1.0.0" } ``` -**Status Code:** 500 Internal Server Error +#### 429 Too Many Requests + +Returned when the processing queue has reached its maximum length. + +```json +{ + "code": 429, + "id": "request-123", + "job_id": "d290f1ee-6c54-4b01-90e6-d701748f0851", + "message": "MAX_QUEUE_LENGTH (100) reached", + "pid": 123, + "queue_id": 140567495862272, + "queue_length": 100, + "build_number": "1.0.0" +} +``` + +#### 500 Internal Server Error + +Returned when an unexpected error occurs during the captioning process. ```json { - "error": "An unexpected error occurred during captioning process.", - "endpoint": "/v1/video/caption", "code": 500, - "id": "unique-request-id", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "An unexpected error occurred during captioning process.", - "pid": 12345, - "queue_id": 6789, - "run_time": 0.001, - "queue_time": 0.0, - "total_time": 0.001, + "id": "request-123", + "job_id": "d290f1ee-6c54-4b01-90e6-d701748f0851", + "message": "An unexpected error occurred during the captioning process.", + "pid": 123, + "queue_id": 140567495862272, "queue_length": 0, "build_number": "1.0.0" } @@ -247,33 +264,30 @@ The response contains the URL of the captioned video file uploaded to cloud stor The endpoint handles the following common errors: -- **Missing or invalid parameters**: If required parameters are missing or have an invalid format, a 400 Bad Request error is returned with an appropriate error message. -- **Font errors**: If the specified font is not available, a 400 Bad Request error is returned with a list of available fonts. -- **Unexpected errors**: If an unexpected error occurs during the captioning process, a 500 Internal Server Error is returned with a generic error message. - -Additionally, the main application context (`app.py`) includes error handling for queue overload. If the maximum queue length (`MAX_QUEUE_LENGTH`) is set and the queue size reaches that limit, a 429 Too Many Requests error is returned with a message indicating that the queue is full. +- **Missing or invalid parameters**: If any required parameters are missing or invalid, a 400 Bad Request error is returned. +- **Font error**: If the requested font family is not available, a 400 Bad Request error is returned, along with a list of available fonts. +- **Queue limit reached**: If the processing queue has reached its maximum length (determined by the `MAX_QUEUE_LENGTH` environment variable), a 429 Too Many Requests error is returned. +- **Unexpected errors**: If an unexpected error occurs during the captioning process, a 500 Internal Server Error is returned. ## 6. Usage Notes - The `video_url` parameter is required, and it should be a valid URL pointing to a video file. - The `captions` parameter is optional. If not provided, the video will be processed without captions. -- The `settings` parameter allows customizing various aspects of the caption appearance and behavior. Refer to the schema for available options. -- The `replace` parameter allows specifying text replacements to be made in the captions. +- The `settings` parameter allows you to customize the appearance and behavior of the captions. Refer to the settings schema for available options. +- The `replace` parameter allows you to specify text replacements to be made in the captions. - The `webhook_url` parameter is optional. If provided, a webhook notification will be sent to the specified URL when the captioning process is complete. -- The `id` parameter is optional and can be used to uniquely identify the request. -- The `language` parameter is optional and specifies the language code for the captions. If not provided, it defaults to 'auto'. +- The `id` parameter is optional and can be used to identify the request. +- The `language` parameter is optional and specifies the language of the captions. If not provided, the language will be automatically detected. ## 7. Common Issues -- Providing an invalid or inaccessible `video_url`. -- Specifying invalid or unsupported values for the `settings` parameters. -- Requesting an unavailable font family. -- Exceeding the maximum queue length, if set. +- **Invalid video URL**: Ensure that the `video_url` parameter points to a valid and accessible video file. +- **Unsupported video format**: The captioning process may not support certain video formats. If you encounter issues, try converting the video to a more common format like MP4 or AVI. +- **Font availability**: The captioning process may not have access to certain font families. If you encounter a font error, choose from the list of available fonts provided in the error response. ## 8. Best Practices -- Validate the `video_url` parameter before sending the request to ensure it points to a valid and accessible video file. -- Test the caption settings with sample videos to ensure the desired appearance and behavior. -- Use the `replace` parameter judiciously to avoid unintended text replacements in the captions. -- Monitor the queue length and adjust the `MAX_QUEUE_LENGTH` setting as needed to prevent queue overload. -- Implement error handling and retry mechanisms in your client application to handle potential errors and failures. \ No newline at end of file +- **Validate input**: Always validate the input parameters to ensure they meet the expected format and requirements. +- **Use webhooks**: Utilize the `webhook_url` parameter to receive notifications when the captioning process is complete, rather than polling for the result. +- **Optimize settings**: Adjust the caption settings based on the video resolution and content to ensure readability and aesthetics. +- **Test locally**: Before deploying to production, test the captioning process locally with various video files and settings to identify and resolve any issues. \ No newline at end of file diff --git a/docs/video/concatenate.md b/docs/video/concatenate.md index 204f2d1..bae4c1c 100644 --- a/docs/video/concatenate.md +++ b/docs/video/concatenate.md @@ -2,7 +2,7 @@ ## 1. Overview -The `/v1/video/concatenate` endpoint is part of the Flask API application and is responsible for combining multiple video files into a single video file. This endpoint fits into the overall API structure as a part of the `v1` blueprint, which is dedicated to handling version 1 of the API. +The `/v1/video/concatenate` endpoint is part of the Video API and is responsible for combining multiple video files into a single video file. This endpoint fits into the overall API structure by providing a way to concatenate videos, which can be useful in various scenarios, such as creating video compilations or combining multiple video segments into a single file. ## 2. Endpoint @@ -19,131 +19,141 @@ The `/v1/video/concatenate` endpoint is part of the Flask API application and is The request body must be a JSON object with the following properties: -- `video_urls` (required, array of objects): An array of video URLs to be combined. Each object in the array must have a `video_url` property (string, URI format) containing the URL of the video file. -- `webhook_url` (optional, string, URI format): The URL to which the webhook notification will be sent upon completion of the video combination process. -- `id` (optional, string): An identifier for the request. +- `video_urls` (required, array of objects): An array of video URLs to be concatenated. Each object in the array must have a `video_url` property (string, URI format) containing the URL of the video file. +- `webhook_url` (optional, string, URI format): The URL to receive a webhook notification when the video concatenation process is complete. +- `id` (optional, string): A unique identifier for the request. -#### Example Request +The `validate_payload` decorator in the route file enforces the following JSON schema for the request body: ```json { - "video_urls": [ - { - "video_url": "https://example.com/video1.mp4" + "type": "object", + "properties": { + "video_urls": { + "type": "array", + "items": { + "type": "object", + "properties": { + "video_url": {"type": "string", "format": "uri"} + }, + "required": ["video_url"] + }, + "minItems": 1 + }, + "webhook_url": {"type": "string", "format": "uri"}, + "id": {"type": "string"} }, - { - "video_url": "https://example.com/video2.mp4" - } - ], - "webhook_url": "https://example.com/webhook", - "id": "request-123" + "required": ["video_urls"], + "additionalProperties": False } ``` -```bash -curl -X POST \ - https://api.example.com/v1/video/concatenate \ - -H 'x-api-key: YOUR_API_KEY' \ - -H 'Content-Type: application/json' \ - -d '{ +### Example Request + +```json +{ "video_urls": [ - { - "video_url": "https://example.com/video1.mp4" - }, - { - "video_url": "https://example.com/video2.mp4" - } + {"video_url": "https://example.com/video1.mp4"}, + {"video_url": "https://example.com/video2.mp4"}, + {"video_url": "https://example.com/video3.mp4"} ], "webhook_url": "https://example.com/webhook", - "id": "request-123" - }' + "id": "unique-request-id" +} +``` + +```bash +curl -X POST \ + -H "x-api-key: YOUR_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "video_urls": [ + {"video_url": "https://example.com/video1.mp4"}, + {"video_url": "https://example.com/video2.mp4"}, + {"video_url": "https://example.com/video3.mp4"} + ], + "webhook_url": "https://example.com/webhook", + "id": "unique-request-id" + }' \ + https://your-api-endpoint.com/v1/video/concatenate ``` ## 4. Response ### Success Response -**Status Code:** `200 OK` - -**Response Body:** +The success response follows the general response format defined in the `app.py` file. Here's an example: ```json { - "code": 200, - "id": "request-123", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "response": "https://example.com/combined-video.mp4", - "message": "success", - "run_time": 10.234, - "queue_time": 0.0, - "total_time": 10.234, - "pid": 12345, - "queue_id": 1234567890, - "queue_length": 0, - "build_number": "1.0.0" + "endpoint": "/v1/video/concatenate", + "code": 200, + "id": "unique-request-id", + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "response": "https://cloud-storage.com/combined-video.mp4", + "message": "success", + "pid": 12345, + "queue_id": 67890, + "run_time": 10.234, + "queue_time": 2.345, + "total_time": 12.579, + "queue_length": 0, + "build_number": "1.0.0" } ``` -The response contains the URL of the combined video file uploaded to cloud storage. +The `response` field contains the URL of the combined video file uploaded to cloud storage. ### Error Responses -**Status Code:** `500 Internal Server Error` +- **400 Bad Request**: Returned when the request body is missing or invalid. +- **401 Unauthorized**: Returned when the `x-api-key` header is missing or invalid. +- **429 Too Many Requests**: Returned when the maximum queue length is reached (if configured). +- **500 Internal Server Error**: Returned when an unexpected error occurs during the video concatenation process. -**Response Body:** +Example error response: ```json { - "code": 500, - "id": "request-123", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "Error message describing the issue", - "pid": 12345, - "queue_id": 1234567890, - "queue_length": 0, - "build_number": "1.0.0" + "code": 400, + "id": "unique-request-id", + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "message": "Invalid request payload: 'video_urls' is a required property", + "pid": 12345, + "queue_id": 67890, + "queue_length": 0, + "build_number": "1.0.0" } ``` -**Status Code:** `429 Too Many Requests` +## 5. Error Handling -**Response Body:** +The endpoint handles the following common errors: -```json -{ - "code": 429, - "id": "request-123", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "MAX_QUEUE_LENGTH (10) reached", - "pid": 12345, - "queue_id": 1234567890, - "queue_length": 10, - "build_number": "1.0.0" -} -``` - -## 5. Error Handling +- Missing or invalid request parameters: Returns a 400 Bad Request error with a descriptive error message. +- Authentication failure: Returns a 401 Unauthorized error if the `x-api-key` header is missing or invalid. +- Queue length exceeded: Returns a 429 Too Many Requests error if the maximum queue length is reached (if configured). +- Unexpected exceptions during video concatenation: Returns a 500 Internal Server Error with the exception message. -- Missing or invalid `video_urls` parameter: Returns a `400 Bad Request` error. -- Invalid `webhook_url` format: Returns a `400 Bad Request` error. -- Error during the video combination process: Returns a `500 Internal Server Error` with the error message. -- If the maximum queue length is reached (determined by the `MAX_QUEUE_LENGTH` environment variable), the endpoint returns a `429 Too Many Requests` error. +The main application context (`app.py`) also includes error handling for queue-related errors, such as reaching the maximum queue length. ## 6. Usage Notes -- The `video_urls` parameter must contain at least one video URL. -- The `webhook_url` parameter is optional and can be used to receive a notification when the video combination process is complete. -- The `id` parameter is optional and can be used to identify the request. +- The order of the video files in the `video_urls` array determines the order in which they will be concatenated. +- The endpoint supports various video file formats, but the specific supported formats may depend on the underlying video processing library (e.g., FFmpeg). +- The combined video file will be uploaded to cloud storage, and the response will include the URL of the uploaded file. +- If a `webhook_url` is provided, a webhook notification will be sent to that URL when the video concatenation process is complete. ## 7. Common Issues -- Providing invalid video URLs or URLs that are not accessible. -- Attempting to combine video files with incompatible codecs or formats. -- Reaching the maximum queue length, which can cause requests to be rejected with a `429 Too Many Requests` error. +- Providing invalid or inaccessible video URLs in the `video_urls` array. +- Exceeding the maximum queue length, if configured, which can result in a 429 Too Many Requests error. +- Encountering issues during the video concatenation process, such as unsupported video formats or corrupted video files. ## 8. Best Practices -- Ensure that the provided video URLs are valid and accessible. -- Consider the video file formats and codecs when combining videos to avoid compatibility issues. -- Monitor the queue length and adjust the `MAX_QUEUE_LENGTH` environment variable as needed to prevent the queue from becoming overloaded. -- Implement error handling and retries for failed requests. \ No newline at end of file +- Validate the video URLs before sending the request to ensure they are accessible and in a supported format. +- Monitor the queue length and adjust the maximum queue length as needed to prevent overloading the system. +- Implement retry mechanisms for failed requests or consider using a message queue system for better reliability and scalability. +- Implement proper error handling and logging to aid in troubleshooting and monitoring. +- Consider implementing rate limiting or throttling mechanisms to prevent abuse or excessive load on the system. \ No newline at end of file diff --git a/generate_docs.py b/generate_docs.py old mode 100644 new mode 100755 index 836dedb..b07af70 --- a/generate_docs.py +++ b/generate_docs.py @@ -44,23 +44,30 @@ def load_app_context(): I am providing you with a Python file containing API endpoint definitions. First, here is the main application context from app.py that shows how the API is structured and handled: - + +** app.py below + {app_context} +** app.py DONE + Now, please read through the following endpoint code and analyze it in the context of the main application: - + +**endpoint below + {file_content} Please generate detailed documentation in Markdown format as follows: + 1. Overview: Describe the purpose of the endpoint and how it fits into the overall API structure shown in app.py. 2. Endpoint: Specify the URL path and HTTP method. 3. Request: - Headers: List any required headers, such as the x-api-key headers. - Body Parameters: List the required and optional parameters, including the parameter type and purpose. - - specifically study the validate_payload directive in the routes files to build the documentation + - Specifically study the validate_payload directive in the routes file to build the documentation - Example Request: Provide a sample request payload and a sample curl command. 4. Response: - - Success Response: Show the status code, and provide an example JSON response for a successful request. + - Success Response: Reference the endpoint and general response from the app.py to show a full sample response from the api - Error Responses: Include examples of common error status codes, with example JSON responses for each. 5. Error Handling: - Describe common errors, like missing or invalid parameters, and indicate which status codes they produce