Editorial Advanced Creator #58

jakubjezek001 · 2024-12-16T12:26:39Z

Changelog description

This PR introduces a significant improvement to the Editorial workflow by splitting it into two distinct approaches - Simple and Advanced. The Simple workflow maintains the current functionality for basic editorial operations, while the new Advanced workflow provides more sophisticated features for handling complex media folder structures, multiple file types, and enhanced representation capabilities.

The Advanced workflow allows for more flexible content discovery in clip folders, better version handling, and improved control over how different media types are processed and published. This makes it especially useful for studios with complex editorial pipelines involving various media types and folder structures.

Additional information

Split editorial creator into Simple and Advanced workflows
Moved existing editorial creator to 'editorial_simple.py' for clarity
Added new 'editorial_advanced.py' with enhanced features:
- Advanced folder structure scanning
- Improved version detection from filenames
- More flexible product type handling
- Enhanced representation filtering system
- Support for multiple media types (images, sequences, video, audio)
- Better thumbnail handling
- Configurable version handling (incremental, from file, locked)
Updated settings structure to support both workflows
Enhanced product type presets with more detailed configuration options
Improved validation for product type configurations
Added support for custom tags in representations
Updated collectors to handle new advanced workflow features
Enhanced documentation and tooltips in settings

Testing steps

Dependent on changes from ynput/ayon-core#1060

Basic Workflow Testing:
- Verify existing Simple workflow still functions as before
- Test publishing with EDL/XML files
- Confirm basic shot creation works
Advanced Workflow Testing:
- Test with complex folder structures containing various media types
- Verify version detection from filenames
- Check representation filtering with different file types
- Test thumbnail detection and handling
- Verify correct handling of image sequences
- Reviewable files are attached to versions
Configuration Testing:
- Test different version handling methods (incremental, from file, locked)
- Verify product type preset configurations
- Test custom tag functionality
- Check representation filtering rules
Migration Testing:
- Verify existing projects continue to work with Simple workflow
- Test migration of existing configurations to new format
Edge Cases:
- Test with missing or incomplete media
- Verify handling of unusual folder structures
- Check behavior with mixed media types

- Renamed and restructured the simple and advanced editorial creator classes. - Updated class descriptions to reflect new functionalities. - Added support for advanced workflows, including clip matching and enhanced settings. - Introduced new data models for handling multiple video file representations. - Implemented validation for file extensions in the representation model.

- Renamed class for clarity - Updated product type presets to use advanced options - Simplified default variants assignment with walrus operator - Cleaned up unnecessary content type field in settings

- Renamed `CLIP_ATTR_DEFS` to `CREATOR_CLIP_ATTR_DEFS`. - Updated labels for clarity in attribute definitions. - Changed media path handling to use a folder path instead of individual media file paths. - Added a method to get product type preset names, simplifying the logic for defining attributes. - Cleaned up some comments and organized code structure.

- Introduced new product type classes: Image, Render, Model, Camera, and Workfile. - Updated labels for existing product types to be more descriptive. - Enhanced the logic for creating clip instances with better media path handling. - Improved validation for unique product presets in settings. - Refactored code to streamline instance creation processes.

- Added regex import for better string matching. - Improved logging to show allowed product presets and clip properties. - Updated logic to filter out unnecessary data from pre-create inputs. - Enhanced folder traversal to match clips with media files more efficiently. - Introduced a new method for including files based on product names. - Changed the way product type presets are retrieved, now returning a dict instead of a list.

- Introduced a new option to ignore clips with no content. - Updated functions to handle the new parameter throughout the processing logic. - Refactored product data handling and file inclusion methods for better clarity and efficiency. - Enhanced logging for clip content processing.

- Simplified variable names for clarity. - Streamlined the loop for creating OTIO timelines. - Consolidated instance data updates to reduce redundancy. - Improved logic for checking clip content and related warnings. - Separated shot product instance creation into its own method. - Enhanced product naming logic with clearer structure.

- Introduced new regex patterns for column and reminder variants. - Updated existing regex searches to use the new patterns. - Added logging for missing product data and filtering processes. - Cleaned up code formatting and improved readability in several areas.

- Added dot prefix to extensions if missing - Made extension checks case-insensitive - Cleaned up logging statements for clarity

- Introduced a new versioning type setting. - Added regex pattern to extract version from filenames. - Updated instance creation logic to handle different versioning strategies (incremental, from file, locked). - Enhanced product data handling with grouped representations and reviewable checks.

- Introduced a new content type mapping for different media types. - Updated logic to determine product data type based on file extensions. - Enhanced the process of creating product instances with better path handling. - Improved version extraction from representation files instead of clip content items.

- Removed unnecessary log warnings for properties. - Simplified the creation of timelines by including sequence names. - Enhanced representation data structure with optional output names. - Added new families to the clip instance collector. - Introduced a function to find string differences between file names.

…h-to-ayon-38

- Updated frame rate value for consistency. - Added support for retrieving tags and custom tags in representations. - Improved pattern matching logic to handle cases with no patterns set. - Ensured thumbnail content type gets a specific tag. - Enhanced data structure to include tags when collecting clip instances.

Renamed identifiers for various editorial instance creators to include "_advanced" suffix. Updated regex patterns for variant matching and removed old patterns. Added logging statements for better debugging visibility. Adjusted collection checks in the clip instance collector to accommodate new identifiers.

- Modified the VARIANTS_PATTERN to include digits. - Enhanced suffix processing by adding a match check before replacing product names. - Updated how product names are handled in attribute definitions for better clarity and functionality. - Added default_enabled property to ProductTypeAdvancedPresetItem for improved preset management. - Cleaned up documentation formatting for consistency.

- Reorganized import statements for better clarity. - Removed unused imports to streamline the code. - Adjusted formatting for consistency and readability. - Cleaned up commented-out code sections.

Updated the condition to check for "thumb" in the suffix. Now it correctly adds the optional output name suffix when it's not a thumbnail.

client/ayon_traypublisher/plugins/create/create_editorial_simple.py

client/ayon_traypublisher/plugins/publish/collect_clip_instances.py

- Simplified superclass initialization. - Streamlined access to creator settings. - Removed redundant checks for default variants. - Consolidated key filtering logic in the create method.

- Added error handling for file path retrieval. - Simplified FPS assignment logic. - Cleaned up path data extraction from pre-create data. - Removed unnecessary logging statements to reduce clutter. - Improved clarity in return types for a function.

Updated variable names from 'files' to 'filenames' and 'folders' to 'foldernames' for better readability. This change improves code clarity when iterating through directories and handling file collections.

- Updated variable names for clarity - Changed `files` to `filenames` for consistency - Adjusted logic to filter filenames correctly - Ensured proper suffix extraction from filtered filenames

- Changed product name check to use inequality for clarity. - Made file extension checks case-insensitive by lowering extensions. - Ensured all extensions are consistently formatted in lowercase.

- Simplified the matching files filter process. - Merged content type validation into the main loop. - Removed redundant checks for extensions and patterns. - Improved readability by reducing nested conditions.

- Simplified the matching files check by continuing early if none are found. - Lowercased file extensions for consistency. - Adjusted how single matching files are handled to avoid unnecessary lists. - Cleaned up code structure for better readability and maintainability.

- Simplified the logic for checking if representations exist. - Changed how reviewable status is determined, using nested loops instead of a comprehension for better readability. - Added early exit in loops to improve efficiency when finding reviewable products.

…h-to-ayon-38

- Removed deprecated attributes from creator settings. - Updated docstring for clarity on return values. - Added specific labels to UI separators for better organization. - Streamlined whitespace and character stripping in string difference function.

Removed unused import for pretty printing. Streamlined the code by cleaning up unnecessary dependencies to improve readability and maintainability.

…al-publish-to-ayon-38' into feature/AY-7125_advanced-editorial-publish-to-ayon-38

Cleaned up the description text for versioning options by removing unnecessary escape characters. This makes it clearer and easier to read.

64qam · 2025-01-15T15:59:30Z

I got the same error as during testing of this PR for a client:

  File "C:\Users\qam\AppData\Local\Ynput\AYON\dependency_packages\ayon_2411151105_windows.zip\dependencies\pyblish\plugin.py", line 528, in __explicit_process
    runner(*args)
  File "C:\Users\qam\AppData\Local\Ynput\AYON\addons\core_1.0.13\ayon_core\plugins\publish\collect_otio_frame_ranges.py", line 44, in process
    otio_avalable_range = otio_clip.available_range()
opentimelineio._otio.CannotComputeAvailableRangeError: Cannot compute available range: No available_range set on media reference on clip: Clip("jt3dfile_0100_0060_b1c1", MissingReference('', None, None, {}), TimeRange(RationalTime(0, 25), RationalTime(9, 25)), {'cmx_3600': {'reel': 'jt3dfile00100050b1c1'}})

MilaKudr

I had the same issue like Qam with official core addon 1.0.13, but wit dev core addon (1.0.12+dev1) publishing works properly. Reviewables are missing.

jakubjezek001 · 2025-01-22T13:54:25Z

I got the same error as during testing of this PR for a client:

As @MilaKudr was mentioning, this is only happening if ynput/ayon-core#1060 is not part of your bundle.

client/ayon_traypublisher/plugins/create/create_editorial_advanced.py

Added a cautionary comment regarding the potential risks of guessing the thumbnail file. This helps clarify the code's intent and warns future developers about possible issues.

…nced.py Co-authored-by: Jakub Trllo <[email protected]>

Updated the way clip content is assigned by removing unnecessary path replacement. This makes the code cleaner and more straightforward.

Updated the way tags and custom tags are handled by using `deepcopy` to ensure modifications don't affect the original data. This change improves data integrity when working with representation presets.

…h-to-ayon-38

- Removed `media_path` from function parameters and instance data updates. - Updated logic to handle `editorialSourcePath` directly in instances. - Cleaned up comments related to trimming functionality.

…al-publish-to-ayon-38' into feature/AY-7125_advanced-editorial-publish-to-ayon-38

jakubjezek001 added 16 commits December 6, 2024 13:41

Refactor advanced editorial creator settings

38c7462

- Renamed class for clarity - Updated product type presets to use advanced options - Simplified default variants assignment with walrus operator - Cleaned up unnecessary content type field in settings

Improve file filtering logic

c944fbf

- Added dot prefix to extensions if missing - Made extension checks case-insensitive - Cleaned up logging statements for clarity

Merge branch 'develop' into feature/AY-7125_advanced-editorial-publis…

e3546f4

…h-to-ayon-38

jakubjezek001 self-assigned this Dec 16, 2024

jakubjezek001 linked an issue Dec 16, 2024 that may be closed by this pull request

AY-7125_Advanced Editorial publish to AYON #38

Open

jakubjezek001 requested review from 64qam, iLLiCiTiT, MilaKudr and robin-ynput December 16, 2024 12:26

jakubjezek001 added sponsored This is directly sponsored by a client or community member type: feature Adding something new and exciting to the product labels Dec 16, 2024

jakubjezek001 mentioned this pull request Dec 16, 2024

Refactor OTIO frame range collection ynput/ayon-core#1060

Open

jakubjezek001 added 2 commits December 16, 2024 23:48

Refactor imports and clean up code

12ccfcf

- Reorganized import statements for better clarity. - Removed unused imports to streamline the code. - Adjusted formatting for consistency and readability. - Cleaned up commented-out code sections.

Fix suffix check for output name generation

7ecdbc7

Updated the condition to check for "thumb" in the suffix. Now it correctly adds the optional output name suffix when it's not a thumbnail.

iLLiCiTiT reviewed Jan 10, 2025

View reviewed changes

client/ayon_traypublisher/plugins/create/create_editorial_simple.py Outdated Show resolved Hide resolved

iLLiCiTiT reviewed Jan 10, 2025

View reviewed changes

client/ayon_traypublisher/plugins/publish/collect_clip_instances.py Outdated Show resolved Hide resolved

iLLiCiTiT reviewed Jan 10, 2025

View reviewed changes

client/ayon_traypublisher/plugins/publish/collect_clip_instances.py Outdated Show resolved Hide resolved

jakubjezek001 added 11 commits January 14, 2025 15:15

Refactor code for clarity and efficiency

ebd5f58

- Simplified superclass initialization. - Streamlined access to creator settings. - Removed redundant checks for default variants. - Consolidated key filtering logic in the create method.

Fix variable names for clarity in file processing

bcdcd93

Updated variable names from 'files' to 'filenames' and 'folders' to 'foldernames' for better readability. This change improves code clarity when iterating through directories and handling file collections.

Refactor file handling in collection processing

f359d06

- Updated variable names for clarity - Changed `files` to `filenames` for consistency - Adjusted logic to filter filenames correctly - Ensured proper suffix extraction from filtered filenames

Fix comparison logic and improve filters

c0f26a6

- Changed product name check to use inequality for clarity. - Made file extension checks case-insensitive by lowering extensions. - Ensured all extensions are consistently formatted in lowercase.

Refactor file filtering logic

0cfb11d

- Simplified the matching files filter process. - Merged content type validation into the main loop. - Removed redundant checks for extensions and patterns. - Improved readability by reducing nested conditions.

Merge branch 'develop' into feature/AY-7125_advanced-editorial-publis…

f6248ab

…h-to-ayon-38

Clean up imports in advanced editorial creation

459aeea

Removed unused import for pretty printing. Streamlined the code by cleaning up unnecessary dependencies to improve readability and maintainability.

jakubjezek001 requested a review from iLLiCiTiT January 14, 2025 15:32

jakubjezek001 added 2 commits January 14, 2025 16:32

Merge remote-tracking branch 'origin/feature/AY-7125_advanced-editori…

484fd46

…al-publish-to-ayon-38' into feature/AY-7125_advanced-editorial-publish-to-ayon-38

Fix description formatting in version settings

b6d2c9e

Cleaned up the description text for versioning options by removing unnecessary escape characters. This makes it clearer and easier to read.

MilaKudr reviewed Jan 16, 2025

View reviewed changes

iLLiCiTiT reviewed Jan 23, 2025

View reviewed changes

client/ayon_traypublisher/plugins/create/create_editorial_advanced.py Outdated Show resolved Hide resolved

iLLiCiTiT reviewed Jan 23, 2025

View reviewed changes

client/ayon_traypublisher/plugins/create/create_editorial_advanced.py Outdated Show resolved Hide resolved

iLLiCiTiT reviewed Jan 23, 2025

View reviewed changes

client/ayon_traypublisher/plugins/create/create_editorial_advanced.py Outdated Show resolved Hide resolved

iLLiCiTiT reviewed Jan 23, 2025

View reviewed changes

client/ayon_traypublisher/plugins/create/create_editorial_advanced.py Outdated Show resolved Hide resolved

jakubjezek001 and others added 7 commits February 6, 2025 16:22

Add note about thumbnail file handling

a8bc68e

Added a cautionary comment regarding the potential risks of guessing the thumbnail file. This helps clarify the code's intent and warns future developers about possible issues.

Update client/ayon_traypublisher/plugins/create/create_editorial_adva…

934e72c

…nced.py Co-authored-by: Jakub Trllo <[email protected]>

Simplify clip content assignment

438fdab

Updated the way clip content is assigned by removing unnecessary path replacement. This makes the code cleaner and more straightforward.

Fix tag handling in representation presets

d353975

Updated the way tags and custom tags are handled by using `deepcopy` to ensure modifications don't affect the original data. This change improves data integrity when working with representation presets.

Merge branch 'develop' into feature/AY-7125_advanced-editorial-publis…

f25975e

…h-to-ayon-38

Remove unused media path references

ecfd898

- Removed `media_path` from function parameters and instance data updates. - Updated logic to handle `editorialSourcePath` directly in instances. - Cleaned up comments related to trimming functionality.

Merge remote-tracking branch 'origin/feature/AY-7125_advanced-editori…

89befc8

…al-publish-to-ayon-38' into feature/AY-7125_advanced-editorial-publish-to-ayon-38

jakubjezek001 requested review from MilaKudr and iLLiCiTiT February 6, 2025 15:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Editorial Advanced Creator #58

Editorial Advanced Creator #58

jakubjezek001 commented Dec 16, 2024 •

edited

Loading

64qam commented Jan 15, 2025

MilaKudr left a comment

jakubjezek001 commented Jan 22, 2025

Editorial Advanced Creator #58

Are you sure you want to change the base?

Editorial Advanced Creator #58

Conversation

jakubjezek001 commented Dec 16, 2024 • edited Loading

Changelog description

Additional information

Testing steps

64qam commented Jan 15, 2025

MilaKudr left a comment

Choose a reason for hiding this comment

jakubjezek001 commented Jan 22, 2025

jakubjezek001 commented Dec 16, 2024 •

edited

Loading