Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/improve resource locking scheme for operations #6

Open
wants to merge 55 commits into
base: master
Choose a base branch
from

Conversation

justin2wilson
Copy link

@justin2wilson justin2wilson commented Aug 1, 2024

This MR introduces the following changes to the way operations lock
resources:

Introduces a parent-child relationship between operations where a
"child" operation that is being run by its "parent" can share
resource locks with the parent.

In the event that an operation cannot acquire ALL of its required
resource locks, it will release any resource locks it did
successfully acquire and then retry later. This gives each thread
that is currently attempting to acquire resource locks a chance to
do so.

BobObara and others added 30 commits March 13, 2024 14:30
Added a virtual method to qtBaseTaskNode to setup a context menu for a qtTaskNode.
8136ca7 ENH: Added the ability to override context menus for qtTaskNodes

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3000
Also added an iterator section to an existing test.

Closes #530
767b5a9 BUG: Fixing Iterator API for ValueItems

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Reviewed-by: David Thompson <[email protected]>
Merge-request: !3001
SMTK XML Attribute File now supports instantiation of templates inside the Analyses
XML Element Block.
On some systems the layout was not allowing the contents to fully utilize the space available.
The widget now uses 2 layouts - a VBox for the toolbar (if needed) and a HBox for the drawer and diagram itself.
014d8f4 ENH: Replaced GridLayout in qtDiagram

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Reviewed-by: David Thompson <[email protected]>
Merge-request: !3005
6501704 Fix a reference-to-temporary issue.
fe03bef Fix a connect mode crash; allow modes to observe operations.

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3006
837e48a ENH: Added Template Support When Defining Analyses

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3004
You can now specify a category constraint as a category expression instead of defining it as sets of included and excluded category names.
This not only provided greater flexibility but is also easier to define.  For example in an SBT file this would like the following:

          <CategoryExpression InheritanceMode="Or">(a &amp; !b) &amp; (d | 'category with spaces') </CategoryExpression>

Note that in XML &amp; represents &

In this example the expression will match if the test set of categories contains **a** and either **d** or **category with spaces** but not **b**

Also bumped the file versions of both the XML (to version 8) and JSON (version 7) for Attribute Resources in order to support these changes.
71d276f ENH: Supporting Category Expressions

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3007
This one is actually in the source tree.
937102f smtk/attribute: install the `categories/Grammar.h` header

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Acked-by: Robert M. O'Bara <[email protected]>
Merge-request: !3008
ff1731a smtk/io: fix `XmlV8StringWriter.h` filename typo

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Acked-by: Robert M. O'Bara <[email protected]>
Merge-request: !3009
This updates the bundle to agree with the Xcode usage now deployed on
machines.
d1a4899 ci: update superbuild bundle for Xcode update

Acked-by: Kitware Robot <[email protected]>
Merge-request: !3010
d1a4899 ci: update superbuild bundle for Xcode update

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3010
This allows the Diagram Drawer to be resizable.
You can now use the following symbols to represent logical operators:

* And
  * ∧, *, &
* Or
  * ∨, +, |
* Complement
  * ¬, ~, !

Also expressions can now have white spaces at the beginning and end of the expression.
a09e8d4 ENH:  Expanding the symbols that can be used in Category Expressions

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3013
66596af ENH: Adding Splitter to Diagram UI

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3012
Starting in Version 8 XML Template Files, you can now define vector-based Properties on Attribute Resources and Attributes.
Currently vector of doubles and vectors of strings are supported but this can be easily extended.

Here is an example and is available in the data/attributes/attribute_collections directory as propertiesExample.sbt.

.. code-block:: XML

<SMTK_AttributeResource Version="8">
  <Properties>
    <Property Name="pi" Type="Int"> 42 </Property>
    <Property Name="pd" Type="double"> 3.141 </Property>
    <Property Name="ps" Type="STRING">Test string</Property>
    <Property Name="pb" Type="bool"> YES </Property>
    <Property Name="animals" Type="vector[string]">
      <Value>the dog</Value>
      <Value>a cat</Value>
    </Property>
  </Properties>
  <Definitions>
    <AttDef Type="Test"/>
  </Definitions>
  <Attributes>
    <Att Name="foo" Type="Test">
      <Properties>
        <Property Name="pi" Type="int"> 69 </Property>
        <Property Name="pd" Type="Double"> 3.141 </Property>
        <Property Name="ps" Type="String"></Property>
        <Property Name="pb" Type="Bool"> 1 </Property>
        <Property Name="pvd" Type="vector[double]">
          <Value>10.0</Value>
          <Value>20.0</Value>
        </Property>
      </Properties>
    </Att>
  </Attributes>
</SMTK_AttributeResource>
4ba3180 ENH: Supporting Vector Properties in XML Attribute Templates

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Reviewed-by: David Thompson <[email protected]>
Merge-request: !3014
DoubleItems can now have units explicitly assigned to them provided that their Definition does not
specify units.  This allows Items coming from the same Definition to have different units.

Modified API

* ``DoubleItemDefinition::hasSupportedUnits`` has been moved to ValueItemDefinition

Added API

* ValueItem
  * units() - returns the native units for the item
  * supportedUnits() - returns the supported units for the item.  If there is no Units System assigned to its definition or if its units are supported by the Units System, an empty string is returned else it returns its units.
* ValueItemDefinition
  * supportedUnits() - similar in concept as ValueItem's
* DoubleItem
  * setUnits() - explicitly sets the units of the item
  * units() - overridden to support explicit units
  * hasExplicitUnits() - returns true if the item has explicit units.

When changing the units of an Item, the system will see if the Item's current input string values are compatible, if they are not, the input value units are replaced with the new ones.

See smtk/attribute/testing/cxx/unitDoubleTest.cxx for an example.

Both XML and JSON formats have been updated to support this functionality as well as qtInputsItem.cxx, qtDoubleUnitsLineEdit{.h, .cxx}.
631e337 ENH: Supporting Explicit Units for DoubleItems

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3016
SMTK now supports the ability to assign a function to an Attribute Item that will
be used to determine if an Item is currently relevant.  ``Item::isRelevant``
method has been refactored and most of the original logic has been moved to a new method called ``Item::defaultIsRelevant``.
To set a custom relevancy function, use ``Item::setCustomIsRelevant``.

To determine if an Item is relevant you will still call ``isRelevant`` and it will use the custom relevancy function if one has been set, else it will call the default method.

Please see ``smtk/attribute/testing/cxx/customIsRelevantTest.cxx`` for an example of how to use this functionality.
BobObara and others added 25 commits June 7, 2024 12:47
e8904d5 ENH: Added ability to assign custom relevancy functions to Items

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3015
The issue was that the original implementation simply reconstructed the entire widget when requesting it updated itself from its underlying item.  This potentially caused the widget to change its position w/r its siblings.

This change now prevents the widget from being rebuilt.
6366746 BUG: Fix qtReferenceItem::updateItem

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Reviewed-by: David Thompson <[email protected]>
Merge-request: !3018
SMTK now supports the ability to assign a function to a Value Item Definition that would
be used to determine if a Value Item's discrete enumeration value  is currently relevant.

Please see ``smtk/attribute/testing/cxx/customIsRelevantTest.cxx`` for an example of how to use this functionality.
0eea374 ENH: Add Custom isEnumRelevant Functionality

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3017
eed8ecf BUG: Adding Missing cassert include

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3019
0ccb188 Fix #540 by using `units::PreferredUnits`.

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Acked-by: Robert M. O'Bara <[email protected]>
Merge-request: !3020
1. Attribute Resource Associations are now copied in ``copyFinalized()``

2. Attribute Active Categories are now copied in ``copyInitialized()``

3. Attribute Resource Analyses are now copied in ``copyInitialized()``
Also added assignment method to Analyses class.

4. Properties on Resources (in particular with Attribute and Graph Resources are now copied)
The issue was that source and copied resource uuids were not being added to the object mapping in
their clone methods.  **Note: All Resources that can be cloned will need to do this.**

5. Removed undefinedRole as the default role type when creating links. This lead
to mistakes where specified roles were not being used when creating between resources.
This role type has been removed. Resource based Links now use the same value to represent an Link with an invalid role type.
Also by default, links with invalid role types  will no longer be copied when a resource is copied.

smtk/resource/testing/python/testCloneResources.py has been updated to verify these changes.

Also replaced all std::cerr and std::cout statements in smtk/attribute/Resource.cxx with Logger messages.
57b3100 BUG: Adding Issues with Copying Attribute Resources

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3021
You can now assign units to Definitions and Attributes.  This can be useful when an Attribute
represents a concept that has units but does not have an explicit value.  For example, if an
Attribute represents a temperature field that would be created by a simulation, you may want to
assign a default unit to its Definition (such as Kelvin) but allow the user to change the
Attribute's units to Celsius.

The rules for assigning local units to an Attribute that override those inherited through its definition
are the same as the case of assigning a value with units to a ValueItem.
7fd49b5 ENH: Adding Units Support for Definitions and Attributes

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Reviewed-by: David Thompson <[email protected]>
Merge-request: !3023
The method will now the attribute specific copy options if they are provided.
Also removed explicit attribute copy options that are the defaults for the case no attribute copy options are provided.

Also added some additional outputs to the testCloneResources test.

This may be related to Issue #538
Closes #541
8391421 BUG: Attribute Resource Copy Finalize Ignoring Attribute Copy Options

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Reviewed-by: David Thompson <[email protected]>
Merge-request: !3024
0432f77 Add documentation of `qtItem` design patterns.
d6fd13c Fix an issue with the way 3-d widgets were updated.

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3025
5e9f789 Copy links properly.

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3026
Fixes sequencing issue with observers when operations are launched asynchronously.
:smtk:`smtk::attribute::Definition` now derives from :smtk:`smtk::resource::Component`.
This means that Definitions now have UUIDs and may also have properties.

**Note** You do not need to specify UUIDs in either the XML or JSON file formats.  If an
ID is not specified, one will be assigned to it.

**Note** All previous JSON and XML files are supported by this change.  Version 8 XML and 7 JSON files and later
will support storing the IDs.  These formats will also support properties defined on Definitions.
As in the case for properties on the Resource and Attributes, the XML format only supports reading properties.

Developer changes
~~~~~~~~~~~~~~~~~~

** API Breakage: ** Since classes derived from the resource component class must provide a
method to return a shared pointer to a :smtk:`smtk::resource::Resource` instance via a member functions called
resource() and since the Definition class already had a method called resource() that returned a shared pointer
to its owning :smtk:`smtk::attribute::Resource`, this resulted in breaking API.  A new method called attributeResource()
was added to Definition that provides that same functionality as its original resource() method.  A simple name replacement is
all that is needed to resolve compilation errors resulting from this change.

:smtk:`smtk::attribute::Attribute::setId()` method was not being properly supported and now generates an error message if called.

The code used to parse property information in XML files has been relocated from the XMLV5Parser to its own file so it
can be reused.
fb78645 ENH: Attribute Definitions are now Resource Components

Acked-by: Kitware Robot <[email protected]>
Acked-by: buildbot <[email protected]>
Merge-request: !3027
d8a4da4 Changes to "CallObserversOnMain" behaviors.

Acked-by: Kitware Robot <[email protected]>
Tested-by: buildbot <[email protected]>
Merge-request: !3022
This MR introduces the following changes to the way operations lock
resources:

1. Introduces a parent-child relationship between operations where a
   "child" operation that is being run by its "parent" can share
   resource locks with the parent.

2. In the event that an operation cannot acquire ALL of its required
   resource locks, it will release any resource locks it did
   successfully acquire and then retry later. This gives each thread
   that is currently attempting to acquire resource locks a chance to
   do so.
@justin2wilson justin2wilson force-pushed the feature/improve-resource-locking-scheme-for-operations branch from bf7e065 to 71ee674 Compare August 1, 2024 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants