Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ProtocolTests] Refactor XML serialization codegen template, #3277

Merged
merged 4 commits into from
Jan 31, 2025

Conversation

SergeyRyabinin
Copy link
Contributor

Also fixes fixes wrong timestamp format in a corner case

Issue #, if available:
Protocol tests
Description of changes:
Refactor how XML serialization logic is generated by creating macro functions that can generate a corresponding code, instead of repeating the logic in the every case.
Check all that applies:

  • Did a review by yourself.
  • Added proper tests to cover this PR. (If tests are not applicable, explain.)
  • Checked if this PR is a breaking (APIs have been changed) change.
  • Checked if this PR will not introduce cross-platform inconsistent behavior.
  • Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

  • Linux
  • Windows
  • Android
  • MacOS
  • IOS
  • Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@SergeyRyabinin SergeyRyabinin force-pushed the sr/pt/xmlSerializationCodegenRef branch from 0f07498 to a9ff6ed Compare January 30, 2025 00:08
@@ -53,25 +53,25 @@ Aws::String XmlTimestampsRequest::SerializePayload() const
if(m_epochSecondsHasBeenSet)
{
XmlNode epochSecondsNode = parentNode.CreateChildElement("epochSeconds");
epochSecondsNode.SetText(m_epochSeconds.ToGmtString(Aws::Utils::DateFormat::ISO_8601));
epochSecondsNode.SetText(m_epochSeconds.ToGmtString(Aws::Utils::DateFormat::${timestamptFormatStr}));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

${timestamptFormatStr} this might be a missing code gen case

@@ -1,7 +1,7 @@
add_project(rest-xml-1-input-protocol-tests
"Tests for the protocol rest-xml of AWS C++ SDK"
testing-resources
aws-cpp-sdk-rest-xml-protocol-namespace
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should cmake be changing in this revision? some seem to be getting -namespace others removing -namespace anything up there?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is the issue of the code gen not being stable. Will fix soon.

for(const auto& mapItem : m_userMetadata)
{
XmlNode userMetadataMapEntryNode = userMetadataParentNode.CreateChildElement("entry");
XmlNode userMetadataKeyNode = userMetadataMapEntryNode.CreateChildElement("key");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if possible can it be simplified into
userMetadataMapEntryNode.CreateChildElement("key").SetText(mapItem.first)
and
userMetadataMapEntryNode.CreateChildElement("value").SetText(mapItem.first)
from the codegen?
i.e. can we support chaining for the setters at a node level for all its immediate children?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One way I have dealt with this in the past is from java (using recursive function and a context) to generate the code and then just use the code in the template. This will greatly simplify the velocity template and push the computation on the Java side

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both key and value types can be other than Strings,

please refer to a generated example:

   XmlNode intEnumMapParentNode = parentNode.CreateChildElement("intEnumMap");
   for(const auto& mapItem : m_intEnumMap)
   {
     XmlNode intEnumMapMapEntryNode = intEnumMapParentNode.CreateChildElement("entry");
     XmlNode intEnumMapKeyNode = intEnumMapMapEntryNode.CreateChildElement("key");
     intEnumMapKeyNode.SetText(mapItem.first);
     XmlNode intEnumMapValueNode = intEnumMapMapEntryNode.CreateChildElement("value");
     ss << mapItem.second;
     intEnumMapValueNode.SetText(ss.str());
     ss.str("");
   }

the current approach is to create a parent element following the model, then create key/value elements following the same model.
The macro serializing the each individual element does not care about if we are within Map or List, it just knows the current item to be serialized under the current parent.
Trying to optimize the appearance of generated code will make the codegen even harder.

Move to generating the code to .java is also quite debatable.
It may appear to be easier on a first glance, but on a fresh look, the complexity looks equal. The .vm allows to match the generated output to the template, while the simple streaming of new code into the buffer using any programming language does not.

The current goal is to make SDK compliant with protocol tests.
Refactoring and optimizations of generated models will follow later.

@SergeyRyabinin SergeyRyabinin force-pushed the sr/pt/xmlSerializationCodegenRef branch from a9ff6ed to 692c95c Compare January 30, 2025 23:55
@sbera87 sbera87 self-requested a review January 31, 2025 14:48
Copy link
Contributor

@sbera87 sbera87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. left ideas for later optimizaton

@SergeyRyabinin SergeyRyabinin merged commit 7f9d72c into main Jan 31, 2025
3 checks passed
@SergeyRyabinin SergeyRyabinin deleted the sr/pt/xmlSerializationCodegenRef branch January 31, 2025 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants