Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka seekTImestamp should also support timestamp in 2024-01-29T19:35:21.959 format #621

Closed
authorjapps opened this issue Jan 29, 2024 · 6 comments · Fixed by #619
Closed
Assignees

Comments

@authorjapps
Copy link
Owner

Related to Issue #615

// 1) Currently we supply the timestamp this way i.e. either hardcoding or getting it from earlier steps/props 
// (implemented in #615 )
"seekTImestamp": "1706244382000". 

The below TODO after #615 is done

Support The Following also:

  • As a user of the library e.g. as a Dev/Tester, how will the user supply/generate the timestamp value during the test run?

AC1:
So, provide the mechanism to get the Long epoch value inferred from the datetimestamps formated values:
e.g.
The following also should be acceptable to the framework:

// 2) hardcoded or getting it from earlier steps/props
"seekTImestamp": "2024-01-29T19:35:21.959" 

// 3) Generating via a placeholder token (this mechanism is already available)
"seekTImestamp": "${LOCAL.DATETIME.NOW:yyyy-MM-dd'T'HH:mm:ss.SSS}" 

So we need to support all of the above usages.

Actually points 2 and 3 will work identical way once implemented.

@a1shadows
Copy link
Collaborator

a1shadows commented Jan 31, 2024

@authorjapps In addition to this, do you think out of the box, doing something like:
"seekTImestamp": "${$.prior_step.resonse.generated_timestamp}" will work?

Then we can generate the timestamp to consume from on a previous step?

@authorjapps
Copy link
Owner Author

@authorjapps In addition to this, do you think out of the box, doing something like: "seekTImestamp": "${$.prior_step.resonse.generated_timestamp}" will work?

Then we can generate the timestamp to consume from on a previous step?

Hello @a1shadows , I suggest you please pick this after your other PR is done or at least after it comes to a logical end.
Also I would advice not to distracted by this new ticket ✋


Ok, now to answer your query, let me try to explain:
"${$.prior_step.resonse.generated_timestamp}" => This works anyways in the existing framework. That's not the problem here

The problem is, your current PR implementation may not work(or may work once you complete the full implementation) for timestamps other than ECPOCH time in long/integer format.

This ticket #621 was raised to support the below format as well, so that we allow EPOCH as well as the standard format i.e.
"seekTImestamp": "2024-01-29T19:35:21.959"

If your current PR will allow both formats, I think we can simply close this ticket #621 without further spedning time on this.

@a1shadows
Copy link
Collaborator

Related to Issue #615

// 1) Currently we supply the timestamp this way i.e. either hardcoding or getting it from earlier steps/props 
// (implemented in #615 )
"seekTImestamp": "1706244382000". 

The below TODO after #615 is done

Support The Following also:

* As a user of the library e.g. as a Dev/Tester, how will the user supply/generate the timestamp value during the test run?

AC1: So, provide the mechanism to get the Long epoch value inferred from the datetimestamps formated values: e.g. The following also should be acceptable to the framework:

// 2) hardcoded or getting it from earlier steps/props
"seekTImestamp": "2024-01-29T19:35:21.959" 

// 3) Generating via a placeholder token (this mechanism is already available)
"seekTImestamp": "${LOCAL.DATETIME.NOW:yyyy-MM-dd'T'HH:mm:ss.SSS}" 

So we need to support all of the above usages.

Actually points 2 and 3 will work identical way once implemented.

@authorjapps I can definitely incorporate this into my PR, but I have a concern here. If we want to support formatted strings, for instance 'yyyy-MM-dd'T'HH:mm:ss.SSS' should we not be taking the pattern as input as well along with the actual timestamp? Or will we only support one pattern?

Or would it be better to make LOCAL.DATETIME.NOW support epoch format as well?

@authorjapps
Copy link
Owner Author

Related to Issue #615

// 1) Currently we supply the timestamp this way i.e. either hardcoding or getting it from earlier steps/props 
// (implemented in #615 )
"seekTImestamp": "1706244382000". 

The below TODO after #615 is done
Support The Following also:

* As a user of the library e.g. as a Dev/Tester, how will the user supply/generate the timestamp value during the test run?

AC1: So, provide the mechanism to get the Long epoch value inferred from the datetimestamps formated values: e.g. The following also should be acceptable to the framework:

// 2) hardcoded or getting it from earlier steps/props
"seekTImestamp": "2024-01-29T19:35:21.959" 

// 3) Generating via a placeholder token (this mechanism is already available)
"seekTImestamp": "${LOCAL.DATETIME.NOW:yyyy-MM-dd'T'HH:mm:ss.SSS}" 

So we need to support all of the above usages.
Actually points 2 and 3 will work identical way once implemented.

@authorjapps I can definitely incorporate this into my PR, but I have a concern here. If we want to support formatted strings, for instance 'yyyy-MM-dd'T'HH:mm:ss.SSS' should we not be taking the pattern as input as well along with the actual timestamp? Or will we only support one pattern?

Or would it be better to make LOCAL.DATETIME.NOW support epoch format as well?

@a1shadows, Good question mate 👍

I think it would be better the below way, if you can accommodate in your PR please.
This will make things clean in terms of usability as well as implementation.

e.g.

  • For Epoch:
"seekEpoch": "<Epoch long number>"  <---- Keep this as String type please (not Long/Int)
  • and
    for Timestamp:
"seekTImestamp": { 
   "dateTimestamp" : "2024-01-29T19:35:21.959",
   "format" : "yyyy-MM-dd'T'HH:mm:ss.SSS"
}

"seekTImestamp": { 
   "dateTimestamp" : "2024-01-29T19:35:21.959340",
   "format" : "yyyy-MM-dd'T'HH:mm:ss.ssssss"
}

// This was user will have more flexibility to supply the values.

I suggest you implement this "seekTImestamp" in a separate PR, but I will leave to you to decide :-)

@a1shadows
Copy link
Collaborator

@authorjapps I think we can close this as well

@authorjapps
Copy link
Owner Author

Closed by #615

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment