Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/Updated validation to allow for missing sigma #503

Merged
merged 19 commits into from
Mar 1, 2024

Conversation

Jerry-Jinfeng-Guo
Copy link
Contributor

Updated the validation, test and documentation.

…gma` and `q_sigma` exist and are valid

updated test case
updated documentation

Signed-off-by: Jerry Guo <[email protected]>
@Jerry-Jinfeng-Guo Jerry-Jinfeng-Guo changed the title Updated validation to allow for missing sigma Feature/Updated validation to allow for missing sigma Feb 20, 2024
@Jerry-Jinfeng-Guo Jerry-Jinfeng-Guo self-assigned this Feb 20, 2024
@Jerry-Jinfeng-Guo Jerry-Jinfeng-Guo added the feature New feature or request label Feb 20, 2024
docs/user_manual/components.md Show resolved Hide resolved
docs/user_manual/components.md Outdated Show resolved Hide resolved
docs/user_manual/components.md Outdated Show resolved Hide resolved
Signed-off-by: Jerry Guo <[email protected]>
…configurations for power_sigma and p_sigma/q_sigma pair.

Signed-off-by: Jerry Guo <[email protected]>
Signed-off-by: Jerry Guo <[email protected]>
docs/user_manual/components.md Outdated Show resolved Hide resolved
src/power_grid_model/validation/validation.py Outdated Show resolved Hide resolved
tests/unit/validation/test_validation_functions.py Outdated Show resolved Hide resolved
tests/unit/validation/test_validation_functions.py Outdated Show resolved Hide resolved
docs/user_manual/components.md Show resolved Hide resolved
src/power_grid_model/validation/validation.py Outdated Show resolved Hide resolved
src/power_grid_model/validation/validation.py Outdated Show resolved Hide resolved
src/power_grid_model/validation/validation.py Outdated Show resolved Hide resolved
src/power_grid_model/validation/validation.py Outdated Show resolved Hide resolved
src/power_grid_model/validation/validation.py Outdated Show resolved Hide resolved
@Jerry-Jinfeng-Guo
Copy link
Contributor Author

I'll happily make the changes and switch to another incremental approach if the whole team agrees @mgovers @petersalemink95 @TonyXiang8787 . Else I'll stick to my approach, and perhaps simplify some of the code to reduce reading complexity.

Summary:
The issue relates to the check regarding the power_sigma inside *sym_power_sensor. Previously it lives inside required::*sym_power_sensor. Since data input can contain multitude of such sensors, I proposed the solution to tailor the required field to data. Martijn suggested that they should be checked next to the regular required check, by first removing them from required entirely and then followed by a dedicated power_sigma/p_sigma/q_sigma check.

While I agree with Martijn's comment on part of the code's reading complexity, I do not see this incremental check for every new feature as a better solution. I stand corrected. Could use some input here, thanks in advance!

@TonyXiang8787
Copy link
Member

TonyXiang8787 commented Feb 26, 2024

@mgovers @Jerry-Jinfeng-Guo My proposal is to push this with minimal necessary refactoring. Do not change the logic of how validation function works unless it is really needed.

@mgovers
Copy link
Member

mgovers commented Feb 26, 2024

@mgovers @Jerry-Jinfeng-Guo My proposal is to push this with minimal necessary refactoring. Do not change the logic of how validation function works unless it is really needed.

it feels like we're building up some serious tech debt here that may make changes harder in the future and sets a precedent. what's your rationale?

@TonyXiang8787
Copy link
Member

TonyXiang8787 commented Feb 26, 2024

@mgovers @Jerry-Jinfeng-Guo My proposal is to push this with minimal necessary refactoring. Do not change the logic of how validation function works unless it is really needed.

it feels like we're building up some serious tech debt here that may make changes harder in the future and sets a precedent. what's your rationale?

My rationale is that we might one day move the whole validation process in C++ core (but still separate optional function in C API) with semi-automatically generated code based on the pre-defined attributes constraints.

@mgovers
Copy link
Member

mgovers commented Feb 26, 2024

@mgovers @Jerry-Jinfeng-Guo My proposal is to push this with minimal necessary refactoring. Do not change the logic of how validation function works unless it is really needed.

it feels like we're building up some serious tech debt here that may make changes harder in the future and sets a precedent. what's your rationale?

My rationale is that we might one day move the whole validation process in C++ core (but still separate optional function in C API) with semi-automatically generated code based on the pre-defined attributes constraints.

while i very much like that, that don't believe that that's anywhere near prioritized by @petersalemink95 now

@petersalemink95
Copy link
Member

@mgovers @Jerry-Jinfeng-Guo My proposal is to push this with minimal necessary refactoring. Do not change the logic of how validation function works unless it is really needed.

it feels like we're building up some serious tech debt here that may make changes harder in the future and sets a precedent. what's your rationale?

My rationale is that we might one day move the whole validation process in C++ core (but still separate optional function in C API) with semi-automatically generated code based on the pre-defined attributes constraints.

while i very much like that, that don't believe that that's anywhere near prioritized by @petersalemink95 now

I think this wil happen in the long future. For now, I'd like to keep the code similar everywhere and not make new tech dept for different checks. We can do them all in a similar way as has been done before

@Jerry-Jinfeng-Guo
Copy link
Contributor Author

@mgovers @Jerry-Jinfeng-Guo My proposal is to push this with minimal necessary refactoring. Do not change the logic of how validation function works unless it is really needed.

it feels like we're building up some serious tech debt here that may make changes harder in the future and sets a precedent. what's your rationale?

My rationale is that we might one day move the whole validation process in C++ core (but still separate optional function in C API) with semi-automatically generated code based on the pre-defined attributes constraints.

while i very much like that, that don't believe that that's anywhere near prioritized by @petersalemink95 now

I think this wil happen in the long future. For now, I'd like to keep the code similar everywhere and not make new tech dept for different checks. We can do them all in a similar way as has been done before

Well, I will ask for some clearance here then: my proposed solution (A) and Martijn's opinion (B) are both trying to do just this, but from different perspective. Mine kept the original checks and only made it more tailored; Marthijn's follows existing paradigm of incrementally adding new checks whenever there are new features needed special treatment. In a way, this is the result of keeping the same goal but with different interpretations. So here comes the question, A or B? @TonyXiang8787 @petersalemink95

@TonyXiang8787
Copy link
Member

@Jerry-Jinfeng-Guo @mgovers @petersalemink95 the situation about power_sigma, p_sigma and q_sigma is really exceptional. Making some tailored code for that would not build much technical debt, I think.

So I am in favor of @Jerry-Jinfeng-Guo's current solution.

Copy link
Member

@mgovers mgovers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

love it. just some minor remarks and hopefully we can get this out the door today

src/power_grid_model/validation/validation.py Outdated Show resolved Hide resolved
src/power_grid_model/validation/validation.py Outdated Show resolved Hide resolved
Signed-off-by: Jerry Guo <[email protected]>
Copy link

sonarqubecloud bot commented Mar 1, 2024

@Jerry-Jinfeng-Guo Jerry-Jinfeng-Guo merged commit b02816f into main Mar 1, 2024
27 checks passed
@Jerry-Jinfeng-Guo Jerry-Jinfeng-Guo deleted the feature/updated-power-sigma-validation branch March 1, 2024 16:05
@mgovers mgovers mentioned this pull request Jun 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] In validation function, it still requires power sigma for power sensor
4 participants