Add TPC-DS DDL test cases #487
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Scan Summary
| Tool | Critical | High | Medium | Low | Status |
|---|---|---|---|---|---|
| Dependency Scan (java) | 0 | 2 | 3 | 0 | ✅ |
| Python Security Analysis | 3 | 0 | 0 | 0 | ❌ |
| Java Source Analyzer | 0 | 0 | 0 | 1 | ✅ |
| Python Source Analyzer | 0 | 0 | 0 | 0 | ✅ |
| Security Audit for Infrastructure | 0 | 0 | 0 | 0 | ✅ |
| Secrets Audit | 0 | 2 | 0 | 0 | ✅ |
| Scala Security Audit | 0 | 6 | 0 | 0 | ❌ |
| Shell Script Analysis | 0 | 0 | 0 | 0 | ✅ |
| Class File Analyzer | 0 | 6 | 0 | 0 | ❌ |
Recommendation
Please review the findings from Code scanning alerts before approving this pull request. You can also configure the build rules or add suppressions to customize this bot 👍
Codecov Report
@@ Coverage Diff @@
## master #487 +/- ##
==========================================
- Coverage 81.48% 80.52% -0.97%
==========================================
Files 116 116
Lines 4235 4247 +12
Branches 516 514 -2
==========================================
- Hits 3451 3420 -31
- Misses 507 546 +39
- Partials 277 281 +4
Continue to review full report at Codecov.
|
| Field("sr_reason_sk", int), | ||
| Field("sr_ticket_number", int), | ||
| Field("sr_return_quantity", int), | ||
| Field("sr_return_amt", int), |
There was a problem hiding this comment.
Spark use the long type to define these field type and some other, do we need to follow those ? Althought the standard tpcds seems does not require the type of identifier which allowed both int and long.
There was a problem hiding this comment.
FYI, apache/spark#32037. I am going to change Spark's definition. It's no reason to use long for those fields
There was a problem hiding this comment.
ah I see, let's wait that PR merged first
There was a problem hiding this comment.
the spark pr merged, can you proceed?
|
thanks merging to master for v1.2.0 |
Why are the changes needed?
Verifying TPC-DS DDLs, we should cover DML, DQL later, and maybe benchmark tests
How was this patch tested?
Add some test cases that check the changes thoroughly including negative and positive cases if possible
Add screenshots for manual tests if appropriate
Run test locally before make a pull request