Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure Phoenix server scan page timeout #24689

Merged
merged 1 commit into from
Jan 15, 2025
Merged

Conversation

virajjasani
Copy link
Member

@virajjasani virajjasani commented Jan 12, 2025

Description

Phoenix provides a way to configure Page size while creating JDBC Connection. Page size in Phoenix is important aspect to control how much work a single HBase RPC handler should perform before timing out. When the given RPC thread times out based on the page size, Phoenix client retrieves dummy result from the server and resume the operation by sending another RPC request, until eventually the result iteration is completed.

As Phoenix supports both OLTP and OLAP workloads, setting the page size timeout is crucial to manage HBase server resources efficiently. While by default, Phoenix client relies on HBase RPC timeout values to derive Page size, using Configuration property is flexible and efficient way for the clients to have better control.

Additional context and related issues

Release notes

## Phoenix
* Allow configuring scan page timeout with the `phoenix.server-scan-page-timeout` configuration property. ({issue}`24689`)

@cla-bot cla-bot bot added the cla-signed label Jan 12, 2025
@github-actions github-actions bot added the docs label Jan 12, 2025
@ebyhr
Copy link
Member

ebyhr commented Jan 13, 2025

I'm not sure if we really need this property or not. Can't we set this property via phoenix.connection-url or phoenix.config.resources?

@virajjasani virajjasani force-pushed the pagesize branch 2 times, most recently from 3fab98c to 19e2653 Compare January 14, 2025 04:14
@virajjasani virajjasani changed the title Phoenix Connector: Configure Page size while creating Connection Configure Phoenix server scan page timeout Jan 14, 2025
@virajjasani
Copy link
Member Author

I'm not sure if we really need this property or not. Can't we set this property via phoenix.connection-url or phoenix.config.resources?

Unfortunately, neither of them covers it. For instance, phoenix.config.resources can cover generic HBase RPC timeout config but that will be applicable to all Scanners. Using Connection props is the only way to ensure a given use case (specifically OLAP) can use fine tuned timeout so that other latency sensitive requests don't have to wait for too long.

@ebyhr
Copy link
Member

ebyhr commented Jan 14, 2025

@virajjasani The answer doesn't explain why phoenix.connection-url doesn't work. Most JDBC drivers allow setting properties by appending key-value in the URL. Could you explain why it doesn't work in Phoenix JDBC driver?

@virajjasani
Copy link
Member Author

virajjasani commented Jan 14, 2025

Since the early days, very limited properties have been allowed in the url:

  • AutoCommit
  • TenantId
  • CurrentSCN
  • BuildIndexAt
  • UpsertBatchSize
  • UpsertBatchSizeBytes
  • Consistency
  • RequestMetric
  • schema

These are simple property names (when compared with phoenix.server.page.size.ms and many others). I am not aware whether simplicity is the reason or whether the fact that the above listed properties are the most fundamental ones is the reason behind allowing to derive only these properties from the url whereas the remaining ones are expected to be provided with Properties object.

@virajjasani
Copy link
Member Author

virajjasani commented Jan 14, 2025

I also see that we are using this in test setup but it is of no use because the real value of namespace mapping is coming from elsewhere, using it in url has no impact:

    public String getJdbcUrl()
    {
        return format("jdbc:phoenix:localhost:%d:/hbase;phoenix.schema.isNamespaceMappingEnabled=true", port);
    }

There is different story for isNamespaceMappingEnabled though and I would like to fix (remove) it in separate PR. It is sort of not so useful property anymore.

@virajjasani virajjasani force-pushed the pagesize branch 2 times, most recently from 0aed6a9 to 95df4ba Compare January 14, 2025 06:02
@virajjasani virajjasani force-pushed the pagesize branch 2 times, most recently from c7e3bf3 to f7a85fb Compare January 14, 2025 23:47
@virajjasani virajjasani force-pushed the pagesize branch 2 times, most recently from 00320de to fc8b4da Compare January 15, 2025 01:13
@virajjasani virajjasani force-pushed the pagesize branch 2 times, most recently from 881223d to ec4953b Compare January 15, 2025 01:50
@ebyhr
Copy link
Member

ebyhr commented Jan 15, 2025

@virajjasani Please avoid doing force-push with rebase as much as possible. It's hard to understand the diff since the last review on GitHub UI.

@ebyhr ebyhr merged commit 1b1359c into trinodb:master Jan 15, 2025
19 checks passed
@github-actions github-actions bot added this to the 469 milestone Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

4 participants