Connecting to the Polaris S3 storage bucket

[srdjanma]

Hi,

I’m trying to connect to the Polaris S3 storage bucket but seem to be missing some critical details:

The actual S3 bucket name and its region.
AWS Access Key and Secret Key, or any other authentication details I should use.
Specific instructions, if the service uses non-standard S3-compatible storage.
Could you help me set this up? Let me know if additional steps are required on my side.

Thanks,
Srdan

[cwognum]

@srdjanma Thanks for reaching out! We don't provide direct access to the bucket. Instead, we use JWTs that you can request by going through the Hub's authentication layer, e.g. using the Polaris client. This is done to implement access control for the artifacts on the Hub (e.g. for the competition, you shouldn't be able to access the test set yet).

Connection through the main Python API is effortless and in most cases you shouldn't even have to worry about the details of our storage layer.

For example, the following will automatically fetch the JWT and use it to access the needed data from the Hub.

import polaris as po

po.load_competition("asap-discovery/antiviral-admet-2025")

I hope that helps!

[srdjanma]

Thanks.. I think I am getting too many various errors with the initialization of Polaris on my system, just to even open the database.. At this point I am not sure what to do, as it has been taking too long for me to try to sort it out, I am using Macbook and VS code if it matters at all (I guess not). Is there any other way to go around this, like downloading a raw data from somewhere? Thanks

[cwognum]

I'm sorry to hear that. The use of Polaris should be simple and fast. I would love to hear more about the issues you're running into. To help you out, but also so we can further improve the robustness. Would you mind filing a bug report in this repo? I'll make sure someone helps you out as soon as possible.

To answer your question: We do provide the raw data packages. The file is stored here: https://fs.polarishub.io/2025-01-asap-discovery/raw_data_package.zip. See also https://github.com/asapdiscovery/asap-polaris-blind-challenge-examples.

Now having said that, you will still need to use Polaris for getting the test set inputs and to eventually submit your results.

[srdjanma]

Hey, I have just managed to solve the issue. Turns out that the AWS issue I was having was caused by an invalid region configuration, that I was trying to set up in configure.py file directly. So later on, setting the region to a valid value like apac using the AWS_DEFAULT_REGION and AWS_REGION environment variables resolved the issue. Sorry for the noise I am not too experienced setting up these kind of environments :D Thanks a lot for the feedback!! I am looking forward making a good use of the database :D

[cwognum]

No worries at all.

That's actually valuable feedback. boto3 is a popular tool and people may have (unknowingly) configured their environment for an earlier project which clashes with Polaris. Especially since we abstract away the details of our storage layer, I can see that be confusing.

We should explicitly configure boto3 in our code (i.e. specify the region, at least) so that our config takes precedence within the context of Polaris.

Pinging @jstlaurent for his thoughts.

[jstlaurent]

@srdjanma : Sorry to hear you had issues in setting up Polaris. As @cwognum said, we aim to make the experience as easy and simple as it could be. We can do better here.

Your issue is the second time this week that using boto3 in Polaris has caused problems. I'm working on an alternative (in this branch) that should solve this, by using an alternative library. There's a few things I need to validate before it's ready for release, but it should solve future issues like the one you experienced.

[cwognum]

Happy the issue got resolved! If you run into any other problems, don't hesitate to reach out!

[srdjanma]

Cheers guys, you are very helpful and nice :) Thanks!