Update the CLI install script release source

[mikeee]

Expected Behavior

Release versions should be retrieved successfully.

Actual Behavior

Users may hit the 60 requests per hour limit (per IP) when calling the github api unauthenticated.

Steps to Reproduce the Problem

Call any github api endpoint 60 times, run the install script which fails to find a release version for the CLI and the invalid version is not handled.

Proposed Solution

A "version" branch is set up and maintained by a workflow which is triggered by dispatch (primarily intended for the release job to trigger this workflow on completion).
The version branch will contain a docs folder with a single file containing the latest version tag for the CLI and will be served by gh-pages or an azure static site (with preference for gh-pages).
The install scripts will no longer hit the API for the latest tag.

Release Note

RELEASE NOTE: FIX Dapr CLI install/update release version retrieval

[mikeee]

@dapr/maintainers-cli - thoughts on the proposed solution?

[dapr-bot]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

[mikeee]

/keep-alive

[dapr-bot]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

[dapr-bot]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

[antontroshin]

Hi @mikeee
Gathered some general information about Github limits, my test experience and other observation notes.

Unauthenticated user requests: 60 per hour. source
Authenticated user requests: 5000 per hour. source
GitHub Pages sites have a soft bandwidth limit of 100 GB per month source

Running Dapr install script using curl or wget or (similar per platform) which are counted as unauthenticated requests.
Hitting 60 per hour limit while not something that happens very often, but it is possible.
When reaching the limit, Github often resets the limit after 1 minute of waiting, but it is not consistent. I've had a scenario when even after 30 minutes, the calls were still rate-limited.
If the requests continue to reach the limit, Github exponentially increases the "reset" time.

There are a few methods of checking if calls are rate-limited. source

1. Use API GET /rate_limit
2. Use specific response headers to determine the rate limit status.

While the Github pages or external hosting of the latest version in the file circumvent the unauthenticated API limits, I do have a concern about it.
The main concern is the traffic limit, when if the limit is reached for that month, then the script won't work for the rest of the month for ALL users. While the rate limit is only per user/IP and for a short amount of time.
Second is the availability of the hosting. If it's Github pages and something is wrong with Github, most likely (but not necessarily all together), Github Pages, API, and Assets won't be available anyway.
But in the case of third-party hosting, like Azure (etc.) the availability of that hosting will impact all users, while in fact Github is available and doesn't have issues.

In the following document, Github suggests making authenticated requests, exactly for the reason to avoid exceeding the rate limit.

We can do a mixed approach to solve this issue or start by adding multiple solutions as we encounter related issues.

My suggestion would be the latter.

1. Add support for optional authentication header, i.e. GITHUB_TOKEN, and use it in requests when available.
2. Add support for checking rate limit, using API and/or response headers, and retry every 60 seconds until the "remaining" is reset and larger than 0.
3. Add Github Action to publish the gh-pages and file with the latest release version. Use it as a backup resource to obtain the latest version when Github rate limit is encountered.

Appreciate any thoughts, considerations, or details I might have missed.