[Slack] A rate-limit has been reached

[obaidch24]

Extension

https://raycast.com/mommertf/slack

Raycast Version

1.81.2

macOS Version

14.6.1

Description

Error:

Error: A rate-limit has been reached, you may retry this request in 30 seconds
    at n5 (/Users/obaid.samadian/.config/raycast/extensions/d5d27df2-9818-41ea-b683-0e4af3f4be3d/search.js:43:25526)
    at e.<anonymous> (/Users/obaid.samadian/.config/raycast/extensions/d5d27df2-9818-41ea-b683-0e4af3f4be3d/search.js:45:10270)
    at Generator.next (<anonymous>)
    at p (/Users/obaid.samadian/.config/raycast/extensions/d5d27df2-9818-41ea-b683-0e4af3f4be3d/search.js:45:2332)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Steps To Reproduce

1. Open Channel

Current Behaviour

No response

Expected Behaviour

No response

[raycastbot]

Thank you for opening this issue!

🔔 @momme-rtf @Elliot67 @jfkisafk @thomaslombart @RobErskine you might want to have a look.

💡 Author and Contributors commands

The author and contributors of mommertf/slack can trigger bot actions by commenting:

• @raycastbot close this issue Closes the issue.
• @raycastbot close as not planned Closes the issue as not planned.
• @raycastbot rename this issue to "Awesome new title" Renames the issue.
• @raycastbot reopen this issue Reopens the issue.
• @raycastbot assign me Assigns yourself to the issue.
• @raycastbot good first issue Adds the "Good first issue" label to the issue.
• @raycastbot keep this issue open Make sure the issue won't go stale and will be kept open by the bot.

[thomaslombart]

Are you getting this consistently, or does the error go away after a few seconds or minutes? If the latter happens, it's normal behavior.

[obaidch24]

Are you getting this consistently, or does the error go away after a few seconds or minutes? If the latter happens, it's normal behavior.

Always

[thomaslombart]

Could you share a recording? It'll make it easier to see what's happening.

[obaidch24]

Could you share a recording? It'll make it easier to see what's happening.

Sure.

Screen.Recording.2024-08-21.at.10.53.21.mov

[obaidch24]

@thomaslombart When I try to select Unread messages or Open Channel for the first time, it shows loading for more than 1 minute and then throws a rate-limit error. After that, it immediately throws the rate-limit error.

[Elliot67]

@obaidch24 Are you in a very large Slack workspace ? How many users and public channels are in it ?

---

Rate limiting might be the alternative ending to #13067 .
users.list and conversations.list are both rate limited "Tier 2" (20 requests per minute).

We can retrieve 1_000 users per requests and 1_000 conversations which means rate limiting could start to appear on workspace with more than ~15k users.

Simplest solution is to fetch and cache all the data bit by bit, preferably in the background if raycast allows it.

[obaidch24]

@obaidch24 Are you in a very large Slack workspace ? How many users and public channels are in it ?

Rate limiting might be the alternative ending to #13067 . users.list and conversations.list are both rate limited "Tier 2" (20 requests per minute).

We can retrieve 1_000 users per requests and 1_000 conversations which means rate limiting could start to appear on workspace with more than ~15k users.

Simplest solution is to fetch and cache all the data bit by bit, preferably in the background if raycast allows it.

Yes, the workspace is quite large. I'm not sure how many users and public channels it has. I guess it should have more than 4K active users.

[thomaslombart]

It could indeed be due to the large workspace. @obaidch24, if you have other Slack workspaces, could you test the extension in a smaller one and let me know if it works?

[obaidch24]

It could indeed be due to the large workspace. @obaidch24, if you have other Slack workspaces, could you test the extension in a smaller one and let me know if it works?

Unfortunately, I don't have another one.

[raycastbot]

This issue has been automatically marked as stale because it did not have any recent activity.

It will be closed if no further activity occurs in the next 10 days to keep our backlog clean 😊

[obaidsamadian]

@Elliot67 @thomaslombart Will this issue be addressed?

[thomaslombart]

I'm fully-focused on another project, so I won't have much bandwidth for that. Please feel free to submit a PR, and I'll review it 🙂

[jhasubhash]

Is there a way to just get and build the slack extension locally and submit the changes?
Any documentation on how to do dev setup?

I have a sample slack app, where I have this code to handle the same issue.
#15402 (comment)

[thomaslombart]

Here's our guide about contributing to extensions: Contribute to an extension

[jhasubhash]

I debugged it a bit.
I am able to fetch the userList with over 3k entries without hitting Rate limit.
But while fetching conversationList, I hit rate limit very early, maybe the entries per request is very low for conversation api.
I tried following code and it made the request wait for 30 seconds after retry limit is hit. And it started working.

- slackWebClient = new WebClient(token, { rejectRateLimitedCalls: true });
+ slackWebClient = new WebClient(token, { retryConfig: retryPolicies.fiveRetriesInFiveMinutes });

The problem is I have over 18k channels/conversations. And the method getConversations is being called 6 times. So, it is kind of fetching 18k*6 entries and the max number of entries returned in a single request is 500. So it lead to a lot of request being made. With code change I am able to get upto 9k entries (it took more than 2 minutes) and then I ran into following error

Error: Worker terminated due to reaching memory limit: JS heap out of memory

We can do following re-factoring to fix these issue

• Add the above code to enable retry
• Segregate getSlackMembers and getConversations calls, so that even if on succeeds we can show some list to user
• Load the list incrementally instead of blocking the user until the complete list is loaded.
• Fix code so that multiple requests to getConversations can be avoided (by enabling caching)

[jhasubhash]

I have made the change and it should work in most of the cases.
But I am unable to raise a PR for review.
While executing

npx @raycast/api@latest publish

I am running into following

You reached the limit of commands of the free plan for your organization. Upgrade your plan here:
https://raycast.com/upgrade/subhashjha/gNoWf5Jm1xmpXrez

@thomaslombart Do I need to upgrade my account or is there a workaround ?

[deinspanjer]

I'm just starting out with Raycast and got my org to approve this extension in Slack, but I'm hitting the exact same problems described here.

Are there any further developments to allow users in large Slack organizations to use the extension?

[thomaslombart]

@jhasubhash I'll take a look at your error and get back to you on that 🙂

[grzegorzkrukowski]

@jhasubhash are you trying to our public store or some private store?

The free plan allows you to have up to 5 commands in a private store - after that, you need to upgrade your organization to a paid plan, you can read more about it on our pricing page: https://www.raycast.com/pricing

However, if you see this message while trying to participate in our public store, it means there's a bug in our publishing code that we need to check.