Skip to content

Commit

Permalink
reordering troubleshooting
Browse files Browse the repository at this point in the history
  • Loading branch information
andrewjschuang committed Feb 27, 2025
1 parent 79be8f3 commit 29ccf71
Show file tree
Hide file tree
Showing 3 changed files with 80 additions and 95 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,18 @@ This doc describes some common solutions for fixing issues with [pipedream.com](

Start by filling out the request form at [https://pipedream.com/support](https://pipedream.com/support), providing detailed information about your issue.

### How do I share my workflow with Support?

First, navigate to your **Project Settings** and share your project with Pipedream Support.

If your workflow is _not_ part of a Project, go to the **Workflow Settings** to grant access to us.

When filling out the request form at [https://pipedream.com/support](https://pipedream.com/support), please provide detailed information along with the URL from your browser's address bar, which should look something like:

```
https://pipedream.com/@yourworkspace/workflows/projects/yourproject/test-workflow-pabc123
```

## A feature isn't working on pipedream.com

If you're seeing an issue with [pipedream.com](https://pipedream.com) (for example, the site won't load, or you think you've found a bug), try each of the following steps, checking to see if they fix the problem:
Expand All @@ -20,6 +32,10 @@ If you're seeing an issue with [pipedream.com](https://pipedream.com) (for examp

If you're still seeing the issue after trying these steps, please [report a bug](https://github.com/PipedreamHQ/pipedream/issues/new?assignees=&labels=bug&template=bug_report.md&title=%5BBUG%5D+).

## I need help with my GitHub Synced Project

[See FAQ here](/workflows/git/#faq).

## My workflow isn't working

If you're encountering a specific issue in a workflow, try the following steps, checking to see if they fix the problem:
Expand All @@ -30,9 +46,11 @@ If you're encountering a specific issue in a workflow, try the following steps,

If you're still seeing the issue after trying these steps, please reach out in [the community](https://pipedream.com/support).

## I need help with my GitHub Synced Project
## How do I invoke another workflow?

[See FAQ here](/workflows/git/#faq).
We provide a [Trigger Workflow](https://pipedream.com/apps/helper-functions/actions/trigger-workflow) action in the [Helper Functions](https://pipedream.com/apps/helper-functions) app. [See more here](/workflows/building-workflows/code/nodejs/#invoke-another-workflow).

Another option is to make an HTTP request to a Pipedream HTTP webhook trigger.

## Why is my trigger not emitting events?

Expand All @@ -56,22 +74,6 @@ These sources will fetch new events on a regular interval, based on a schedule y

In most cases, Pipedream will automatically fetch recent historical events to help enable easier workflow development. Sources for apps like [Twitter](https://pipedream.com/apps/twitter/triggers/search-mentions) and [Spotify](https://pipedream.com/apps/spotify/triggers/new-playlist) require we poll their endpoints in order to fetch new events.

## Where do I find my workflow's ID?

Open [https://pipedream.com](https://pipedream.com) and visit your workflow. Copy the URL that appears in your browser's address bar. For example:

```
https://pipedream.com/@dylburger/p_abc123/edit
```

Your workflow's ID is the value that starts with `p_`. In this example: `p_abc123`.

## How do I invoke another workflow?

We provide a [Trigger Workflow](https://pipedream.com/apps/helper-functions/actions/trigger-workflow) action in the [Helper Functions](https://pipedream.com/apps/helper-functions) app. [See more here](/workflows/building-workflows/code/nodejs/#invoke-another-workflow).

Another option is to make an HTTP request to a Pipedream HTTP webhook trigger.

## Where do I find my event source's ID?

Open [https://pipedream.com/sources](https://pipedream.com/sources) and click on your event source. Copy the URL that appears in your browser's address bar. For example:
Expand All @@ -92,22 +94,37 @@ To troubleshoot, you can look at the errors in the [source](/workflows/building-

Pipedream displays warnings below steps in certain conditions. These warnings do not stop the execution of your workflow, but can signal an issue you should be aware of.

## Limit Exceeded Errors
### Code was still running when the step ended

Pipedream sets [limits](/workflows/limits/) on runtime, memory, and other execution-related properties. If you exceed these limits, you'll receive one of the errors below. [See the limits doc](/workflows/limits/) for details on specific limits.
This error occurs when Promises or asynchronous code is not properly finished before the next step begins execution.

See the [Asynchronous section of the Node.js documentation](/workflows/building-workflows/code/nodejs/async/#the-problem) for more details.

### Undeployed changes — You have made changes to this workflow. Deploy the latest version from the editor

On workflows that are not [synced with GitHub](/workflows/git/), you may notice the following warning at the top of your workflow:

> **Undeployed changes** — You have made changes to this workflow. Deploy the latest version from the editor
This means that you've made some changes to your workflow that you haven't yet deployed. To see a diff of what's changed, we recommend [enabling GitHub sync](/workflows/git/), where you'll get a full commit history of changes made to your workflows, synced to your own GitHub repo.

## Errors

## Quota Exceeded
### Limit Exceeded Errors

Pipedream sets [limits](/workflows/limits/) on runtime, memory, and other execution-related properties. If you exceed these limits, you'll receive one of the errors below. [See the limits doc](/workflows/limits/) for details on specific limits.

### Quota Exceeded

On the Free tier, Pipedream imposes a limit on the [daily credits](/workflows/limits/#daily-credits-limit) across all workflows and sources. If you hit this limit, you'll see a **Quota Exceeded** error.

Paid plans have no credit limit. [Upgrade here](https://pipedream.com/pricing).

## Runtime Quota Exceeded
### Runtime Quota Exceeded

You **do not** use credits testing workflows, but workspaces on the **Free** plan are limited to {process.env.DAILY_TESTING_LIMIT} of test runtime per day. If you exceed this limit when testing in the builder, you'll see a **Runtime Quota Exceeded** error.

## Timeout
### Timeout

Event sources and workflows have a [default time limit on a given execution](/workflows/limits/#time-per-execution). If your code exceeds that limit, you may encounter a **Timeout** error.

Expand All @@ -116,7 +133,7 @@ To address timeouts, you'll either need to:
1. Figure out why your code is running for longer than expected. It's important to note that **timeouts are not an issue with Pipedream — they are specific to your workflow**. Often, you're making a request to a third party API that doesn't respond in the time you expect, or you're processing a large amount of data in your workflow, and it doesn't complete before you hit the execution limit.
2. If it's expected that your code is taking a long time to run, you can raise the execution limit of a workflow in your [workflow's settings](/workflows/building-workflows/settings/#execution-timeout-limit). If you need to change the execution limit for an event source, please [reach out to our team](https://pipedream.com/support/).

## Out of Memory
### Out of Memory

Pipedream [limits the default memory](/workflows/limits/#memory) available to workflows and event sources. If you exceed this memory, you'll see an **Out of Memory** error. **You can raise the memory of your workflow [in your workflow's Settings](/workflows/building-workflows/settings/#memory)**.

Expand All @@ -125,13 +142,13 @@ This can happen for two main reasons:
1. When you load a large file or object into the workflow's memory (e.g. when you save the content in a variable). Where possible, consider streaming the file to / from disk, instead of storing it in memory, using a [technique like this](/workflows/building-workflows/code/nodejs/http-requests/#download-a-file-to-the-tmp-directory).
2. When you have many steps in your Pipedream workflow. When your workflow runs, Pipedream runs a separate process for each step in your workflow. That incurs some memory overhead. Typically this happens when you have more than 8-10 steps. When you see an OOM error on a workflow with many steps, try increasing the memory.

## Rate Limit Exceeded
### Rate Limit Exceeded

Pipedream limits the number of events that can be processed by a given interface (e.g. HTTP endpoints) during a given interval. This limit is most commonly reached for HTTP interfaces - see the [QPS limits documentation](/workflows/limits/#qps-queries-per-second) for more information on that limit.

**This limit can be raised for HTTP endpoints**. [Reach out to our team](https://pipedream.com/support/) to request an increase.

## Request Entity Too Large
### Request Entity Too Large

By default, Pipedream limits the size of incoming HTTP payloads. If you exceed this limit, you'll see a **Request Entity Too Large** error.

Expand All @@ -140,35 +157,35 @@ Pipedream supports two different ways to bypass this limit. Both of these interf
- You can send large HTTP payloads by passing the `pipedream_upload_body=1` query string or an `x-pd-upload-body: 1` HTTP header in your HTTP request. [Read more here](/workflows/building-workflows/triggers/#sending-large-payloads).
- You can upload multiple large files, like images and videos, using the [large file upload interface](/workflows/building-workflows/triggers/#large-file-support).

## Function Payload Limit Exceeded
### Function Payload Limit Exceeded

The total size of `console.log()` statements, [step exports](/workflows/#step-exports), and the original event data sent to workflows and sources cannot exceed a combined size of {process.env.FUNCTION_PAYLOAD_LIMIT}. If you produce logs or step exports larger than this - for example, passing around large API responses, CSVs, or other data - you may encounter a **Function Payload Limit Exceeded** in your workflow.

Often, this occurs when you pass large data between steps using [step exports](/workflows/#step-exports). You can avoid this error by [writing that data to the `/tmp` directory](/workflows/building-workflows/code/nodejs/working-with-files/#writing-a-file-to-tmp) in one step, and [reading the data into another step](/workflows/building-workflows/code/nodejs/working-with-files/#reading-a-file-from-tmp), which avoids the use of step exports and should keep you under the payload limit.

Pipedream also compresses the function payload from your workflow, which can yield roughly a 2x-3x increase in payload size (somewhere between `12MB` and `18MB`), depending on the data.

## JSON Nested Property Limit Exceeded
### JSON Nested Property Limit Exceeded

Working with nested JavaScript objects that have more than 256 nested objects will trigger a **JSON Nested Property Limit Exceeded** error.

Often, objects with this many nested objects result from a programming error that explodes the object in an unexpected way. Please confirm the code you're using to convert data into an object is correctly parsing the object.

## Event Queue Full
### Event Queue Full

Workflows have a maximum event queue size when using concurrency and throttling controls. If the number of unprocessed events exceeds the [maximum queue size](/workflows/building-workflows/settings/concurrency-and-throttling/#increasing-the-queue-size-for-a-workflow), you may encounter an **Event Queue Full** error.

[Paid plans](https://pipedream.com/pricing) can [increase their queue size up to {process.env.MAX_WORKFLOW_QUEUE_SIZE}](/workflows/building-workflows/settings/concurrency-and-throttling/#increasing-the-queue-size-for-a-workflow) for a given workflow.

## Credit Budget Exceeded
### Credit Budget Exceeded

Credit Budgets are configurable limits on your credit usage at the account or workspace level.

If you're receiving this warning on a source or workflow, this means your allocated Credit Budget has been reached for the defined period.

You can increase this limit at any time in the [billing area of your settings](https://pipedream.com/settings/billing).

## Pipedream Internal Error
### Pipedream Internal Error

A `Pipedream Internal Error` is thrown whenever there's an exception during the building or executing of a workflow that's outside the scope of the code for the individual components (steps or actions).

Expand Down Expand Up @@ -220,20 +237,42 @@ Instead, use the specific package that exports the `pick` module alone:
import pick from "lodash.pick"
```

## Code was still running when the step ended
## Is there a way to replay workflow events programmatically?

This error occurs when Promises or asynchronous code is not properly finished before the next step begins execution.
Not via the API, but you can bulk select and replay failed events using the [Event History](/workflows/event-history/).

See the [Asynchronous section of the Node.js documentation](/workflows/building-workflows/code/nodejs/async/#the-problem) for more details.
## How do I store and retrieve data across workflow executions?

### How do I share my workflow with Support?
If you operate your own database or data store, you can connect to it directly in Pipedream.

First, navigate to your **Project Settings** and share your project with Pipedream Support.
Pipedream also operates a [built-in key-value store](/workflows/data-management/data-stores/) that you can use to get and set data across workflow executions and different workflows.

If your workflow is _not_ part of a Project, go to the **Workflow Settings** to grant access to us.
## How do I delay the execution of a workflow?

When filling out the request form at [https://pipedream.com/support](https://pipedream.com/support), please provide detailed information along with the URL from your browser's address bar, which should look something like:
Use Pipedream's [built-in Delay actions](/workflows/building-workflows/control-flow/delay/) to delay a workflow at any step.

```
https://pipedream.com/@yourworkspace/workflows/projects/yourproject/test-workflow-pabc123
```
## How can my workflow run faster?

Here are a few things that can help your workflow execute faster:

1. **Increase memory:** Increase your [workflow memory](/workflows/building-workflows/settings/#memory) to at least 512 MB. Raising the memory limit will proportionally increase CPU resources, leading to improved performance and reduced latency.

2. **Return static HTTP responses:** If your workflow is triggered by an HTTP source, return a [static HTTP response](/workflows/building-workflows/triggers/#http-responses) directly from the trigger configuration. This ensures the HTTP response is sent to the caller immediately, before the rest of the workflow steps are executed.

3. **Simplify your workflow:** Reduce the number of [steps](/workflows/#code-actions) and [segments](/workflows/building-workflows/control-flow/#workflow-segments) in your workflow, combining multiple steps into one, if possible. This lowers the overhead involved in managing step execution and exports.

4. **Activate warm workers:** Use [warm workers](/workflows/building-workflows/settings/#eliminate-cold-starts) to reduce the startup time of workflows. Set [as many warm workers](/workflows/building-workflows/settings/#how-many-workers-should-i-configure) as you want for high-volume traffic.

## How can I save common functions as steps?

You can create your own custom triggers and actions ("components") on Pipedream using [the Component API](/workflows/contributing/components/api/). These components are private to your account and can be used in any workflow.

You can also publish common functions in your own package on a public registry like [npm](https://www.npmjs.com/) or [PyPI](https://pypi.org/).

## Is Puppeteer supported in Pipedream?

Yes, see [our Puppeteer docs](/workflows/building-workflows/code/nodejs/browser-automation/#puppeteer) for more detail.

## Is Playwright supported in Pipedream?

Yes, see [our Playwright docs](/workflows/building-workflows/code/nodejs/browser-automation/#playwright) for more detail.
4 changes: 0 additions & 4 deletions docs-v2/pages/troubleshooting/_meta.tsx

This file was deleted.

50 changes: 0 additions & 50 deletions docs-v2/pages/troubleshooting/faq.mdx

This file was deleted.

0 comments on commit 29ccf71

Please sign in to comment.