Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Usage of streams causing file descriptor leak #12173

Closed
Hirevo opened this issue Apr 29, 2024 · 3 comments
Closed

Usage of streams causing file descriptor leak #12173

Hirevo opened this issue Apr 29, 2024 · 3 comments

Comments

@Hirevo
Copy link

Hirevo commented Apr 29, 2024

Describe the bug

I am attempting to make a SvelteKit API endpoint that makes a request to another backend API and streams the response body of its own response body, using fetch and response.body, like so:

// `filterHeaders` is simply a function to filter only some desired headers.

/** @type {import('./$types').RequestHandler} */
export async function GET(event) {
    const requestHeaders = filterHeaders(event.request.headers, ['range']);
    const response = await fetch('http://example.com/some/example/endpoint', {
        signal: event.request.signal,
        headers: requestHeaders,
    });
    const responseHeaders = filterHeaders(
        response.headers,
        ['content-length', 'content-type', 'content-range', 'accept-ranges'],
    );
    return new Response(response.body, {
        status: response.status,
        statusText: response.statusText,
        headers: responseHeaders,
    });
}

While this code works, it seems to leak a file descriptor each time it is requested, especially in the context of HTTP range requests while playing video content.

I have tried many variations of this code, like using pipeThrough or pipeTo, using the pipeline function from the node:stream/promises module, or using axios to make the request and converting the body back to a ReadableStream, but all of them demonstrated the issue, which prompts to me that maybe its the @sveltejs/adapter-node that doesn't close the response stream properly.

After some time, this issue leads to my app being unable to fulfil new requests, and the node process must be restarted.

Reproduction

I have made a minimal demo app to demonstrate the issue here:

https://github.com/Hirevo/sveltekit-stream-issue

The index page has steps of how to observe the issue, using lsof.

Logs

No response

System Info

System:
  OS: macOS 14.4.1
  CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
  Memory: 411.48 MB / 16.00 GB
  Shell: 0.1.0 - /Users/nicolaspolomack/.cargo/bin/psh
Binaries:
  Node: 22.0.0 - ~/.local/state/fnm_multishells/18305_1714256923846/bin/node
  Yarn: 1.22.22 - /usr/local/bin/yarn
  npm: 10.5.1 - ~/.local/state/fnm_multishells/18305_1714256923846/bin/npm
Browsers:
  Chrome: 124.0.6367.91
  Safari: 17.4.1
  Safari Technology Preview: 17.4
npmPackages:
  @sveltejs/adapter-node: ^5.0.1 => 5.0.1
  @sveltejs/kit: ^2.0.0 => 2.5.7
  @sveltejs/vite-plugin-svelte: ^3.0.0 => 3.1.0
  svelte: ^4.2.7 => 4.2.15
  vite: ^5.0.3 => 5.2.10

Severity

annoyance

Additional Information

From some of my research on this, I am aware that undici, which implements fetch in Node, does not release connection resources upon garbage collection.

I made multiple attempts to close it myself, but I was unsuccessful.

@hanszoons
Copy link
Contributor

Perhaps related to #11617?

@t1u1
Copy link

t1u1 commented Jul 25, 2024

Could be also related to #11751

When streaming, the request channel doesn't seem to get closed ever. Not sure at which layer (socket FD or some node abstraction).

@eltigerchino
Copy link
Member

I'm closing this since I'm unable to reproduce the issue with the latest SvelteKit and Vite running on Noda 20.18.0

@eltigerchino eltigerchino closed this as not planned Won't fix, can't repro, duplicate, stale Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants