Allow customization of output format

[ccremer]

Hi,
Thanks for this library, greatly appreciate the effort!

Is there a way to customize the output format of the events transmitted over the wire?
When activating the endpoint over curl, curl sees this:

$ curl -s -N -X POST  "http://localhost:5173/api/v1/sse"
id: 1
event: message
data: the%20time%20is%201734790045014

id: 2
event: message
data: the%20time%20is%201734790046015

id: 3
event: message
data: the%20time%20is%201734790047015

id: 4
event: message
data: the%20time%20is%201734790048016

It would be super helpful to have the server already formatted with different formats:

plaintext:

the%20time%20is%201734790045014
the%20time%20is%201734790046015
the%20time%20is%201734790047015

JSON (1 per line):

{"id":1,"event":"message","data":"the%20time%20is%201734790045014"}
{"id":2,"event":"message","data":"the%20time%20is%201734790046015"}
...

tsv:

id	event	data
1	message	the%20time%20is%201734790045014
2	message	the%20time%20is%201734790046015

csv:

id,event,data
1,message,the%20time%20is%201734790045014
2,message,the%20time%20is%201734790046015

(at least plaintext and JSON would be helpful to start with)

This would be easier to integrate with CLI tools to process events.
I envision a scenario where I could set the format as a query parameter, e.g. http://localhost:5173/api/v1/sse?format=json, where the server-side-code would then read the query parameter and set the format in the producer accordingly. The current format could remain as default to not break existing implementations.

[razshare]

Hello @ccremer , thank you for the issue.

Unfortunately the protocol details cannot be modified in such a way.
The spec for server sent events requires that format.

The data section is simply encoded as uri

sveltekit-sse/src/lib/produce.js

Lines 38 to 42 in e5ade90

	for (const chunk of chunks) {
	controller.enqueue(
	encoder.encode(`data: ${encodeURIComponent(chunk)}\n`),
	)
	}

The spec itself doesn't say anything about uri encoding, but without uri encoding messages would split into chunks on new lines.

It would be a pretty large overhead to then reconstruct the original message on the client side.
It would also require a more complex data structure to actually reconstruct the message, for example each chunk would need to refer to the original message somehow, hence: a more complex structure would be required.

If you want to consume these streams from a CLI you will need to implement an SSE client that follows the spec.
I use this one https://github.com/Azure/fetch-event-source for this library, which depends on fetch.

Once you've done that, you just need to decode each message as uri and you'll get the original string.
Here's an example

sveltekit-sse/src/lib/source.js

Lines 34 to 36 in e5ade90

	onMessage(e) {
	set({ id: e.id, event: e.event, data: decodeURIComponent(e.data) })
	},

Note

If your CLI is indeed also made with NodeJs, Bun, etc, I think they all support the fetch api nowadays.

Note

I just tried using fetch with Bun and I can confirm that it's supported.

---

Let me know if this answers your question.
Also, I'm always glad to take a look at actual code if you provide a reproduction or a short example repository in this case.

[ccremer]

Thanks for answer and explanations.
I completely forgot about the protocol and that's a pity. I ended up with the following code for node v22, without an additional library, which is very basic but good enough for me (in case anyone has the same problem).

async function createSseClient(url: string, apiKey: string) {
  try {
    const response = await fetch(url, {
      headers: {
        authorization: `Bearer ${apiKey}`,
      },
      method: 'POST',
    });

    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }

    const reader = response?.body?.getReader();

    while (reader && true) {
      const { value, done } = await reader.read();

      if (done) {
        break;
      }

      const decoder = new TextDecoder();
      const text = decoder.decode(value);

      const lines = text.split('\n');

      for (const line of lines) {
        if (line.trim() === '') {
          continue; // Skip empty lines
        }

        const [key, ...valueParts] = line.split(':');

        if (key === 'data') {
          const data = valueParts.join(':').trim();
          console.log(data);
        }
      }
    }
  } catch (error) {
    console.error('Error connecting to SSE stream:', error);
  }
}

const API_ACCESS_KEY = process.env.API_ACCESS_KEY;
if (!API_ACCESS_KEY) {
  throw new Error('API_ACCESS_KEY environment variable is required.');
}

const url = process.argv.slice(2)[0];
if (!url) {
  throw new Error('URL of migration endpoint is required as first argument.');
}

createSseClient(url, API_ACCESS_KEY);

This code just prints the lines to console.

Anyway, thanks for the help.