sixel rendering issues in wezterm and tmux

[Nemo157]

I'm really not sure where the issue is, I spent a while trying to come up with something more useful to open an issue about, but I couldn't track down anything really useful.

When using sixel preview rendering it fails to display when running in wezterm directly, or when running in tmux within both wezterm and foot. In wezterm, with or without tmux, it mostly seems to render previous lines leftover from scrolling where the image should be. In tmux+foot it mostly seems to be blank, but e.g. when scrolling messages there's a single frame flicker of the image.

It does seem to work correctly when running directly in foot.

I also use sixel rendering in vifm which generally works in all the above situations, and running the ratatui-image example seems to work too. Just in case it was related to the StatefulProtocol which it says is more reliable I ported iamb over to use that, it didn't seem to make any difference.

[reynoldsme]

@Nemo157 did you try directly setting the image preview protocol in your iamb config, just in case the ratatui-image examples and iamb are using different image protocol detection logic?

For me running an iamb debug build of latest main commit and WezTerm-20240203-110809-5046fc22-Ubuntu20.04.AppImage, images show up as expected, but a copy of the first line of some images will sometimes appear and disappear at column 0 of the line on which the image occurs. If this is inconsistent between platforms or even just different machines, it makes me suspect the issue may be on the Wezterm side.

The ratatui-image readme does imply there are known bugs when using ratatui-image in sixel mode in Wezterm https://github.com/benjajaja/ratatui-image?tab=readme-ov-file#compatibility-matrix , but I don't know if this specific issue is something that is already being tracked on not. There is an open (but locked) issue for sixel on the Wezterm side, and I see the ratatui-image author has commented on that issue in the past. wez/wezterm#217

[Nemo157]

Yes, I've always been hardcoding the protocol in the config. I think I have noticed that specific bug too, but less often than issues with the rendering in the frame the image is meant to be. I wouldn't be surprised if this is all bugs in the TEs, I'm just a bit unsure since I didn't notice the same issues with other sixel apps.

One thought I just had; in tmux I do notice that rendering sixel images can be very slow, maybe the issue there is that iamb is re-rendering too often and tmux never actually completes showing the image before it gets re-rendered.

While trying to find out whether I could use the other protocols through tmux, I ran across kovidgoyal/kitty#5664, that doesn't seem to be supported by iamb or wezterm, but maybe would be simpler than trying to fix/workaround the myriad of sixel bugs that seem to exist.

[reynoldsme]

I just tried iamb in sixel mode with the latest release of [rio](https://github.com/raphamorim/rio/releases/tag/v0.1.12] and at least on my machine (x86_64 Linux under x11), sixel images seem to work perfectly both inside and out of tmux! May be worth a try if you're willing to switch terminals.

edit: I've still not found any issues using rio with iamb + sixel, but in tmux I'm seeing this same issue with the image being visible for a fraction of a second or not at all between redraws.

Warning: this is conjecture, I know just enough about this stuff to know what I don't know (I think).

I think tmux always "renders" each intermediate state to the output stream since it doesn't have a way to know if your "real" terminal doing the actual rendering is keeping up or not (so it's not doing any frame skipping like some terminals), and I would expect that at least the escape sequence ordering would remain intact.

With tmux in the mix, the whole terminal state has to be calculated twice. Once in tmux, and once by the "real" terminal emulator. Those two representations could also be very different depending on what escape sequences are received by tmux and what type of terminal tmux thinks it's attached to since tmux will try its best to convert escape sequences to what is most appropriate for your actual terminal.

Maybe the version of the terminal output as rendered by tmux is actually wrong depending on what combination of features it tries to use depending on the terminal (clearing the screen before the sixel renders as you described), or maybe tmux happens to choose a combination escape sequences to represent the terminal state which are buggy depending on the terminal, even though iamb by itself produces output that works with that same terminal.