Support u16/f32 pixmaps

[RazrFalcon]

Good to have. This is technically already supported, but we have to provide a sane public API for it.

[virtualritz]

Can you elaborate a bit?
What would be needed to get such a Pixmap out from the crate? Especially the f32 kind?

[virtualritz]

Also: this seems to be at odds with one of the non-goals on the readme.

[RazrFalcon]

Well, Pixmap is just a storage of bytes. The actual rendering is done using u16/f32 pipelines already. So all we have to do is to figure out a nice API for those Pixmaps. It could be 3 different types. It could be a generic parameter. It could be just an enum and a Pixmap would always be u8. Many possibilities. Many design decisions. And I simply had no time to look into it.

PS: Readme is somewhat outdated.

[virtualritz]

I guess one option is to make it Pixmap<T, F> where F is a transfer/mapping function.

E.g.:

use num::{traits::ConstZero, Num};
use rand::prelude::*;
use smallvec::SmallVec;

struct Pixmap<T, F>
where
    T: Clone + Num + ConstZero,
    F: FnMut([f32; 4]) -> [T; 4],
{
    pixels: Vec<T>,
    mapper: F,
}

impl<T, F> Pixmap<T, F>
where
    T: Clone + Num + ConstZero,
    F: FnMut([f32; 4]) -> [T; 4],
{
    pub fn new(width: u32, height: u32, mapper: F) -> Self {
        Self {
            pixels: vec![T::ZERO; (width * height) as _],
            mapper,
        }
    }
}

fn main() {
    let mapper_u8_naive = |rgba: [f32; 4]| {
        rgba
            .iter()
            .map(|v| (v * 255.0) as u8)
            .collect::<SmallVec<[u8; 4]>>()
            .into_inner()
            .unwrap()
    };

    let pixmap = Pixmap::new(320, 240, mapper_u8_naive);

    let mut rng = rand::thread_rng();

    // Map a value between 0..1 to 0..255 w. random noise dithering.
    let mapper_u8_dither = |rgba: [f32; 4]| {
        const ONE: f32 = u8::MAX as _;
        const MIN: f32 = 0.0;
        const MAX: f32 = u8::MAX as _;
        const DITHER: f32 = 0.5;

        rgba
            .iter()
            .map(|v| (ONE * v + DITHER * rng.gen::<f32>()).clamp(MIN, MAX) as u8)
            .collect::<SmallVec<[u8; 4]>>()
            .into_inner()
            .unwrap()
    };

    let mut rng = rand::thread_rng();

    // Map a value between 0..1 to 0..16383 and values above in the remaining range,
    // clipping at 2.0 and w. random noise dithering.
    let mapper_u16_headroom_dither = |rgba: [f32; 4]| {
        const ONE: f32 = (u16::MAX / 2) as _;
        const MIN: f32 = 0.0;
        const MAX: f32 = u16::MAX as _;
        const DITHER: f32 = 0.5;

        rgba
            .iter()
            .map(|v| (ONE * v + DITHER * rng.gen::<f32>()).clamp(MIN, MAX) as u8)
            .collect::<SmallVec<[u8; 4]>>()
            .into_inner()
            .unwrap()
    };
}

Playground

This allows a user to map to types that are not even PODs, e.g. f16 (i.e. the half crate) and do on-the-fly color conversion.

For example, if the assumed working space was ACEScg and the output required to be linear sRGB in u16, a resp. transfer function could be added inside the mapper closure.

tiny-skia would ship ready-made mappers, e.g. U8_SRGB_GAMMA that just takes the raw f32 input and maps it to gamma-corrected sRGB.

[virtualritz]

The above could also be called CustomPixmap or the like so the API remains the same.

[virtualritz]

Would a PR with the above approach have any chance of approval? 😁

[RazrFalcon]

Transferring function is definitely not an option. Pixmap must not alter an underlying data.
Pixmap must remain just a dumb Vec<T> wrapper.

[virtualritz]

Hmm, but Pixmap uses per-component u8 at the moment, so there is some transfer function already in the current implemenation, or am I missing somthing?

I presume the current hard-wired function is from (assumed) linear sRGB u16/f32 to gamma-corrected sRGB u8.

When T is is not known, how would data be stored in the target Pixmap by tiny-skia w/o such a function?

[RazrFalcon]

Right now, Pixmap is premultiplied sRGB (non linear, no gamma-corrected). And yes, u8 is being promoted to u16/f32 when needed.
But it would not be required for u16/f32 pixmaps.

Either way, this would be a pretty complicated change and I'm not expecting anyone to tackle it.

[virtualritz]

The main reason I asked for support for higher bit depths is banding on gradients.

This is especially relevant if the output of tiny-skia is further modified, e.g. with a color grade operation. This is one of the use cases I deal with for the typst-render output in my app (typst-render uses tiny-skia as the rendering backend).

But banding could be prevented/hidden with dithering. I opened an issue in the new color crate for this.

I think this could be relevant here too resp. be a 'workaround' until u16/f32 support lands in tiny-skia, one day.