update yuvRgbConvCoeffs slightly

[for13to1]

Minor modifications here can bring the conversion result closer to the theoretical value.

[ChristianFeldmann]

Could you share which formula and rounding you used for this? Has been a while since I calculated these.
Since we are rounding to RGB 8 bit I don't think that there will be a measurable difference in the output. Which does not mean that we should not have correct values here. More for prioritization purposes.

[for13to1]

Could you share which formula and rounding you used for this? Has been a while since I calculated these. Since we are rounding to RGB 8 bit I don't think that there will be a measurable difference in the output. Which does not mean that we should not have correct values here. More for prioritization purposes.

Well, the conversion from YUV to RGB can be described as this form:

in which the parameters are obtained from the official document of ITU Rec BT series, as following:

# bt601
k_R, k_G, k_B = 0.299, 0.587, 0.114
# bt709
k_R, k_G, k_B = 0.2126, 0.7152, 0.0722
# bt2020
k_R, k_G, k_B = 0.2627, 0.6780, 0.0593

It could be inferred that the original conversation in this repo is for 8bit data with 16bit decimal precision.

For 8bit RGB: if full, $l_T=0, h_T=255$; if limited, $l_T=16, h_T=235$.
For 8bit YUV: $m_C=128$ fixed, if full, $l_Y=0, h_Y=255, l_C=0, h_C=255$; if limited $l_Y=16, h_Y=235, l_C=16, h_C=240$.

Then we can get:

// yuv2rgb_bt601_full2full_data8_coefF
{   {(double)1, (double)0, (double)701/500, -(double)22432/125},
    {(double)1, -(double)25251/73375, -(double)209599/293500, (double)9939296/73375},
    {(double)1, (double)443/250, (double)0, -(double)28352/125}
},
// yuv2rgb_bt709_full2full_data8_coefF
{   {(double)1, (double)0, (double)3937/2500, -(double)125984/625},
    {(double)1, -(double)1674679/8940000, -(double)4185031/8940000, (double)4687768/55875},
    {(double)1, (double)4639/2500, (double)0, -(double)148448/625}
},
// yuv2rgb_bt2020_full2full_data8_coefF
{   {(double)1, (double)0, (double)7373/5000, -(double)117968/625},
    {(double)1, -(double)5578351/33900000, -(double)19368871/33900000, (double)99788888/1059375},
    {(double)1, (double)9407/5000, (double)0, -(double)150512/625}
},
// yuv2rgb_bt601_limited2full_data8_coefF
{   {(double)85/73, (double)0, (double)35751/22400, -(double)2847823/12775},
    {(double)85/73, -(double)1287801/3287200, -(double)10689549/13148800, (double)1016668969/7498925},
    {(double)85/73, (double)22593/11200, (double)0, -(double)3536578/12775}
},
// yuv2rgb_bt709_limited2full_data8_coefF
{   {(double)85/73, (double)0, (double)200787/112000, -(double)15847451/63875},
    {(double)85/73, -(double)28469543/133504000, -(double)71145527/133504000, (double)585342011/7613900},
    {(double)85/73, (double)236589/112000, (double)0, -(double)18460997/63875}
},
// yuv2rgb_bt2020_limited2full_data8_coefF
{   {(double)85/73, (double)0, (double)376023/224000, -(double)29829679/127750},
    {(double)85/73, -(double)94831967/506240000, -(double)329270807/506240000, (double)12790351251/144357500},
    {(double)85/73, (double)479757/224000, (double)0, -(double)37402261/127750}
},

Then left shift by 16 and with rounding, we can get the fully fixed version as I proposed in this PR.

[ValeZAA]

Also do not forget that AviSynth/AviSynthPlus#259

That means that instead of +/-127 (or +/-32767), it must be handled as 128+/-127.5 (or 32768 +/-32767.5) before it gets truncated back to 8 (16) bit range. Then the conversion must use factor of 112 / 127.5 (that is 224.0/255.0)

[for13to1]

Also do not forget that AviSynth/AviSynthPlus#259

That means that instead of +/-127 (or +/-32767), it must be handled as 128+/-127.5 (or 32768 +/-32767.5) before it gets truncated back to 8 (16) bit range. Then the conversion must use factor of 112 / 127.5 (that is 224.0/255.0)

Hi @ValeZAA
The coefficient 224/255 you mentioned is for RGB2YUV, more specifically for full RGB to limited YUV.
The reverse conversion (YUV2RGB) I proposed is exactly using 128 (8bit digital value) for chroma scale to [-0.5,0.5] (analog value).
Only 128 (and its shifts like 32768) is needed, which means 127.5 or 32767.5 is not.

Would you explain the link you attached more clearly? What it is that I missed in the conversion formula?

[ValeZAA]

The coefficient 224/255 you mentioned is for RGB2YUV, more specifically for full RGB to limited YUV

Yes, you are right.

Cb = Clip1_C ( Round( ( ( 1 << BitDepth_C ) − 1 ) * E′_PB ) + ( 1 << ( BitDepth_C − 1 ) ) )

Chroma in Cb, Cr in float and goes from -0.5 and +0.5 and those map to 0.5 and 255.5 before rounding and clipping.

analog value

I mean... It is just float value, still digital float. Analog YIQ and YUV are very different matrices.

[for13to1]

The coefficient 224/255 you mentioned is for RGB2YUV, more specifically for full RGB to limited YUV

Yes, you are right.

Cb = Clip1_C ( Round( ( ( 1 << BitDepth_C ) − 1 ) * E′_PB ) + ( 1 << ( BitDepth_C − 1 ) ) )

Chroma in Cb, Cr in float and goes from -0.5 and +0.5 and those map to 0.5 and 255.5 before before rounding and clipping.

analog value

I mean... It is just float value, still digital float. Analog YIQ and YUV are very different matrices.

Well, the "analog value" I used is for electrical level values in range [0,1] and those in [-0.5,0.5], biased relative values.

Could you please explain more clearly, where is it that 255.5 or 127.5 coming from?

[ChristianFeldmann]

So I also took a look and I think the whole conversion here is not really correct. I looked at the standard (https://www.itu.int/rec/R-REC-BT.2020/en) and inverted the formulas in there (10 bit YUV limited range to RGB 8 bit full range) and got quite different values:

{76608, 110445, -12325, -42793, 140914}

I think I never calculated these values myself but inherited them from previous code. Maybe this code needs a good rework overall. Ultimately I was planning on implementing an external library for the conversions aniways. I was thinking about zImg because that can do all color conversions as well as dithering. It also has some assembly optimizations ...

The calculation you presented above is not really derived from the BT.2020 standard, is it?

[for13to1]

So I also took a look and I think the whole conversion here is not really correct. I looked at the standard (https://www.itu.int/rec/R-REC-BT.2020/en) and inverted the formulas in there (10 bit YUV limited range to RGB 8 bit full range) and got quite different values:

{76608, 110445, -12325, -42793, 140914}

I think I never calculated these values myself but inherited them from previous code. Maybe this code needs a good rework overall. Ultimately I was planning on implementing an external library for the conversions aniways. I was thinking about zImg because that can do all color conversions as well as dithering. It also has some assembly optimizations ...

The calculation you presented above is not really derived from the BT.2020 standard, is it?

Hi, @ChristianFeldmann

10 bit YUV limited range to RGB 8 bit full range

Generally, this conversion could be divided into 2 parts:

1. 10bit limited YUV to 10bit full RGB;
2. 10bit RGB to 8bit RGB

The first part is where conversion coefficients are needed for sure, the latter one can be done with just right-shift or dithering.

With precondition as above, 16bit decimal precision coefficient for 10bit limited YUV to 10bit full RGB through BT.2020 is:

76533, 110337, -12313, -42752, 140776

The calculation you presented above is not really derived from the BT.2020 standard, is it?

The key coefficient I presented as above (k_R, k_G, k_B = 0.2627, 0.6780, 0.0593) is exactly the same as what is stipulated in the BT.2020 standard document.
Therefore, the fixed point version I presented is exactly derived from the BT.2020 standard (in more detail, it is under constraint that no bit_depth change should happen during the conversion).

[for13to1]

Maybe this code needs a good rework overall.

@ChristianFeldmann

The calculation in the original code is OK.
As you can see, there is only little difference in the patch I proposed (File Changed tab in this PR).

[ValeZAA]

ITU-T Series H Supplement 15 (01/2017) says (https://www.itu.int/itu-t/recommendations/rec.aspx?rec=13243)

what the inverted matrix should be. Remember, the main matrix is the matrix from R'G'B' to Y'Cb'Cr'. That matrix has 4 digits and is not allowed to be higher presicion. It is the Y'Cb'Cr to R'G'B' that is inverted, and depends on bitness. 16 bits may need more digits in the matrix.

[for13to1]

ITU-T Series H Supplement 15 (01/2017) says (https://www.itu.int/itu-t/recommendations/rec.aspx?rec=13243)

what the inverted matrix should be. Remember, the main matrix is the matrix from R'G'B' to Y'Cb'Cr'. That matrix has 4 digits and is not allowed to be higher presicion. It is the Y'Cb'Cr to R'G'B' that is inverted, and depends on bitness. 16 bits may need more digits in the matrix.

Hi @ValeZAA

This ITU-T H.Sup15 document you mentioned is about "Conversion and coding practices for HDR/WCG Y'CbCr 4:2:0 video with PQ transfer characteristics".
Do you mean that HDR/WCG content should be considered?

This PR is about BT.601/BT.709/BT.2020 conversion, and more precisely, is about linear matrix conversion, with no consideration about gamma/EOTF/OETF.

[ValeZAA]

This PR is about BT.601/BT.709/BT.2020 conversion, and more precisely, is about linear matrix conversion, with no consideration about gamma/EOTF/OETF.

And that is what that document is about too. I quoted from it, see screenshot. But, k_R, k_G, k_B = 0.2627, 0.6780, 0.0593 are derived from BT.2020 primaries and white point.

[for13to1]

This PR is about BT.601/BT.709/BT.2020 conversion, and more precisely, is about linear matrix conversion, with no consideration about gamma/EOTF/OETF.

And that is what that document is about too. I quoted from it, see screenshot. But, k_R, k_G, k_B = 0.2627, 0.6780, 0.0593 are derived from BT.2020 primaries and white point.

Sorry, I am confused. What is wrong with "k_R, k_G, k_B = 0.2627, 0.6780, 0.0593 are derived from BT.2020 primaries and white point"? Could you explain it?

[ValeZAA]

What is wrong with "k_R, k_G, k_B = 0.2627, 0.6780, 0.0593 are derived from BT.2020 primaries and white point"? Could you explain it?

Nothing is wrong. I am saying that WCG is implicitly used in matrix. Indeed,

D65 is xy 0.3127 0.3290

Red: 0.708 0.292

Green 0.170 0.797

Blue 0.131 0.046

Then use H.273 document eq 39— 44 https://www.itu.int/rec/T-REC-H.273-202309-T/en

Kr and Kb are calculated using those 8 numbers above.

[for13to1]

What is wrong with "k_R, k_G, k_B = 0.2627, 0.6780, 0.0593 are derived from BT.2020 primaries and white point"? Could you explain it?

Nothing is wrong. I am saying that WCG is implicitly used in matrix. Indeed,

D65 is xy 0.3127 0.3290

Red: 0.708 0.292

Green 0.170 0.797

Blue 0.131 0.046

Then use H.273 document eq 39— 44 https://www.itu.int/rec/T-REC-H.273-202309-T/en

Kr and Kb are calculated using those 8 numbers above.

When talking about BT.601/BT.709/BT.2020, shouldn't we just refer to BT.601/BT.709/BT.2020?
What's the point diving into HDR or WCG? I still didn't get it.

Do you mean that HDR and WCG options need taking into account?
But, then it will be not only about linear matrix conversion, Gamma/EOTF/OETF will be involved, which cannot be done within this conversion.
Values here are non-linear type during the conversion.

[ValeZAA]

When talking about BT.601/BT.709/BT.2020, shouldn't we just refer to BT.601/BT.709/BT.2020

Where is your math provided in those docs? No, we need to read all related to series H docs. Again, supplement 15 talks about matrix and chroma siting and upscaling from 420 to 444 algorithm, its FIR filter for topleft or left chroma siting, it is implememted in YUView.

Values here are non-linear type during the conversion.

Yes, R'G'B'.

[for13to1]

When talking about BT.601/BT.709/BT.2020, shouldn't we just refer to BT.601/BT.709/BT.2020

Where is your math provided in those docs? No, we need to read all related to series H docs. Again, supplement 15 talks about matrix and chroma siting and upscaling from 420 to 444 algorithm, its FIR filter for topleft or left chroma siting, it is implememted in YUView.

Values here are non-linear type during the conversion.

Yes, R'G'B'.

Have you reviewed the modifications in this PR? They are about BT.601/BT.709/BT.2020 only.

we need to read all related to series H docs

No, series BT docs is enough, because only BT.601/BT.709/BT.2020 coefficients are modified for now, they are prefixed with "BT", NOT "H".

And they have nothing to do with dithering or gamma or chroma-downsample or HDR or WCG you mentioned above.
They are just only linear matrix conversion stuffs.

If you care about those things that could be added into this project, you can open new issues or PR.