Sync with 3.0.0. #92

LaurenzV · 2024-02-07T09:03:31Z

My first attempt. let's see how it goes.

Legend:
🟢: Port of commit succeeded
⚪️: Commit is not relevant
🟡: I have uncertainties about that commit
🔴: Port of commit is failing
⚫️: Commit has not been ported yet.

Table:

Status	Commit message	HB Link
⚪️	Typo	Link
⚪️	[ci] Use different executer for win64 jobs	Link
⚪️	[docs] Fix subset documentation	Link
⚪️	[subset] Improve alignment of --help-all message	Link
⚪️	[subset] fox for (Chain)ContextFormat3: subset lookupRecord	Link
⚪️	[style] Fall back to GPOS 'size' feature for optical size in style API	Link
⚪️	[style] Rename HB_STYLE_TAG_SLANT to HB_STYLE_TAG_SLANT_ANGLE	Link
⚪️	[style] Add HB_STYLE_TAG_SLANT_RATIO	Link
⚪️	Fix build with -Dexperimental_api=true	Link
⚪️	[style] Prepare for release	Link
⚪️	[style] Fix documentation	Link
⚪️	[style] Try to fix MSVC build	Link
⚪️	[style] Try harder to fix MSVC build	Link
⚪️	[style] Change private slant tag	Link
⚪️	[style] Fix float conversion warning	Link
⚪️	Call atexit() via hb_atexit()	Link
🟢	[ot-shape] Enabled two more features: 'Harf' and 'Buzz'	Link
⚪️	[style] Fix another float conversion warning	Link
⚪️	Add fallback atexit implementation using template descrutors	Link
⚪️	[util/shape-consumer] Simplify font tracking	Link
⚪️	[util] Fix leak of an hb_font_t	Link
🟢	[Unicode 14] Add new `hb_script_t` values	Link
⚪️	[Unicode 14] Update the UCD table	Link
🟢	[Unicode 14] Update the script direction list	Link
🟢	[Unicode 14] Update the Arabic table	Link
🟢	[Unicode 14] Update the Indic table	Link
🟢	[Unicode 14] Update the vowel constraint table	Link
⚪️	[Unicode 14] Update the emoji table	Link
🟢	[Unicode 14] Update the USE table	Link
🟢	[Unicode 14] Send all the new scripts to USE	Link
⚪️	[Unicode 14] Add tests	Link
🟢	[Unicode 14] Update the Arabic joining script list	Link
⚪️	Merge pull request #3206 from harfbuzz/unicode-14	Link
⚪️	[meson] Require ragel 6.10	Link
⚪️	[ragel] Regenerate state machine files with 6.10	Link
⚪️	[meson] Add ragel subproject	Link
🟢	[USE] Send Khitan Small Script and Yezidi to USE	Link
⚪️	[style] Change tag type to hb_style_tag_t	Link
⚪️	CMake: Detect pthread with module FindThread	Link
⚪️	[subset] In preperation for 3.0.0 release remove legacy subsetting api methods.	Link
⚪️	[subset] restore hb_subset_input_unicode/glyph_set () to documentation.	Link
⚪️	Merge pull request #3210 from NikitaFeodonit/main	Link
⚪️	[meson] Add ragel_subproject option	Link
⚪️	[ci] Build ragel on the Linux CI job	Link
⚪️	[ci] Ragel subproject requires meson 0.55.0	Link
⚪️	[ci] meson 0.55.0 didn‚Äôt work for harfbuzz	Link
⚪️	Centralize math include	Link
⚪️	3.0.0	Link

harfbuzz/harfbuzz@5a1a6f1fe

harfbuzz/harfbuzz@67f1aa4d2

harfbuzz/harfbuzz@7b05eec05

harfbuzz/harfbuzz@11d5334d8

harfbuzz/harfbuzz@ec5688f50

LaurenzV · 2024-02-07T10:04:46Z

So as far as I can tell arabic tables and ucd tables are all covered by gen-universal-table.py? So for harfbuzz/harfbuzz@18a46eeaf and harfbuzz/harfbuzz@26e1498e7 it should be fine to just update that script to 14.0?

EDIT: Nevermind, seems like the corresponding Python file in harfbuzz is gen-use-table... So I'm not sure how to deal with this commit?

LaurenzV · 2024-02-07T10:19:25Z

And it seems like we have gen-unicode-is-emoji-ext-pict.py for emojis but no corresponding .rs file? How do I deal with that?

harfbuzz/harfbuzz@20736d9bb

harfbuzz/harfbuzz@2c3e4b279

harfbuzz/harfbuzz@f1493357f

harfbuzz/harfbuzz@470686e8

RazrFalcon · 2024-02-07T12:20:22Z

Wow, you have put a lot of effort into it. Much appreciated!

[style] Fall back to GPOS 'size' feature for optical size in style API

Yes, we don't have such API.

[util/shape-consumer] Simplify font tracking

Yes, unrelated.

[Unicode 14] Add tests

Yes, we don't have those.

[meson] Require ragel 6.10

Yes, we use master branch already.

So as far as I can tell arabic tables and ucd tables are all covered by gen-universal-table.py?

Afair, hb-ot-shape-complex-arabic-table.hh was ported by hand. At least there are no script for it and the file doesn't start with the this file was generated comment.
So you have to update it manually (which is very easy). Or port harfbuzz's script if you feel like it (harder).

As for hb-ucd-table.hh, we don't have it at all. We either use 3rd-party dependencies or our own scripts. Therefore updating to 14.0.0 in Python scripts + updating unicode-* crates should be enough.

And it seems like we have gen-unicode-is-emoji-ext-pict.py for emojis but no corresponding .rs file? How do I deal with that?

src/unicode.rs:is_emoji_extended_pictographic
I will update docs.

LaurenzV · 2024-02-07T12:33:32Z

Gotcha!

RazrFalcon · 2024-02-07T12:53:05Z

We also have src/unicode.rs:check_unicode_version tests, in case you haven't saw those. They all should be 14.0 now, I guess.

LaurenzV · 2024-02-07T13:00:26Z

Ah, I guess we need to update the dependencies too, then.

LaurenzV · 2024-02-07T13:02:16Z

Should I submit PR to those (unicode-cc and unicode-bidi)? Or is it faster if you just do it when you have time?

LaurenzV · 2024-02-07T13:07:18Z

#[derive(Clone, Copy, PartialEq, Debug)]
pub enum JoiningType {
    U = 0,
    L = 1,
    R = 2,
    D = 3,
    // We don't have C, like harfbuzz, because Rust doesn't allow duplicated enum variants.
    GroupAlaph = 4,
    GroupDalathRish = 5,
    T = 7,
    X = 8, // means: use general-category to choose between U or T.
}

Why would adding C here make it duplicated?

RazrFalcon · 2024-02-07T13:16:32Z

I will update my unicode crates soon.

Why would adding C here make it duplicated?

Because Rust doesn't allow enums to have duplicate values. The C++ code is:

enum hb_arabic_joining_type_t {
  JOINING_TYPE_D		= 3,
  JOINING_TYPE_C		= JOINING_TYPE_D,
}

LaurenzV · 2024-02-07T13:18:26Z

Right, so I always need to use D instead of C.

RazrFalcon · 2024-02-07T13:35:54Z

Yep.

RazrFalcon · 2024-02-07T13:45:20Z

Updated my unicode-* crates. Use version 0.2 for both.

harfbuzz/harfbuzz@18a46eeaf

LaurenzV · 2024-02-07T14:01:46Z

Okay, just the emoji commit is missing now: harfbuzz/harfbuzz@35ac0de42

I'm unsure what to do with it though. They basically add this file: https://www.unicode.org/Public/emoji/latest/emoji-test.txt

But it seems that this file doesn't contain any information on extended pictographics. But it seems we only use the file to generate that one method. Does this mean we can ignore this file?

RazrFalcon · 2024-02-07T15:10:10Z

Yes, I think we can ignore that emoji change. It doesn't seems to affect us.
harfbuzz is not just a shaper, but rather a font processing toolkit. It has far more features that we do not implement in rustybuzz.

The rule of thumb: if tests are passing - we're fine.

LaurenzV · 2024-02-07T15:32:30Z

I guess porting itself is not as hard as I thought, just time-consuming. But I will probably change my mind once I encounter my first not-passing test and spend hours with the debugger. 😄

RazrFalcon · 2024-02-07T15:41:18Z

Pretty much.

khaledhosny · 2024-02-07T22:13:22Z

Okay, just the emoji commit is missing now: harfbuzz/harfbuzz@35ac0de42

I'm unsure what to do with it though. They basically add this file: https://www.unicode.org/Public/emoji/latest/emoji-test.txt

But it seems that this file doesn't contain any information on extended pictographics. But it seems we only use the file to generate that one method. Does this mean we can ignore this file?

This file is used to generate emoji test data (IIRC, to test that all emoji sequences are shaped as one cluster)

LaurenzV added 6 commits February 7, 2024 09:53

Format

9eddb0f

[ot-shape] Enabled two more features: 'Harf' and 'Buzz'

6cd7cf9

harfbuzz/harfbuzz@5a1a6f1fe

[Unicode 14] Add new hb_script_t values

4a77527

harfbuzz/harfbuzz@67f1aa4d2

[Unicode 14] Update the script direction list

d910a32

harfbuzz/harfbuzz@7b05eec05

[Unicode 14] Update the Indic table

69ae0b0

harfbuzz/harfbuzz@11d5334d8

[Unicode 14] Update the vowel constraint table

b9eb3f8

harfbuzz/harfbuzz@ec5688f50

LaurenzV added 4 commits February 7, 2024 11:28

[Unicode 14] Update the USE table

74472fd

harfbuzz/harfbuzz@20736d9bb

[Unicode 14] Send all the new scripts to USE

0263c56

harfbuzz/harfbuzz@2c3e4b279

[Unicode 14] Update the Arabic joining script list

f3b771a

harfbuzz/harfbuzz@f1493357f

[USE] Send Khitan Small Script and Yezidi to USE

3c4cc52

harfbuzz/harfbuzz@470686e8

Update unicode_norm to 14.0

33d217c

LaurenzV added 5 commits February 7, 2024 14:51

Update unicode norm to 14 in test

0e8de06

[Unicode 14] Update the Arabic table

c97ba6a

harfbuzz/harfbuzz@18a46eeaf

Remove useless import in gen-indic-table.py

54ef990

Bump Unicode crates

457119a

Fix formatting

3830820

Fix formatting

c8db555

LaurenzV marked this pull request as ready for review February 7, 2024 15:28

RazrFalcon merged commit 0f0b732 into harfbuzz:master Feb 7, 2024
1 check passed

LaurenzV deleted the 3.0.0 branch February 7, 2024 15:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sync with 3.0.0. #92

Sync with 3.0.0. #92

LaurenzV commented Feb 7, 2024 •

edited

Loading

LaurenzV commented Feb 7, 2024 •

edited

Loading

LaurenzV commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024 •

edited

Loading

LaurenzV commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024 •

edited

Loading

LaurenzV commented Feb 7, 2024

LaurenzV commented Feb 7, 2024

LaurenzV commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024 •

edited

Loading

LaurenzV commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024

LaurenzV commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024

LaurenzV commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024

khaledhosny commented Feb 7, 2024

Sync with 3.0.0. #92

Sync with 3.0.0. #92

Conversation

LaurenzV commented Feb 7, 2024 • edited Loading

LaurenzV commented Feb 7, 2024 • edited Loading

LaurenzV commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024 • edited Loading

LaurenzV commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024 • edited Loading

LaurenzV commented Feb 7, 2024

LaurenzV commented Feb 7, 2024

LaurenzV commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024 • edited Loading

LaurenzV commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024

LaurenzV commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024

LaurenzV commented Feb 7, 2024

RazrFalcon commented Feb 7, 2024

khaledhosny commented Feb 7, 2024

LaurenzV commented Feb 7, 2024 •

edited

Loading

LaurenzV commented Feb 7, 2024 •

edited

Loading

RazrFalcon commented Feb 7, 2024 •

edited

Loading

RazrFalcon commented Feb 7, 2024 •

edited

Loading

RazrFalcon commented Feb 7, 2024 •

edited

Loading