llama.cpp: add model_converters variant

[i0ntempest]

Description

Type(s)

• bugfix
• enhancement
• security fix

Tested on

macOS x.y
Xcode x.y / Command Line Tools x.y.z

Verification

Have you

• followed our Commit Message Guidelines?
• squashed and minimized your commits?
• checked that there aren't other open pull requests for the same change?
• referenced existing tickets on Trac with full URL in commit message?
• checked your Portfile with port lint?
• tried existing tests with sudo port test?
• tried a full install with sudo port -vst install?
• tested basic functionality of all binary files?
• checked that the Portfile's most important variants haven't been broken?

[macportsbot]

Notifying maintainers:
@blair for port py-tensorflow_estimator.
@mdouze for port libfaiss.
@beauby for port libfaiss.
@emcrisostomo for port py-absl, py-tensorboard.
@cjones051073 for port py-blis, py-confection, py-plac, py-preshed, py-srsly, py-thinc, py-wasabi.
@reneeotten for port py-cookiecutter.
@stromnov for port py-scikit-learn.

[i0ntempest]

@mascguy Please take a look!
This massive PR took me like 2 days, and there's almost certainly something that I fricked up. For one I cannot track the dependency relations of all the python ports I changed anymore.
What I've done:
Ran through all the dependencies needed for the four conversion scripts to work, adding compatibility up to Python 3.12 (or 3.13 if the package supports it), and updating them if possible.

What I've tested and works:

1. Importing gguf, torch, tensorflow, transformers in python
2. Running the tensorflow test script from Apple here
3. Running all the conversion scripts without args (that is I did not test any actual conversions)

Some notes:

1. py-preshed must be downgraded for py-spaCy to build
2. py-tensorflow-macos was updated to use tensorflow pypi instead of tensorflow-macos, which seems defunct. This is the only way we can get to python 3.12, and I didn't dare touch the actual py-tensorflow port
3. Multiple build workarounds needed because our numpy is ancient
4. Old protobuf3-cpp blocked me from updating grpc

[mascguy]

I'll take a closer look.

Also adding @essandess for review/comment, as he's contributed to a number of Python ports in this space.

[i0ntempest]

I haven’t ironed out all the problems yet but all 3 CI tests fail at 381 minutes, is this a timeout?
Also I can’t see any log from 15.0 CI.

[i0ntempest]

Yea it's still py312-spaCy failing

[mascguy]

Yea it's still py312-spaCy failing

Would it be possible to update some of the more core dependencies - like py-onyx, etc - to get us closer? Maybe move those to a separate PR, which can pass CI?

[i0ntempest]

Yea it's still py312-spaCy failing

Would it be possible to update some of the more core dependencies - like py-onyx, etc - to get us closer? Maybe move those to a separate PR, which can pass CI?

I can but I doubt that'll fix anything, the failure is not caused by the 6h timeout. If you download the full log of the 14.0 check you may see a bunch of cython related errors (like I said above). I could not repro those errors on my system as the port builds just fine here. Can you check if you get this error?

[mascguy]

Yea it's still py312-spaCy failing

Would it be possible to update some of the more core dependencies - like py-onyx, etc - to get us closer? Maybe move those to a separate PR, which can pass CI?

I can but I doubt that'll fix anything, the failure is not caused by the 6h timeout. If you download the full log of the 14.0 check you may see a bunch of cython related errors (like I said above). I could not repro those errors on my system as the port builds just fine here. Can you check if you get this error?

Unfortunately py-pytorch is still failing to build on Ventura, so the patches may need to be adjusted.

[i0ntempest]

Unfortunately py-pytorch is still failing to build on Ventura, so the patches may need to be adjusted.

Well, better tag some C/C++ (and cython) experts for help. Debugging this kind of build is pretty much above my skill level.

[essandess]

FWIW, I've also been trying unsuccessfully for over two years to update MacPorts's transformers, PyTorch, and associated dependencies (#19976, #19975, #19974, #19969, #19968, #19967, and several more).

The broken dependency chain and update workflow became untenable, so I had to migrate to pypi/virtualenv/conda as appropriate. I've been trying my hand at this again lately, but am still running into the same issues that @i0ntempest details above.

Does the PG python provide a method of installing specific versions of Python packages the way cargo.crates or virtualenv work to resolve conflicts like this? I see this Python package version issue in other ports such as #27227 where the installer itself uses virtualenv to download pinned package versions from pypi, but this breaks the MacPorts CI.

cc: @reneeotten