Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prototype support for PEP 739 #13945

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

Conversation

FFY00
Copy link
Member

@FFY00 FFY00 commented Nov 22, 2024

The overall goal is to support targeting Python installations without need for introspection, given all necessary information is given externally.

This is built upon:

This PR is just a prototype, as I work to get all necessary pieces finalized.

@FFY00
Copy link
Member Author

FFY00 commented Jan 20, 2025

Rebased on master.

@FFY00 FFY00 force-pushed the pep-739-prototype branch 2 times, most recently from 64f9781 to e9deda3 Compare January 20, 2025 23:42
Copy link
Member

@eli-schwartz eli-schwartz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is starting to look quite nice! \o/ How close are we to getting the PEP approved and deployed (e.g. in a cpython alpha)?

test cases/unit/125 python extension/build-details.json Outdated Show resolved Hide resolved
mesonbuild/modules/python.py Outdated Show resolved Hide resolved
@FFY00
Copy link
Member Author

FFY00 commented Jan 21, 2025

This is starting to look quite nice! \o/ How close are we to getting the PEP approved and deployed (e.g. in a cpython alpha)?

Thanks!

I have requested a review for the PEP, so hopefully we should hear something soon. If it gets accepted, I'll add the file to CPython as soon as possible, so it should be available on the following release. I'll also publish a package to generate the build config file to PyPI, so that we can use it on older Python versions, or Python implementations that do not ship it, as long as you are able to run code on the target.

Afterwards, in a followup to PEP 739, I also want to standardize a file for the installation environment, which will provide the installation paths. We'll ship a file for the default environment, and unless we run into some issue, I'll also get venv to generate a file for the virtual environment it creates. With the build config + environment config, we should be able to provide all Python-related information statically to Meson.

@FFY00 FFY00 force-pushed the pep-739-prototype branch 3 times, most recently from e3b2bfb to 7ca5393 Compare January 21, 2025 01:30
@FFY00 FFY00 force-pushed the pep-739-prototype branch from 7ca5393 to 43fcae5 Compare January 22, 2025 01:17
@FFY00 FFY00 force-pushed the pep-739-prototype branch from 43fcae5 to f30406d Compare January 26, 2025 05:56
@FFY00
Copy link
Member Author

FFY00 commented Feb 5, 2025

The PEP has been accepted.

https://peps.python.org/pep-0739/

I'll add support for it in Python 3.14, and then come back to this PR.

Copy link
Member

@eli-schwartz eli-schwartz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic news! Looking forward to local testing as soon as the backportable version exists. Some more quick comments.

if mesonlib.version_compare(schema_version, f'> {self.IMPLEMENTED_VERSION}'):
mlog.log(
f'python.build_config has schema_version {schema_version!r}, '
f'but we only implement suport for {self.IMPLEMENTED_VERSION!r}, '
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
f'but we only implement suport for {self.IMPLEMENTED_VERSION!r}, '
f'but we only implement support for {self.IMPLEMENTED_VERSION!r}, '

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And a few lines up as well.

Comment on lines +472 to +475
mlog.debug(
f'{pkg_name!r} could not be found in {pkg_libdir_origin}, '
'this is likely due to a relocated python installation'
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if we end up line-breaking we would usually do it like:

            mlog.debug(f'{pkg_name!r} could not be found in {pkg_libdir_origin}, '
                       'this is likely due to a relocated python installation')

Comment on lines -259 to +265
extra_paths: T.List[str] = self.env.coredata.optstore.get_value(key)[:]
extra_paths: T.List[str] = self.env.coredata.optstore.get_value(key)[:] + self.extra_paths
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I see what you're doing here. You allow a PkgConfigDependency to take an additional PKG_CONFIG_PATH which is joined to the one in the cross file, which also means we don't need to push/pop os.environ.

On the flip side, the way we previously used it was via PKG_CONFIG_LIBDIR which was more narrowly scoped. I suspect this change means we can accidentally fall back to a different python than the one we were searching for (but still the same major.minor version at least)...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah hold on, we anyways fall back to checking for pkg-config without the internal LIBPC which means this is actually doing exactly what we want with one less check. Heh. :)

@@ -256,7 +262,7 @@ def _check_pkgconfig(self, pkgbin: ExternalProgram) -> T.Optional[str]:
def _get_env(self, uninstalled: bool = False) -> EnvironmentVariables:
env = EnvironmentVariables()
key = OptionKey('pkg_config_path', machine=self.for_machine)
extra_paths: T.List[str] = self.env.coredata.optstore.get_value(key)[:]
extra_paths: T.List[str] = self.env.coredata.optstore.get_value(key)[:] + self.extra_paths
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need the cryptic copy of the first list anymore, right?

Comment on lines +48 to +51
if sys.version_info >= (3, 8):
from functools import cached_property
else:
cached_property = property
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a lazy_property in utils somewhere now. It's provides more or less what cached_property does (although it's simpler and expects the attribute to be write once)

Comment on lines +2983 to +2986
with tempfile.NamedTemporaryFile(mode='w', delete=False, encoding='utf-8') as python_build_config_file:
json.dump(python_build_config, fp=python_build_config_file)

with tempfile.NamedTemporaryFile(mode='w', delete=False, encoding='utf-8') as cross_file:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neither of these get cleaned up right now, do they?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants