Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix hardcoded reliance on pip #1882

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

vpetrigo
Copy link

@vpetrigo vpetrigo commented Jan 9, 2025

This is an attempt to fix the issue described here juftin/hatch-pip-compile#85.

As it was discussed there, since hatch already depends on uv there is no need to rely on pip install during environment requirements resolution, because it may be not available.

@@ -160,9 +160,9 @@ def ensure_plugin_dependencies(self, dependencies: list[Requirement], *, wait_me
if dependencies_in_sync(dependencies):
return

pip_command = [sys.executable, '-u', '-m', 'pip']
pip_command = [sys.executable, '-u', '-m', 'uv', 'pip']
Copy link
Contributor

@juftin juftin Jan 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uv makes a singular Python import available, find_uv_bin - that may be more reliable than python -m uv ... 🤔
https://github.com/astral-sh/uv/blob/54b3a438d03483c6e76f05168bd417657b693e4b/python/uv/_find_uv.py#L8C5-L8C16

from uv import find_uv_bin
...
uv_bin = find_uv_bin()
pip_command = [uv_bin, '-u', '-m', 'pip']

This likely works the same though

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, I was imagining the fix for this would be some complicated piece of work - but this is a really nice, targeted swap of pip for uv pip only in ensure_plugin_dependencies

Copy link
Contributor

@juftin juftin Jan 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EDIT: This test is passing confirming this works, it will fail if you swap git+https://github.com/vpetrigo/hatch.git@fix/hardcoded-reliance-on-pip with hatch

FROM fedora:latest

RUN dnf install -y git

COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/

RUN uv python install 3.12
RUN uv tool install git+https://github.com/vpetrigo/hatch.git@fix/hardcoded-reliance-on-pip
ENV PATH="/root/.local/bin:${PATH}"

WORKDIR /work

RUN touch /work/pyproject.toml

RUN cat <<EOF > /work/pyproject.toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "foo"
version = "0.0.1"
dependencies = ["requests"]

[tool.hatch.env]
requires = [
    "hatch-pip-compile"
]

[tool.hatch.envs.default]
installer = "uv"
type = "pip-compile"
pip-compile-resolver = "uv"
pip-compile-installer = "uv"
EOF

RUN mkdir /work/foo
RUN touch /work/foo/__init__.py

CMD ["hatch", "run", "head", "requirements.txt"]
docker build . --tag hatch-pip-compile-test
docker run --rm -it hatch-pip-compile-test
#
# This file is autogenerated by hatch-pip-compile with Python 3.12
#
# - requests
#

certifi==2024.12.14
    # via requests
charset-normalizer==3.4.1
    # via requests

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This actually fails now since we've replaced sys.executable -m pip with find_uv_bin 😢

Looks like uv doesn't know what environment to install into

error: No virtual environment found; run `uv venv` to create an environment, or pass `--system` to install into a non-virtual environment

Copy link
Author

@vpetrigo vpetrigo Jan 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rolled back for now to [sys.executable, '-u', '-m', 'uv', 'pip'].

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What may work is the following command:

uv_bin = find_uv_bin()
pip_command = [uv_bin, 'pip']

pip_command.extend(['install', '--directory', str(pathlib.Path(sys.executable).parent.parent)])

Not sure if it is more reliable than calling a uv module with venv Python.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed those changes for review just in the case that approach with find_uv_bin is preferable. 😅

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about two options that would work here:

The first option essentially calls python -m uv pip which when invoked as a module discovers the Python environment it belongs to

pip_command = [sys.executable, "-m", "uv", "pip"]
pip_command.extend(["install"])

The second option is to call the find_uv_bin method, but also passes the current Python interpreter using the --python argument

uv_binary = find_uv_bin()
pip_command = [uv_binary, "pip"]
pip_command.extend(["install", "--python", sys.executable])

Honestly, I think both of these options do the same thing 🤷 - I'm fine with whatever you and Ofek like

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the implementation for the last option with passing --python switch as it is more readable and less confusing than passing venv Python directory with --directory switch. Run through the tests in hatch and with the container you prepared - all passed on my end.

@vpetrigo vpetrigo force-pushed the fix/hardcoded-reliance-on-pip branch from 56d4787 to b1838f8 Compare January 11, 2025 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants