Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(bug): prefix config environment vars to avoid conflicts #2479

Merged
merged 11 commits into from
Jan 28, 2025

Conversation

ydm
Copy link
Contributor

@ydm ydm commented Jan 27, 2025

What I did

The configuration classes (e.g. ape_ethereum.NetworkConfig or ape_test.CoverageReportsConfig) incorrectly interpret environment variables as part of their own settings.

For example there are terminal and html fields in CoverageReportsConfig:

class CoverageReportsConfig(PluginConfig):
    terminal: Union[bool, dict] = True
    xml: Union[bool, dict] = false
    html: Union[bool, dict] = False

Now, if you try running ape with one of those (somewhat common) names set as an environment variable, it will either crash or, even worse, run with one of its settings likely augmented without the user ever intending it.

$ TERMINAL=xterm ape compile

...

pydantic_core._pydantic_core.ValidationError: 2 validation errors for CoverageReportsConfig
terminal.bool
  Input should be a valid boolean, unable to interpret input [type=bool_parsing, input_value='xterm', input_type=str]
    For further information visit https://errors.pydantic.dev/2.10/v/bool_parsing
terminal.dict[any,any]
  Input should be a valid dictionary [type=dict_type, input_value='xterm', input_type=str]
    For further information visit https://errors.pydantic.dev/2.10/v/dict_type

The reason for this behaviour is that the pydantic-settings module searches for environment variables of the same name as each setting: https://docs.pydantic.dev/latest/concepts/pydantic_settings/#environment-variable-names

I couldn't find a way to disable the environment variable settings.

To solve the problem, this patch introduces the following prefixes based on their root module:

  • APE_
  • APE_CACHE_
  • APE_COMPILE_
  • APE_CONSOLE_
  • APE_ETHEREUM_
  • APE_NETWORKS_
  • APE_NODE_
  • APE_TEST_

The following config classes are affected:

| File                                 | Class                   | Prefix                    | Test |
|--------------------------------------+-------------------------+---------------------------+------|
| src/ape/api/config.py                | ApeConfig               | APE_                      | yes  |
| src/ape/api/config.py                | DeploymentConfig        | APE_                      | NO!  |
| src/ape/api/config.py                | PluginConfig            | APE_                      | NO!  |
| src/ape_cache/config.py              | CacheConfig             | APE_CACHE_                | yes  |
| src/ape_compile/config.py            | Config                  | APE_COMPILE_              | yes  |
| src/ape_console/config.py            | ConsoleConfig           | APE_CONSOLE_              | yes  |
| src/ape_ethereum/ecosystem.py        | BaseEthereumConfig      | APE_ETHEREUM_             | yes  |
| src/ape_ethereum/ecosystem.py        | ForkedNetworkConfig     | APE_ETHEREUM_ (inherited) | yes  |
| src/ape_ethereum/ecosystem.py        | NetworkConfig           | APE_ETHEREUM_             | yes  |
| src/ape_networks/config.py           | CustomNetwork           | APE_NETWORKS_             | yes  |
| src/ape_networks/config.py           | NetworksConfig          | APE_NETWORKS_             | NO!  |
| src/ape_node/provider.py             | EthereumNetworkConfig   | APE_NODE_                 | yes  |
| src/ape_node/provider.py             | EthereumNodeConfig      | APE_NODE_                 | yes  |
| src/ape_test/config.py               | ApeTestConfig           | APE_TEST_                 | yes  |
| src/ape_test/config.py               | CoverageConfig          | APE_TEST_                 | yes  |
| src/ape_test/config.py               | CoverageReportsConfig   | APE_TEST_                 | yes  |
| src/ape_test/config.py               | EthTesterProviderConfig | APE_TEST_                 | yes  |
| src/ape_test/config.py               | GasConfig               | APE_TEST_                 | yes  |
| src/ape_test/config.py               | GasExclusion            | APE_TEST_                 | yes  |
| src/ape_test/config.py               | IsolationConfig         | APE_TEST_                 | yes  |
| tests/functional/test_config.py      | CustomConfig            | APE_API_ (inherited)      | -    |
| tests/functional/test_config.py      | CustomConfig            | APE_API_ (inherited)      | -    |
| tests/functional/test_config.py      | MyConfig                | APE_API_ (inherited)      | -    |
| tests/functional/test_config.py      | MyConfig                | APE_API_ (inherited)      | -    |
| tests/functional/test_config.py      | MyConfig                | APE_API_ (inherited)      | -    |
| tests/functional/test_config.py      | MyConfig                | APE_API_ (inherited)      | -    |
| tests/functional/test_config.py      | SubConfig               | APE_API_ (inherited)      | -    |
| tests/functional/test_config.py      | SubConfig               | APE_API_ (inherited)      | -    |
| tests/functional/test_ecosystem.py   | L2NetworkConfig         | APE_ETHEREUM_ (inherited) | -    |
| tests/functional/test_network_api.py | MyConfig                | APE_ETHEREUM_ (inherited) | -    |

Thus the previously used as an example TERMINAL environment variable now becomes APE_TEST_TERMINAL. The same applies respectively to all other settings as well.

This patch carries the risk of invalidating the settings of users who rely on environment variables based configuration (even though this seems highly unlikely), so make sure to include it in the changelog as a breaking change, should this patch get approved.

@ydm ydm changed the title Prefix config environment vars to avoid conflicts fix(bug): prefix config environment vars to avoid conflicts Jan 27, 2025
src/ape/api/config.py Outdated Show resolved Hide resolved
@antazoey
Copy link
Member

This feature is not documented anywhere, so we should probably add it to the config guide.

I kind of like it under the line:

This helps keep your secrets out of Ape!

Something like:

Similarly, any config key-name can also be set with the same named environment variable (with a prefix).
For example, to set the node URL using an environment variable, do:

```shell
export APE_NODE_URL="https://node.example.com"

@antazoey
Copy link
Member

So I tried to see if I could use APE_API_CONTRACTS_FOLDER but it didn't work.
I realize the problem was in the ApeConfg.__init__.

I had to change:

super(BaseSettings, self).__init__(*args, **kwargs)

to

super().__init__(*args, **kwargs)

At that point I can add a test:

def test_model_validate_handles_environment_variables():
    os.environ["APE_API_CONTRACTS_FOLDER"] = "contracts-env-var-test"
    cfg = ApeConfig()
    assert cfg.contracts_folder == "contracts-env-var-test"

@ydm
Copy link
Contributor Author

ydm commented Jan 27, 2025

OK, let me do that, will get back to you shortly.

Copy link
Member

@fubuloubu fubuloubu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had an issue planning to do this, thanks for taking it up!

Maintainer note: Breaking change for 0.9

src/ape/api/config.py Outdated Show resolved Hide resolved
src/ape/api/config.py Outdated Show resolved Hide resolved
src/ape/api/config.py Outdated Show resolved Hide resolved
@antazoey
Copy link
Member

Maintainer note: Breaking change for 0.9

I am arguing against this for a couple reasons.
1, issues the user mentioned are bad enough to change now. Like a mere TERMINAL environment variable causes problems
2, the feature is un-documented; i was not aware we were supporting environment variables like TERMINAL, etc.

@fubuloubu
Copy link
Member

Maintainer note: Breaking change for 0.9

I am arguing against this for a couple reasons. 1, issues the user mentioned are bad enough to change now. Like a mere TERMINAL environment variable causes problems 2, the feature is un-documented; i was not aware we were supporting environment variables like TERMINAL, etc.

Fair enough! Agree with releasing as a patch update

Documentation can be updated in a later PR

@ydm
Copy link
Contributor Author

ydm commented Jan 28, 2025

@antazoey , please check the latest version, it includes tests and docs.

Copy link
Member

@antazoey antazoey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing! Thank you so much. I think we can take it from here, just a few nits I can address and then merge.

src/ape_compile/main.py Outdated Show resolved Hide resolved
docs/userguides/config.md Outdated Show resolved Hide resolved
docs/userguides/config.md Outdated Show resolved Hide resolved
tests/functional/test_config.py Outdated Show resolved Hide resolved
antazoey
antazoey previously approved these changes Jan 28, 2025
antazoey
antazoey previously approved these changes Jan 28, 2025
@antazoey antazoey enabled auto-merge (squash) January 28, 2025 20:43
@antazoey antazoey merged commit 978c296 into ApeWorX:main Jan 28, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants