Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extra vars poc #203

Merged
merged 18 commits into from
May 23, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.

## [Unreleased]

## [1.2.3] - 2024-04-30

### Changed
- Enhancement for the lookup plugin to support the extra-vars
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should count as a backwards-compatible addition

Suggested change
## [1.2.3] - 2024-04-30
### Changed
- Enhancement for the lookup plugin to support the extra-vars
## [1.3.0] - 2024-04-30
### Added
- Enhancement for the lookup plugin to support the extra-vars

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you elaborate more on that John. should this be minor change or description must be updated?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bumping the version from 1.2.2 to 1.2.3 would imply that the only released additions were bug fixes, but this goes a bit beyond that. We're adding a new method of configuring the plugin, so as long as we maintain backwards compatibility, we should bump the version to 1.3.0 instead.


## [1.2.2] - 2023-09-28

### Changed
Expand Down
139 changes: 112 additions & 27 deletions plugins/lookup/conjur_variable.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,10 @@
author:
- CyberArk BizDev (@cyberark-bizdev)
description:
Retrieves credentials from Conjur using the controlling host's Conjur identity
or environment variables.
Retrieves credentials from Conjur using the controlling host's Conjur identity,
environment variables, or extra-vars.
Environment variables could be CONJUR_ACCOUNT, CONJUR_APPLIANCE_URL, CONJUR_CERT_FILE, CONJUR_AUTHN_LOGIN, CONJUR_AUTHN_API_KEY, CONJUR_AUTHN_TOKEN_FILE
Extra-vars could be conjur_account, conjur_appliance_url, conjur_cert_file, conjur_authn_login, conjur_authn_api_key, conjur_authn_token_file
Conjur info - U(https://www.conjur.org/).
requirements:
- 'The controlling host running Ansible has a Conjur identity.
Expand Down Expand Up @@ -46,16 +47,6 @@
key: identity_file_path
env:
- name: CONJUR_IDENTITY_FILE
authn_token_file:
description: Path to the access token file.
type: path
default: /var/run/conjur/access-token
required: False
ini:
- section: conjur,
key: authn_token_file
env:
- name: CONJUR_AUTHN_TOKEN_FILE
config_file:
description: Path to the Conjur configuration file. The configuration file is a YAML file.
type: path
Expand All @@ -66,6 +57,72 @@
key: config_file_path
env:
- name: CONJUR_CONFIG_FILE
conjur_appliance_url:
description: Conjur appliance url
type: string
required: False
ini:
- section: conjur,
key: conjur_appliance_url
john-odonnell marked this conversation as resolved.
Show resolved Hide resolved
vars:
- name: conjur_appliance_url
env:
- name: CONJUR_APPLIANCE_URL
conjur_authn_login:
description: Conjur authn login
type: string
required: False
ini:
- section: conjur,
key: conjur_authn_login
vars:
- name: conjur_authn_login
env:
- name: CONJUR_AUTHN_LOGIN
conjur_account:
description: Conjur account
type: string
required: False
ini:
- section: conjur,
key: conjur_account
vars:
- name: conjur_account
env:
- name: CONJUR_ACCOUNT
conjur_authn_api_key:
description: Conjur authn api key
type: string
required: False
ini:
- section: conjur,
key: conjur_authn_api_key
vars:
- name: conjur_authn_api_key
env:
- name: CONJUR_AUTHN_API_KEY
conjur_cert_file:
description: Path to the Conjur cert file
type: path
required: False
ini:
- section: conjur,
key: conjur_cert_file
vars:
- name: conjur_cert_file
env:
- name: CONJUR_CERT_FILE
conjur_authn_token_file:
description: Path to the access token file
type: path
required: False
ini:
- section: conjur,
key: conjur_authn_token_file
vars:
- name: conjur_authn_token_file
env:
- name: CONJUR_AUTHN_TOKEN_FILE
"""

EXAMPLES = """
Expand Down Expand Up @@ -263,43 +320,60 @@ def _store_secret_in_file(value):

return [secrets_file.name]


class LookupModule(LookupBase):

def run(self, terms, variables=None, **kwargs):
if terms == []:
raise AnsibleError("Invalid secret path: no secret path provided.")
elif not terms[0] or terms[0].isspace():
raise AnsibleError("Invalid secret path: empty secret path not accepted.")

# We should register the variables as LookupModule options.
#
# Doing this has some nice advantages if we're considering supporting
# a set of Ansible variables that could sometimes replace environment
# variables.
#
# Registering the variables as options forces them to adhere to the
# behavior described in the DOCUMENTATION variable. An option can have
# both a Ansible variable and environment variable source, which means
# Ansible will do some juggling on our behalf.
self.set_options(var_options=variables, direct=kwargs)

appliance_url = self.get_var_value("conjur_appliance_url")
account = self.get_var_value("conjur_account")
authn_login = self.get_var_value("conjur_authn_login")
authn_api_key = self.get_var_value("conjur_authn_api_key")
cert_file = self.get_var_value("conjur_cert_file")
authn_token_file = self.get_var_value("conjur_authn_token_file")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have some unexpected behavior here that breaks backwards compatibility.

Previously, we had a section in the DOCUMENTATION variable that described the conjur_authn_token_file variable, but it was being accessed exclusively as an environment variable, so the specified default value was not being applied. Now that we're accessing the value this way, the default value is being applied when we would expect it not to.

We either need to add some special handling for this variable to maintain backwards compatibility, or release this as part of a major version bump.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the past, the authn_token_file value was generated by either an environmental variable or the default path (/var/run/conjur/access-token).

The newest modifications allow for the authn_token_file value to be pulled from either the environmental variable, extra variables, or the default path (/var/run/conjur/access-token).

The behavior was the same as earlier, but we have updated only the ini name of authn_token_file.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the past, the authn_token_file value was generated by either an environmental variable or the default path (/var/run/conjur/access-token)

The last part of this isn't true - previously, the lookup plugin checked the CONJUR_AUTHN_TOKEN_FILE environment variable, and if didn't exist, no default value was applied, and instead the plugin retrieved a new authn token.

Check out this PR's Jenkins build, which is failing on the lookup plugin's end-to-end tests.

TASK [Retrieve Conjur variable] ************************************************
fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while templating
  '{{lookup('conjur_variable', 'ansible/test-secret')}}'. Error was a
  <class 'ansible.errors.AnsibleError'>, original message: An unhandled exception
  occurred while running the lookup plugin 'conjur_variable'. Error was a
  <class 'ansible.errors.AnsibleError'>, original message: Conjur authn token
  file `/var/run/conjur/access-token` was not found on the host. Conjur authn token
  file `/var/run/conjur/access-token` was not found on the host"}

The existing DOCUMENTATION variable section describing the authn_token_file parameter is inaccurate, but we aren't using self.get_option, so it isn't being enforced. Backwards compatibility should be maintained if we updated the new authn_token_file section to specify that it doesn't have a default value, and isn't required.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To ensure backward compatibility for conjur_authn_token_file. We have removed the default key parameter in the conjur_authn_token_file documentation section and also added an extra check for conjur_authn_token_file.

if not variable_value and key != "conjur_authn_token_file":
raise AnsibleError("The value of the {0} variable is not set".format(key))

The token can either be selected from environmental/extra-vars variables or generated within the code.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


self.set_options(direct=kwargs)
validate_certs = self.get_option('validate_certs')
conf_file = self.get_option('config_file')
as_file = self.get_option('as_file')

if validate_certs is False:
display.warning('Certificate validation has been disabled. Please enable with validate_certs option.')

if 'http://' in str(environ.get("CONJUR_APPLIANCE_URL")):
if 'http://' in str(appliance_url):
raise AnsibleError(('[WARNING]: Conjur URL uses insecure connection. Please consider using HTTPS.'))

conf = _merge_dictionaries(
_load_conf_from_file(conf_file),
{
"account": environ.get('CONJUR_ACCOUNT'),
"appliance_url": environ.get("CONJUR_APPLIANCE_URL")
"account": account,
"appliance_url": appliance_url
} if (
environ.get('CONJUR_ACCOUNT') is not None
and environ.get('CONJUR_APPLIANCE_URL') is not None
account is not None
and appliance_url is not None
)
else {},
{
"cert_file": environ.get('CONJUR_CERT_FILE')
} if (environ.get('CONJUR_CERT_FILE') is not None)
"cert_file": cert_file
} if (cert_file is not None)
else {},
{
"authn_token_file": environ.get('CONJUR_AUTHN_TOKEN_FILE')
} if (environ.get('CONJUR_AUTHN_TOKEN_FILE') is not None)
"authn_token_file": authn_token_file
} if authn_token_file is not None
else {}
)

Expand All @@ -308,10 +382,10 @@ def run(self, terms, variables=None, **kwargs):
identity = _merge_dictionaries(
_load_identity_from_file(identity_file, conf['appliance_url']),
{
"id": environ.get('CONJUR_AUTHN_LOGIN'),
"api_key": environ.get('CONJUR_AUTHN_API_KEY')
} if (environ.get('CONJUR_AUTHN_LOGIN') is not None
and environ.get('CONJUR_AUTHN_API_KEY') is not None)
"id": authn_login,
"api_key": authn_api_key
} if authn_login is not None
and authn_api_key is not None
else {}
)

Expand Down Expand Up @@ -364,3 +438,14 @@ def run(self, terms, variables=None, **kwargs):
return _store_secret_in_file(conjur_variable)

return conjur_variable

def get_var_value(self, key):
try:
variable_value = self.get_option(key)
except KeyError:
raise AnsibleError("{0} was not defined in configuration".format(key))

if not variable_value and key != "conjur_authn_token_file":
raise AnsibleError("The value of the {0} variable is not set".format(key))
john-odonnell marked this conversation as resolved.
Show resolved Hide resolved

return variable_value
3 changes: 2 additions & 1 deletion tests/conjur_variable/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ function run_test_case {
fi

# You can add -vvvvv here for debugging
ansible-playbook 'test_cases/${test_case}/playbook.yml'
export SAMPLE_KEY='set_in_env'
ansible-playbook --extra-vars 'sample_key=set_in_extravars' 'test_cases/${test_case}/playbook.yml'
itsbrugu marked this conversation as resolved.
Show resolved Hide resolved

py.test --junitxml='./junit/${test_case}' \
--connection docker \
Expand Down
Loading