Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs for each ThunderbirdComponentResource to describe resources #104

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@

- Overhauled documentation.
- Only docstrings have changed with this version.
- No actual code changes have been altered.
- No actual code changes have been made.


## v0.0.2
Expand Down
13 changes: 11 additions & 2 deletions docs/getting-started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -239,11 +239,20 @@ So you want to develop a new pattern to stamp out? Here's what you'll need to do
tracked in the project (and other things).
* Any resources you create must have the ``parent=self`` ``pulumi.ResourceOption`` set. Set an appropriate
``depends_on`` value.
* At the end of the ``__init__`` function, you must call ``self.finish()``, passing in a dictionary of ``outputs``
and one of ``resources`` (see :py:meth:`tb_pulumi.ThunderbirdComponentResource.finish`). For
* At the end of the ``__init__`` function, you must call ``self.finish()``, passing in a dictionary of ``resources``
(see :py:meth:`tb_pulumi.ThunderbirdComponentResource.finish`). For
:py:class:`tb_pulumi.monitoring.MonitoringGroup` derivatives, call this at the end of the
:py:meth:`tb_pulumi.monitoring.MonitoringGroup.monitor` function instead.

The ``finish`` function also supports an ``outputs`` option, a dict containing outputs to register with Pulumi using
their ``register_outputs`` function. However,
`Pulumi's documentation <https://www.pulumi.com/docs/iac/concepts/resources/components/#registering-component-outputs>`_`
is unclear on the purpose of this, you cannot access these outputs programmatically, and the
`Pulumi developers also don't know <https://github.com/pulumi/pulumi/issues/2653#issuecomment-484956028>`_ why you
should call it. Its only purpose is within the CLI tool, as simple output at the end of the run. As such, we will stop
allowing this in a future version, opting to make the ``register_outputs`` call with an empty dict, as is common among
Pulumi developers.


Troubleshooting
---------------
Expand Down
24 changes: 17 additions & 7 deletions tb_pulumi/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@
#: Type alias representing valid types to be found among a ThunderbirdPulumiProject's resources
type Flattenable = dict | list | ThunderbirdComponentResource | pulumi.Output | pulumi.Resource

FINISH_OUTPUTS_DEPRECATION_MESSAGE = """Calling ThunderbirdComponentResource.finish with the "outputs" parameter is
DEPRECATED. This parameter will be removed in a future version."""


class ThunderbirdPulumiProject:
"""A collection of related Pulumi resources upon which we can take bulk/collective actions. This class enforces some
Expand Down Expand Up @@ -153,18 +156,25 @@ def __init__(

self.resources: dict = {} #: Resources which are members of this ComponentResource.

def finish(self, outputs: dict[str, Any], resources: dict[str, pulumi.Resource | list[pulumi.Resource]]):
"""Registers the provided ``outputs`` as Pulumi outputs for the module. Also stores the mapping of ``resources``
internally as the ``resources`` member where they can be acted on collectively by a
``ThunderbirdPulumiProject``. Any implementation of this class should call this function at the end of its
``__init__`` function to ensure its state is properly represented.
def finish(self, outputs: dict[str, Any] = {}, resources: dict[str, Flattenable] = {}):
"""Stores the mapping of ``resources`` internally as the ``resources`` member of this component resource's
``ThunderbirdPulumiProject``, where they can be acted on collectively. Any implementation of this class should
call this function at the end of its ``__init__`` function to ensure its state is properly represented.

Values in ``resources`` should be of a type compatible with the :py:data:`Flattenable` custom type.

:param outputs: Dict of outputs to register with Pulumi's ``register_outputs`` function. This parameter is
deprecated and will be removed in a future version. Defaults to {}.
:type outputs: dict[str, Any], optional

Values in ``resources`` should be either a Resource or derivative (such as a ThunderbirdComponentResource).
Alternatively, supply a list or dict of such.
:param resources: Dict of Pulumi resources this component reosurce contains. Defaults to {}.
:type resources: dict[str, Flattenable], optional
"""

# Register resources internally; register outputs with Pulumi
self.resources = resources
if len(outputs) > 0:
pulumi.warn(FINISH_OUTPUTS_DEPRECATION_MESSAGE)
self.register_outputs(outputs)

# Register resources within the project if not excluded
Expand Down
23 changes: 22 additions & 1 deletion tb_pulumi/ci.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,27 @@ class AwsAutomationUser(tb_pulumi.ThunderbirdComponentResource):
Because CI processes affect resources built across multiple environments (which can also be interpreted as
multiple Pulumi stacks), these items are only created in a single stack.

Produces the following ``resources``:

- *user* - `aws.iam.User <https://www.pulumi.com/registry/packages/aws/api-docs/iam/user/>`_ to run CI
operations.
- *access_key* - `aws.iam.AccessKey <https://www.pulumi.com/registry/packages/aws/api-docs/iam/accesskey/>`_ for
that user's authentication.
- *secret* - :py:class:`tb_pulumi.secrets.SecretsManagerSecret` where the access key data is stored.
- *ecr_image_push_policy* - `aws.iam.Policy <https://www.pulumi.com/registry/packages/aws/api-docs/iam/policy/>`_
defining permissions required to push container images to an ECR repository, but only if
``enable_ecr_image_push`` is ``True``.
- *s3_upload_policy* - `aws.iam.Policy <https://www.pulumi.com/registry/packages/aws/api-docs/iam/policy/>`_
defining permissions required to upload files to S3 buckets, but only if ``enable_s3_bucket_upload`` is
``True``.
- *s3_full_access_policy* - `aws.iam.Policy
<https://www.pulumi.com/registry/packages/aws/api-docs/iam/policy/>`_ defining complete, unfettered access to
S3 buckets and their contents, but only if ``enable_full_s3_access`` is ``True``.
- *fargate_deployment_policy* - `aws.iam.Policy
<https://www.pulumi.com/registry/packages/aws/api-docs/iam/policy/>`_ defining permissions needed to deploy
images to a Fargate service, but only if ``enable_fargate_deployments`` is ``True``.


:param name: Name of the IAM user to create.
:type name: str

Expand Down Expand Up @@ -70,6 +91,7 @@ class AwsAutomationUser(tb_pulumi.ThunderbirdComponentResource):

:param opts: Additional pulumi.ResourceOptions to apply to these resources. Defaults to None.
:type opts: pulumi.ResourceOptions, optional

"""

def __init__(
Expand Down Expand Up @@ -306,7 +328,6 @@ def __init__(
)

self.finish(
outputs={'user_name': user.name},
resources={
'user': user,
'access_key': access_key,
Expand Down
31 changes: 26 additions & 5 deletions tb_pulumi/cloudfront.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,32 @@
class CloudFrontS3Service(tb_pulumi.ThunderbirdComponentResource):
"""Serve the static contents of an S3 bucket over a CloudFront Distribution.

Produces the following ``resources``:

- *service_bucket* - `aws.s3.Bucket <https://www.pulumi.com/registry/packages/aws/api-docs/s3/bucket/>`_ in
which to store the static content to be served over the CloudFront Distribution.
- *logging_bucket* - `aws.s3.Bucket <https://www.pulumi.com/registry/packages/aws/api-docs/s3/bucket/>`_ in
which to store the access logs for the service bucket.
- *logging_bucket_ownership* - `aws.s3.BucketOwnershipControls
<https://www.pulumi.com/registry/packages/aws/api-docs/s3/bucketownershipcontrols/>`_ which allow CloudFront
to upload logs into the logging bucket.
- *logging_bucket_acl* - `aws.s3.BucketAclV2
<https://www.pulumi.com/registry/packages/aws/api-docs/s3/bucketaclv2/>`_ allowing CloudFront to control the
logging bucket via the AWS account's canonical user.
- *origin_access_control* - `aws.cloudfront.OriginAccessControl
<https://www.pulumi.com/registry/packages/aws/api-docs/cloudfront/originaccesscontrol/>`_ allowing the
CloudFront Distribution to serve the service bucket's content via CDN.
- *cloudfront_distribution* - `aws.cloudfront.Distribution
<https://www.pulumi.com/registry/packages/aws/api-docs/cloudfront/distribution/>`_ that serves the service
bucket content over a CDN and produces output logs in the logging bucket.
- *service_bucket_policy* - `aws.s3.BucketPolicy
<https://www.pulumi.com/registry/packages/aws/api-docs/s3/bucketpolicy/>`_ allowing the CloudFront
Distribution read access to the objects in the service bucket.
- *invalidation_policy* - `aws.iam.Policy <https://www.pulumi.com/registry/packages/aws/api-docs/iam/policy/>`_
that allows an IAM entity to create cache invalidations in the CloudFront Distribution, which must be done
when the contents of the service bucket are updated. This is not attached to any entities; it exists for
developer use when setting up CI flows.

:param name: A string identifying this set of resources.
:type name: str

Expand Down Expand Up @@ -233,11 +259,6 @@ def __init__(
)

self.finish(
outputs={
'service_bucket': service_bucket.id,
'logging_bucket': logging_bucket.id,
'cloudfront_domain': cloudfront_distribution.domain_name,
},
resources={
'service_bucket': service_bucket,
'logging_bucket': logging_bucket,
Expand Down
45 changes: 27 additions & 18 deletions tb_pulumi/cloudwatch.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,18 @@


class CloudWatchMonitoringGroup(tb_pulumi.monitoring.MonitoringGroup):
"""A ``MonitoringGroup`` that monitors AWS-based resources using AWS's CloudWatch service. This creates an SNS topic
to send its alarms on.
"""A ``MonitoringGroup`` that monitors AWS-based resources using AWS's CloudWatch service and sends alerts using
SNS-to-email.

Produces the following ``resources``:

- *sns_topic* - `aws.sns.Topic <https://www.pulumi.com/registry/packages/aws/api-docs/sns/topic/>`_ to notify
when an alarm in this monitoring group is triggered.
- *sns_subscriptions* - A list of `aws.sns.TopicSubscription
<https://www.pulumi.com/registry/packages/aws/api-docs/sns/topicsubscription/>`_s, one for each entry in
``notify_emails``.
- *alarms* - A list of alarms contained in this monitoring group. These may be any kind of alarm in the
`aws.cloudwatch <https://www.pulumi.com/registry/packages/aws/api-docs/cloudwatch/>`_ library.

:param name: The name of the ``CloudWatchMonitoringGroup`` resource.
:type name: str
Expand Down Expand Up @@ -101,7 +111,6 @@ def monitor(self, outputs):
)

self.finish(
outputs={'sns_topic_arn': sns_topic.arn},
resources={'sns_topic': sns_topic, 'sns_subscriptions': subscriptions, 'alarms': alarms},
)

Expand Down Expand Up @@ -164,15 +173,15 @@ class AlbAlarmGroup(tb_pulumi.monitoring.AlarmGroup):
"""A set of alarms for Application Load Balancers. Contains the following configurable alarms:

- ``alb_5xx``: Alarms on the number of HTTP responses sourced within the load balancer with status codes in the
500-599 range, indicating a count of internal server errors. This metric concerns the load balancer itself,
and does not contain any response codes generated by the targets. Defaults to alarming on 10 errors in 1
minute.
500-599 range, indicating a count of internal server errors. This metric concerns the load balancer itself,
and does not contain any response codes generated by the targets. Defaults to alarming on 10 errors in 1
minute.
- ``target_5xx``: Alarms on the number of HTTP responses sourced within the load balancer's targets with status
codes in the 500-599 range, indicating a count of internal server errors. This metric concerns the
applications the load balancer points to, and does not contain any response codes generated by the load
balancer itself. Defaults to alarming on 10 errors in 1 minute.
codes in the 500-599 range, indicating a count of internal server errors. This metric concerns the
applications the load balancer points to, and does not contain any response codes generated by the load
balancer itself. Defaults to alarming on 10 errors in 1 minute.
- ``response_time``: Alarms on the average response time of HTTP requests. Defaults to alarming if the average
response time over a minute is longer than 1 second.
response time over a minute is longer than 1 second.

Further detail on these metrics and others can be found within `Amazon's ALB metrics documentation
<https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-cloudwatch-metrics.html>`_.
Expand Down Expand Up @@ -306,7 +315,7 @@ class AlbTargetGroupAlarmGroup(tb_pulumi.monitoring.AlarmGroup):
"""A set of alarms for ALB target groups. Contains the following configurable alarms:

- ``unhealthy_hosts``: Alarms on the number of unhealthy hosts in a target group. Defaults to alarm when the
average of unhealthy hosts is over 1 in 1 minute.
average of unhealthy hosts is over 1 in 1 minute.

Further detail on these metrics and others can be found within `Amazon's Target Group metric documentation
<https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-cloudwatch-metrics.html#target-metric-table>`_.
Expand Down Expand Up @@ -422,8 +431,8 @@ class CloudFrontDistributionAlarmGroup(tb_pulumi.monitoring.AlarmGroup):
"""A set of alarms for CloudFront distributions. Contains the following configurable alarms:

- ``distro_4xx``: Alarms on the rate of HTTP responses with status codes in the 400-499 range, indicating an
elevated number of calls to invalid files. This value is reported as a percentage of all responses. Defaults
to alarm on at least 10% 4xx codes in 1 minute.
elevated number of calls to invalid files. This value is reported as a percentage of all responses. Defaults
to alarm on at least 10% 4xx codes in 1 minute.

Further details about these metrics and more can be found in `Amazon's CloudFront distribution documentation
<https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/programming-cloudwatch-metrics.html#cloudfront-metrics-distribution-values>`_.
Expand Down Expand Up @@ -498,9 +507,9 @@ def __init__(
class CloudFrontFunctionAlarmGroup(tb_pulumi.monitoring.AlarmGroup):
"""A set of alarms for CloudFront functions. Contains the following configurable alarms:

- ``cpu_utilization``: Alarms when the function's compute utilization is excessive. This value is reported as a
percentage of its allotted CPU. Defaults to alarm when the function's CPU usage has been over 80% on average for
1 minute.
- ``cpu_utilization``: Alarms when the function's compute utilization is excessive. This value is reported as a
percentage of its allotted CPU. Defaults to alarm when the function's CPU usage has been over 80% on average
for 1 minute.

Further details about these metrics and more can be found in `Amazon's CloudFront Functions documentation
<https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/programming-cloudwatch-metrics.html#cloudfront-metrics-function-values>`_.
Expand Down Expand Up @@ -578,9 +587,9 @@ class EcsServiceAlarmGroup(tb_pulumi.monitoring.AlarmGroup):
"""A set of alarms for ECS services. Contains the following configurable alarms:

- ``cpu_utilization``: Alarms on the overall CPU usage of the entire service, all tasks combined. Threshold is a
percentage.
percentage.
- ``memory_utilization``: Alarms on the overall memory usage of the entire service, all tasks combined.
Threshold is a percentage.
Threshold is a percentage.

Further detail on these metrics and more can be found on `Amazon's documentation
<https://docs.aws.amazon.com/AmazonECS/latest/developerguide/available-metrics.html>_`.
Expand Down
35 changes: 28 additions & 7 deletions tb_pulumi/ec2.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,20 @@ class NetworkLoadBalancer(tb_pulumi.ThunderbirdComponentResource):
"""Construct a NetworkLoadBalancer to route TCP traffic to a collection of backends. This targets backend services
by IP address, connecting a frontend listening port to a backend port on the round-robin load balanced targets.

Produces the following ``resources``:

- *security_group_with_rules* - :py:class:`tb_pulumi.network.SecurityGroupWithRules` defining ingress and egress
rules for the NLB.
- *nlb* - `aws.lb.LoadBalancer <https://www.pulumi.com/registry/packages/aws/api-docs/lb/loadbalancer/>`_ with a
``load_balancer_type`` of ``network``.
- *target_group* - `aws.lb.TargetGroup <https://www.pulumi.com/registry/packages/aws/api-docs/lb/targetgroup/>`_
containing the IPs the NLB is balancing.
- *target_group_attachments* - List of `aws.lb.TargetGroupAttachments
<https://www.pulumi.com/registry/packages/aws/api-docs/lb/targetgroupattachment/>`_, one for each IP address
registered with the NLB.
- *listener* - `aws.lb.Listener <https://www.pulumi.com/registry/packages/aws/api-docs/lb/listener/>`_ for the
NLB.

:param name: A string identifying this set of resources.
:type name: str

Expand Down Expand Up @@ -178,7 +192,6 @@ def __init__(
)

self.finish(
outputs={'dns_name': nlb.dns_name},
resources={
'security_group_with_rules': security_group_with_rules,
'nlb': nlb,
Expand All @@ -192,6 +205,13 @@ def __init__(
class SshableInstance(tb_pulumi.ThunderbirdComponentResource):
"""Builds an EC2 instance which can be accessed with SSH from somewhere on the Internet.

Produces the following ``resources``:

- *instance* - The `aws.ec2.Instance <https://www.pulumi.com/registry/packages/aws/api-docs/ec2/instance/>`_.
- *keypair* - :py:class:`tb_pulumi.ec2.SshKeyPair` used for authenticating to the instance.
- *security_group* - :py:class:`tb_pulumi.network.SecurityGroupWithRules` defining network access to the
instance.

:param name: A string identifying this set of resources.
:type name: str

Expand Down Expand Up @@ -302,10 +322,6 @@ def __init__(
)

self.finish(
outputs={
'instance_dns': instance.public_dns,
'instance_ip': instance.public_ip,
},
resources={
'instance': instance,
'keypair': keypair,
Expand All @@ -322,6 +338,13 @@ class SshKeyPair(tb_pulumi.ThunderbirdComponentResource):
ONE TIME. Each ``pulumi up/preview`` command generates a new keypair, which generates new secret versions (and if
this is attached to an instance downstream, it triggers the recreation of that instance).

Produces the following ``resources``:

- *keypair* - `aws.ec2.KeyPair <https://www.pulumi.com/registry/packages/aws/api-docs/ec2/keypair/>`_ containing
the keypair content.
- *private_key_secret* :py:class:`tb_pulumi.secrets.SecretsManagerSecret` containing the private key data.
- *public_key_secret* :py:class:`tb_pulumi.secrets.SecretsManagerSecret` containing the public key data.

:param name: A string identifying this set of resources.
:type name: str

Expand Down Expand Up @@ -404,9 +427,7 @@ def __init__(
)

self.finish(
outputs={'keypair': keypair.id},
resources={
'private_key': private_key if not public_key else None,
'keypair': keypair,
'private_key_secret': private_key_secret if not public_key else None,
'public_key_secret': public_key_secret if not public_key else None,
Expand Down
Loading