0.66 - unit test fixes

.gitignore

@@ -1,3 +1,4 @@
+*.log
 autotools/install-sh
 autotools/missing
 *.tar.gz
@@ -6,7 +7,6 @@ config.status
 aclocal.m4
 autom4te.cache/*
 configure
-config.log
 **/Makefile.in
 Makefile
 src/main/perl/lib/**/*.pm

ChangeLog

@@ -1,3 +1,19 @@
+Mon Jun 10 07:30:59 2024  Rob Lauer  <[email protected]>
+
+	[0.66]:
+	* NEWS.md: updated
+	* .gitignore: *.log
+	* src/main/perl/run-test: new from test.localstack
+	* src/main/perl/S3TestUtils.pm
+	(set_s3_host): AMAZON_S3_SKIP_ACLS, not SKIP_PERMISSIONS
+	* src/main/perl/t/01-api.t
+	- likewise
+	(like_acl_allrusers_read):
+	- dump flag
+	* src/main/perl/t/04-list-buckets.t: whitespace
+	* src/main/perl/t/06-list-multipart-uploads.t
+	- ignore case on status header (LocalStack <> AWS)
+
 Thu Nov 30 13:56:08 2023  Rob Lauer  <[email protected]>
 
 	[0.66]:

NEWS.md

@@ -24,19 +24,31 @@ noted here, as well.
 > See the documentation for `Amazon::S3`, `Amazon::S3::Bucket` and
 > `Amazon::S3::V2` for more details.
 
-# perl-Amazon-S3 0.66 (2023-12-09)
+# perl-Amazon-S3 0.66 (2024-06-10)
 
-> This version introduces new methods for handling object versions.
+> This version introduces new methods for handling object version and
+> introduces some new methods that more accurately track the S3
+> API. Those new methods are experimental and will be released as
+> version 2.
 
 ## Enhancements
 
 * `Amazon::S3::Bucket`
   * `delete_key()` now accepts an optional version identifier
   * `get_bucket_versioning()` - returns status of bucket versioning
   * `put_bucket_versioning()` - sets the status of bucket versioning
-
 * `Amazon:S3`
   * `list_object_versions()` - method that returns version metadata
+* start of version 2 changes
+* refactoring
+* allow additional headers for several existing methods
+* pod fixes & updates
+
+## Fixes
+
+* unit tests fixes
+  * LocalStack fixes
+  * refactored several tests
 
 # perl-Amazon-S3 0.65 (2023-11-28)
 

README.md

@@ -87,15 +87,26 @@ corrected. Credentials are encrypted if you have encryption modules installed.
 _NEW!_
 
 The `Amazon::S3` modules have been heavily refactored over the last
-year to increase maintainability and to add new features. New features include:
+few releases to increase maintainability and to add new features. New
+features include:
 
-- Addition of version parameter for some methods that can accept
-an object version.
-    - delete\_key
-    - delete\_keys
-    - set\_acl
+- [Amazon::S3::BucketV2](https://metacpan.org/pod/Amazon::S3::BucketV2)
+
+    This new module implements a mechanism to invoke _almost_ all of the
+    S3 APIs using a standard calling method.
+
+    The module will format your Perl objects as XML payloads and enable
+    you to provide all of the parameters required to make an API
+    call. Headers and URI parameters can also be passed to the
+    methods. [Amazon::S3::BucketV2](https://metacpan.org/pod/Amazon::S3::BucketV2) is a subclass of
+    [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon::S3::Bucket), meaning you can still invoke all of the same
+    methods found there.
+
+    See [Amazon::S3::BucketV2](https://metacpan.org/pod/Amazon::S3::BucketV2) for more details.
+
+- Addition of version parameter for `delete_key`
 - Methods that accept a hash reference can now accept a
-`headers` object that contains any additional headers you might want
+`headers` object that may contain any additional headers you might want
 to send with a request. Some of the methods that now allow you to pass
 a header object include:
     - add\_bucket
@@ -153,11 +164,11 @@ dependencies. Below is the original description of the module.
 > The need for this module arose from some work that needed
 > to work with S3 and would be distributed, installed and used
 > on many various environments where compiled dependencies may
-> not be an option. [Net::Amazon::S3](https://metacpan.org/pod/Net%3A%3AAmazon%3A%3AS3) used [XML::LibXML](https://metacpan.org/pod/XML%3A%3ALibXML)
+> not be an option. [Net::Amazon::S3](https://metacpan.org/pod/Net::Amazon::S3) used [XML::LibXML](https://metacpan.org/pod/XML::LibXML)
 > tying it to that specific and often difficult to install
 > option. In order to remove this potential barrier to entry,
-> this module is forked and then modified to use [XML::SAX](https://metacpan.org/pod/XML%3A%3ASAX)
-> via [XML::Simple](https://metacpan.org/pod/XML%3A%3ASimple).
+> this module is forked and then modified to use [XML::SAX](https://metacpan.org/pod/XML::SAX)
+> via [XML::Simple](https://metacpan.org/pod/XML::Simple).
 
 # LIMITATIONS AND DIFFERENCES WITH EARLIER VERSIONS
 
@@ -361,7 +372,7 @@ Create a new S3 client object. Takes some arguments:
 
 Sets or retrieves the signer object. API calls must be signed using
 your AWS credentials. By default, starting with version 0.54 the
-module will use [Net::Amazon::Signature::V4](https://metacpan.org/pod/Net%3A%3AAmazon%3A%3ASignature%3A%3AV4) as the signer and
+module will use [Net::Amazon::Signature::V4](https://metacpan.org/pod/Net::Amazon::Signature::V4) as the signer and
 instantiate a signer object in the constructor. Note however, that
 signers need your credentials and they _will_ get stored by that
 class, making them susceptible to inadvertant exfiltration. You have a
@@ -389,12 +400,12 @@ false value.
 
 - 3. Pass credentials, set `cache_signer` to a false value.
 
-    Unfortunately, while this will prevent [Net::Amazon::Signature::V4](https://metacpan.org/pod/Net%3A%3AAmazon%3A%3ASignature%3A%3AV4)
+    Unfortunately, while this will prevent [Net::Amazon::Signature::V4](https://metacpan.org/pod/Net::Amazon::Signature::V4)
     from hanging on to your credentials, you credentials will be stored in
     the `Amazon::S3` object.
 
     Starting with version 0.55 of this module, if you have installed
-    [Crypt::CBC](https://metacpan.org/pod/Crypt%3A%3ACBC) and [Crypt::Blowfish](https://metacpan.org/pod/Crypt%3A%3ABlowfish), your credentials will be
+    [Crypt::CBC](https://metacpan.org/pod/Crypt::CBC) and [Crypt::Blowfish](https://metacpan.org/pod/Crypt::Blowfish), your credentials will be
     encrypted using a random key created when the class is
     instantiated. While this is more secure than leaving them in
     plaintext, if the key is discovered (the key however is not stored in
@@ -410,9 +421,9 @@ false value.
 - 5. Do nothing...send the credentials, use the default signer.
 
     In this case, both the `Amazon::S3` class and the
-    [Net::Amazon::Signature::V4](https://metacpan.org/pod/Net%3A%3AAmazon%3A%3ASignature%3A%3AV4) have your credentials. Caveat Emptor.
+    [Net::Amazon::Signature::V4](https://metacpan.org/pod/Net::Amazon::Signature::V4) have your credentials. Caveat Emptor.
 
-    See also [Amazon::Credentials](https://metacpan.org/pod/Amazon%3A%3ACredentials) for more information about safely
+    See also [Amazon::Credentials](https://metacpan.org/pod/Amazon::Credentials) for more information about safely
     storing your credentials and preventing exfiltration.
 
 ## region
@@ -461,7 +472,7 @@ buckets owned by the accout or (see below) or `undef` on error.
 
 - buckets
 
-    An array of [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) objects for the account. Returns
+    An array of [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon::S3::Bucket) objects for the account. Returns
     `undef` if there are not buckets or an error occurs.
 
 ## add\_bucket
@@ -504,7 +515,7 @@ See ["delete\_public\_access\_block"](#delete_public_access_block).
 
     Additional headers to send with request.
 
-Returns a [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) object on success or `undef` on failure.
+Returns a [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon::S3::Bucket) object on success or `undef` on failure.
 
 ## bucket
 
@@ -521,12 +532,12 @@ If you do not pass the region or set the `verify_region` value, the
 region will be set to the default region set in your `Amazon::S3`
 object.
 
-See [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) for a complete description of the `bucket`
+See [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon::S3::Bucket) for a complete description of the `bucket`
 method.
 
 ## delete\_bucket
 
-Takes either a [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) object or a reference to a hash
+Takes either a [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon::S3::Bucket) object or a reference to a hash
 containing:
 
 - bucket
@@ -852,11 +863,11 @@ The decoded XML string as a hash object of the last error.
 
 ## last\_response
 
-Returns the last [HTTP::Response](https://metacpan.org/pod/HTTP%3A%3AResponse) object.
+Returns the last [HTTP::Response](https://metacpan.org/pod/HTTP::Response) object.
 
 ## last\_request
 
-Returns the last [HTTP::Request](https://metacpan.org/pod/HTTP%3A%3ARequest) object.
+Returns the last [HTTP::Request](https://metacpan.org/pod/HTTP::Request) object.
 
 ## level
 
@@ -1043,11 +1054,11 @@ Current maintainer: Rob Lauer <[email protected]>
 
 # SEE ALSO
 
-[Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket), [Net::Amazon::S3](https://metacpan.org/pod/Net%3A%3AAmazon%3A%3AS3)
+[Amazon::S3::Bucket](https://metacpan.org/pod/Amazon::S3::Bucket), [Net::Amazon::S3](https://metacpan.org/pod/Net::Amazon::S3)
 
 # COPYRIGHT AND LICENCE
 
-This module was initially based on [Net::Amazon::S3](https://metacpan.org/pod/Net%3A%3AAmazon%3A%3AS3) 0.41, by
+This module was initially based on [Net::Amazon::S3](https://metacpan.org/pod/Net::Amazon::S3) 0.41, by
 Leon Brocard. Net::Amazon::S3 was based on example code from
 Amazon with this notice:
 

src/main/perl/S3TestUtils.pm

@@ -4,7 +4,7 @@ use strict;
 use warnings;
 
 use Data::Dumper;
-use English    qw(-no_match_vars);
+use English qw(-no_match_vars);
 use List::Util qw(any);
 use Readonly;
 use Test::More;
@@ -112,9 +112,7 @@ sub check_test_bucket {
   my $response = eval { return $s3->buckets; };
 
   if ( $EVAL_ERROR || !$response ) {
-    diag(
-      Dumper( [ error => [ $response, $s3->err, $s3->errstr, $s3->error ] ] )
-    );
+    diag( Dumper( [ error => [ $response, $s3->err, $s3->errstr, $s3->error ] ] ) );
 
     BAIL_OUT($EVAL_ERROR);
   }
@@ -142,17 +140,17 @@ sub set_s3_host {
 
   ## no critic (RequireLocalizedPunctuationVars)
 
-  if ( exists $ENV{AMAZON_S3_LOCALSTACK} ) {
+  if ( $ENV{AMAZON_S3_LOCALSTACK} ) {
 
     $host //= $DEFAULT_LOCAL_STACK_HOST;
 
     $ENV{AWS_ACCESS_KEY_ID} = 'test';
 
     $ENV{AWS_SECRET_ACCESS_KEY} = 'test';
 
-    $ENV{AMAZON_S3_EXPENSIVE_TESTS} = $TRUE;
+    $ENV{AMAZON_S3_SKIP_ACLS} = $TRUE;
 
-    $ENV{AMAZON_S3_SKIP_PERMISSIONS} = $TRUE;
+    $ENV{AMAZON_S3_EXPENSIVE_TESTS} = $TRUE;
   }
   elsif ( exists $ENV{AMAZON_S3_MINIO} ) {
 

src/main/perl/run-test

@@ -0,0 +1,31 @@
+#!/bin/bash
+# -*- mode: sh; -*-
+
+service="$1"
+service=${service:-localstack}
+
+if [[ "$service" = "localstack" ]]; then
+    export AMAZON_S3_HOST=s3.localhost.localstack.cloud:4566
+    export ENDPOINT_URL=http://$AMAZON_S3_HOST
+    export AMAZON_S3_LOCALSTACK=1
+elif [[ "$service" = "aws" ]]; then
+    if test -z "$AWS_ACCESS_KEY_ID"; then
+        echo "set AWS_ACCESS_KEY_ID before running test"
+        exit 1;
+    fi
+
+    if test -z "$AWS_SECRET_ACCESS_KEY"; then
+        echo "set AWS_SECRET_ACCESS_KEY before running test"
+        exit 1;
+    fi   
+fi
+
+export BUCKET=net-amazon-s3-test-$AWS_ACCESS_KEY_ID
+export AMAZON_S3_DOMAIN_BUCKET_NAMES=1
+
+if aws s3api --endpoint-url=$ENDPOINT_URL list-buckets  | grep "$BUCKET"; then
+    aws s3 --endpoint-url=$ENDPOINT_URL --recursive rm s3://$BUCKET/
+    aws s3 --endpoint-url=$ENDPOINT_URL rb s3://$BUCKET
+fi
+
+prove -I lib -v t/