Decouple policy logic from resource url for getSignedUrlWithCustomPolicy

[RanVaknin]

Motivation and Context

Fixes #5577 where users cannot specify a wildcard (*) resource URL policy separate from the actual URL to be signed
Current implementation couples the resource URL for both signing and policy specification
This change allows for more flexible URL signing scenarios.

Modifications

Added new resourceUrlPattern field to CustomSignerRequest
Updated CloudFrontUtilities.getSignedUrlWithCustomPolicy to use resourceUrlPattern when specified

Testing

Added a unit test to verify resourceUrlPattern applies the correct logic to the policy.
Added integration tests to verify real-world scenarios: (Passing on intg test acct ✅ )

• Testing with query parameters
• Testing with wildcard paths
• Testing with full wildcard policy

[joviegas]

Can we add a parameterized test to test different possibilities of resourceUrlPattern ?
Something like

@ParameterizedTest
@MethodSource("provideUrlPatternsAndExpectedResources")
void getSignedURLWithCustomPolicy_policyResourceUrlShouldHandleVariousPatterns(
        String resourceUrlPattern, String expectedResource) {
    String baseUrl = "https://d1234.cloudfront.net/images/photo.jpg";
    Instant expiration = Instant.now().plusSeconds(3600);
    CustomSignerRequest request = CustomSignerRequest.builder()
            .resourceUrl(baseUrl)
            .privateKey(keyPair.getPrivate())
            .keyPairId("keyPairId")
            .resourceUrlPattern(resourceUrlPattern)
            .expirationDate(expiration)
            .build();

    SignedUrl signedUrl = cloudFrontUtilities.getSignedUrlWithCustomPolicy(request);

    // Extract and decode the policy
    String encodedPolicy = signedUrl.url().split("Policy=")[1].split("&")[0];
    String standardBase64 = encodedPolicy
            .replace('-', '+')
            .replace('_', '=')
            .replace('~', '/');
    String decodedPolicy = new String(Base64.getDecoder().decode(standardBase64), UTF_8);
    
    // Build expected policy
    StringBuilder expectedPolicy = new StringBuilder();
    StringJoiner conditions = new StringJoiner(",", "{", "}");
    conditions.add("\"DateLessThan\":{\"AWS:EpochTime\":" + expiration.getEpochSecond() + "}");
    
    expectedPolicy.append("{\"Statement\": [{")
            .append("\"Resource\":\"").append(expectedResource).append("\",")
            .append("\"Condition\":").append(conditions)
            .append("}]}");

    assertThat(decodedPolicy.trim()).isEqualTo(expectedPolicy.toString().trim());
    
    // Verify URL is properly encoded
    assertThat(signedUrl.url()).doesNotContain(" ");
    assertThat(signedUrl.url()).matches("^[\\x20-\\x7E]+$"); // Printable ASCII characters only
}

private static Stream<Arguments> provideUrlPatternsAndExpectedResources() {
    return Stream.of(
        // Basic patterns
        Arguments.of("*", "*"),
        Arguments.of("https://d1234.cloudfront.net/*", "https://d1234.cloudfront.net/*"),
        Arguments.of("https://d1234.cloudfront.net/images/*", "https://d1234.cloudfront.net/images/*"),
        
        // Specific file patterns
        Arguments.of("234.cloudfront.net/images/photo.jpg", 
                    "https://d1234.cloudfront.net/images/photo.jpg"),
        Arguments.of("https://d1234.cloudfront.net/*/photo.jpg", 
                    "234.cloudfront.net/*/photo.jpg"),
        
        // Multiple wildcards
        Arguments.of("https://*.cloudfront.net/*/photo.jpg", 
                    "https://*.cloudfront.net/*/photo.jpg"),
        
        // Patterns with special characters that need encoding
        Arguments.of("https://d1234.cloudfront.net/images/photo with spaces.jpg", 
                    "https://d1234.cloudfront.net/images/photo%20with%20spaces.jpg"),
        Arguments.of("https://d1234.cloudfront.net/images/photo+with+plus.jpg", 
                    "234.cloudfront.net/images/photo%2Bwith%2Bplus.jpg"),
        Arguments.of("https://d1234.cloudfront.net/images/photo?param=value", 
                    "234.cloudfront.net/images/photo%3Fparam%3Dvalue"),
        
        // Unicode characters
        Arguments.of("https://d1234.cloudfront.net/images/photo-ümlaut.jpg", 
                    "https://d1234.cloudfront.net/images/photo-%C3%BCmlaut.jpg"),
        
        // Path traversal patterns
        Arguments.of("https://d1234.cloudfront.net/images/../photo.jpg", 
                    "https://d1234.cloudfront.net/images/../photo.jpg"),
        
        // Patterns with query parameters
        Arguments.of("https://d1234.cloudfront.net/images/photo.jpg?size=large&format=png", 
                    "https://d1234.cloudfront.net/images/photo.jpg%3Fsize%3Dlarge%26format%3Dpng")
   

[RanVaknin]

The parameterized test suite is testing for URL encoding transformations that don't exist in the code. In getSignedUrlWithCustomPolicy, the resource pattern is inserted directly into the policy document as a JSON string without modification, and URL-safe encoding (makeStringUrlSafe) happens later to the entire Base64-encoded policy. The test cases expect the Resource field in the policy to contain URL-encoded values (like %20 for spaces), but this is incorrect since the policy is a JSON document, not a URL, and these transformations never occur in the codepath.

For this reason, in this test suite's case, both the Input and Output arguments will be the same every time.

[joviegas]

Thanks Ran
I saw that some how the current implementation on resourceUrl does not handle special cases like below

In CloudFrontUtilitiesIntegrationTest I updated S3_OBJECT_KEY_ON_SUB_PATH to have special charatcters and also updated getSignedUrlWithCustomPolicy_wildCardPath test to cehck for specific url

    private static final String S3_OBJECT_KEY_ON_SUB_PATH = "foo/specific space+plus-üspecial";


    @Test
    void getSignedUrlWithCustomPolicy_wildCardPath() throws Exception {

        System.out.println("bucket® "+bucket);
        String resourceUri = "https://" + domainName;
        Instant expirationDate = LocalDate.of(2050, 1, 1)
                                          .atStartOfDay()
                                          .toInstant(ZoneOffset.of("Z"));

        Instant activeDate = LocalDate.of(2022, 1, 1)
                                      .atStartOfDay()
                                      .toInstant(ZoneOffset.of("Z"));

        CustomSignerRequest request = CustomSignerRequest.builder()
                                                         .resourceUrl(resourceUri + "/foo/specific space+plus-üspecial")
                                                         .privateKey(keyFilePath)
                                                         .keyPairId(keyPairId)
                                                         .policyResource(resourceUri + "/foo/specific space+plus-üspecial")
                                                         .activeDate(activeDate)
                                                         .expirationDate(expirationDate)
                                                         .build();

        SignedUrl signedUrl = cloudFrontUtilities.getSignedUrlWithCustomPolicy(request);


        URI modifiedUri = URI.create(signedUrl.url().replace("/specific space+plus-üspecial","/other-file"));
        SdkHttpClient client = ApacheHttpClient.create();
        HttpExecuteResponse response = client.prepareRequest(HttpExecuteRequest.builder()
                                                                               .request(SdkHttpRequest.builder()
                                                                                                      .encodedPath(modifiedUri.getRawPath() + "?" + modifiedUri.getRawQuery())
                                                                                                      .host(modifiedUri.getHost())
                                                                                                      .method(SdkHttpMethod.GET)
                                                                                                      .protocol("https")
                                                                                                      .build())
                                                                               .build()).call();
        assertThat(response.httpResponse().statusCode()).isEqualTo(200);
    }

But I get

java.lang.IllegalArgumentException: Illegal character in path at index 50: https://d3j5oja9i0ormb.cloudfront.net/foo/specific space+plus-üspecial

We need not fix this in this PR but raise an internal task and discuss this.

[RanVaknin]

Sounds good. I can create a new task on next steps.

For reference, both the JS SDK v3 and Go SDK v2 rely on the standard library's URL parsing, and the provided URLs get properly encoded:
Go SDK impl https://github.com/aws/aws-sdk-go-v2/blob/main/feature/cloudfront/sign/sign_url.go#L194

func main() {
	rawURL := "https://example.cloudfront.net/foo/specific space+plus-üspecial"
	uri, err := url.Parse(rawURL)
	if err != nil {
		fmt.Println("Error:", err)
	} else {
		fmt.Println("Parsed successfully", uri)
	}
}
// Parsed successfully https://example.cloudfront.net/foo/specific%20space+plus-%C3%BCspecial

JS SDK impl https://github.com/aws/aws-sdk-js-v3/blob/main/packages/cloudfront-signer/src/sign.ts#L126

try {
const uri = new URL("https://d3j5oja9i0ormb.cloudfront.net/foo/specific space+plus-üspecial")
    console.log("Parsed ", uri.href)
} catch (e) {
    console.log(e);
}
// "Parsed https://d3j5oja9i0ormb.cloudfront.net/foo/specific%20space+plus-%C3%BCspecial

[sonarqubecloud]

Quality Gate passed

Issues
8 New issues
0 Accepted issues

Measures
0 Security Hotspots
83.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[zoewangg]

Is there any reason we are treating empty as null?

[RanVaknin]

Because resourceUrlPattern maps to the JSON Policy document's Resource field, if a user provides null or "" we default to the resourceUrl itself - meaning, there is no pattern, just use the base url as the Resource field in the policy.

The server side implication of send a policy with a Resource field set to "" will result in a 403. I can't foresee a situation in which users are actively prohibiting all access to their distribution just by passing "".

For example:

        CustomSignerRequest request = CustomSignerRequest.builder()
                                                         .resourceUrl(resourceUrl)
                                                         .privateKey(keyFilePath)
                                                         .keyPairId(keyPairId)
                                                         .resourceUrlPattern("")
                                                         .activeDate(activeDate)
                                                         .expirationDate(expirationDate)
                                                         .build();

Will result in a 403.

[zoewangg]

Right, my question is more around: is this something we want to handle in the SDK? Should we just let it fail on the server side instead?

My concern is that some customers may not realize "" is not supported or have bugs in their code to pass empty string to resourceUrlPattern unintentionally, which will be hidden by this SDK logic.

[RanVaknin]

Ack. I can just use a standard null check.

[zoewangg]

This seems to be implementation detail.

Suggesting:

Allow users to specify resource URL pattern in `CloudFrontUtilities#getSignedUrlWithCustomPolicy`. See [#5577](https://github.com/aws/aws-sdk-java-v2/issues/5577)

[RanVaknin]

Agreed. Do you want me to raise a PR to amend the changelog?

[zoewangg]

Yeah, that'd be great. It may be easier after release is finished.