ci: setup e2e tests workflow and local webpack e2e test

.changeset/chilly-snails-tease.md

@@ -0,0 +1,2 @@
+---
+---

.github/integ-config/local-integ-all.yml

@@ -0,0 +1,4 @@
+tests:
+  - test_name: webpack_e2e_test
+    desc: 'Webpack 5'
+    framework: webpack

.github/workflows/callable-local-e2e-test.yml

@@ -0,0 +1,45 @@
+name: E2E Test
+run-name: e2e.${{ inputs.test_name }}
+
+on:
+  workflow_call:
+    inputs:
+      test_name:
+        required: true
+        type: string
+      framework:
+        required: true
+        type: string
+      timeout_minutes:
+        required: true
+        type: number
+
+jobs:
+  e2e-test:
+    name: E2E ${{ inputs.test_name }}
+    runs-on: ubuntu-latest
+    timeout-minutes: ${{ inputs.timeout_minutes }}
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@24cb9080177205b6e8c946b17badbe402adc938f # v3.4.0 https://github.com/actions/checkout/commit/24cb9080177205b6e8c946b17badbe402adc938f
+        with:
+          path: amplify-data
+      - name: Setup node and build the repository
+        uses: ./amplify-data/.github/actions/node-and-build
+      - name: Run e2e tests for ${{ inputs.test_name }} dev
+        shell: bash
+        working-directory: ./amplify-data
+        env:
+          E2E_FRAMEWORK: ${{ inputs.framework }}
+          E2E_TEST_NAME: ${{ inputs.test_name }}
+        run: |
+          npm run ci:test dev
+      - name: Run e2e tests for ${{ inputs.test_name }} prod
+        shell: bash
+        working-directory: ./amplify-data
+        env:
+          E2E_FRAMEWORK: ${{ inputs.framework }}
+          E2E_TEST_NAME: ${{ inputs.test_name }}
+        run: |
+          npm run ci:test prod

.github/workflows/callable-local-e2e-tests.yml

@@ -26,3 +26,34 @@ jobs:
       - name: Run tests
         working-directory: ./amplify-data
         run: npm run e2e-node:test
+
+  e2e-prep:
+    name: Get local E2E test config
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
+        with:
+          path: amplify-data
+      - name: Read integ config files
+        id: load_config
+        run: |
+          echo "INTEG_CONFIG=$(cat .github/integ-config/local-integ-all.yml | yq '.tests' -o=json | jq -c .)" >> $GITHUB_OUTPUT
+        working-directory: ./amplify-data
+    outputs:
+      integ-config: ${{ steps.load_config.outputs.INTEG_CONFIG }}
+
+
+  e2e-test-runner:
+    name: E2E test runnner
+    needs: e2e-prep
+    secrets: inherit
+    strategy:
+      matrix:
+        integ-config: ${{ fromJson(needs.e2e-prep.outputs.integ-config) }}
+      fail-fast: false
+    uses: ./.github/workflows/callable-local-e2e-test.yml
+    with:
+      test_name: ${{ matrix.integ-config.test_name }}
+      framework: ${{ matrix.integ-config.framework }}
+      timeout_minutes: ${{ matrix.integ-config.timeout_minutes || 35 }}

e2e-TEST-COVERAGE.md

@@ -6,7 +6,7 @@ The following table outlines the current e2e test coverage for various runtimes
 |---------------------|-----------|
 | Chrome           | [✅](.github/workflows/callable-e2e-test.yml) | 
 | Node             | [✅](packages/e2e-tests/node/jest.config.ts) | 
-| Webpack                |    	⚠️     | 
+| Webpack                |    [✅](packages/e2e-tests/webpack/webpack.config.js)	     | 
 | Rollup                 | 	⚠️        |
 | Metro                    | 	⚠️      |
 | React Natives            | 	⚠️      |

package.json

@@ -44,6 +44,7 @@
     "stop:npm-proxy": "tsx scripts/stop_npm_proxy.ts",
     "test": "turbo run test && npm run test:scripts && npm run e2e-exports",
     "test:scripts": "npx jest scripts/",
+    "ci:test": "./scripts/bundler_test.sh",
     "turbo": "turbo",
     "vend": "npm run start:npm-proxy && npm run publish:local"
   },

packages/e2e-tests/README.md

@@ -1,16 +1,18 @@
 # `amplify-data` E2E Tests
 
+## Node testing
+
 This package contains automated tests for validating functionality E2E in Node.
 
-## How it works:
+### How it works:
 
 - All tests within `node` share a single "long-lived" Amplify backend (added to
   the shared Cypress account).
 - Sample will use `amplify-js@unstable` and local versions of packages in this repo.
 - See [test utils](node/utils.ts) for helpers that configure Amplify, generate /
   configure the API client, as well as for additional test utilities and types.
 
-## How to add a new test / test local changes:
+### How to add a new test / test local changes:
 
 1. Create a new file under `__tests__`
 2. Use the helpers to configure Amplify and generate / configure the API client.
@@ -19,12 +21,12 @@ This package contains automated tests for validating functionality E2E in Node.
    ensure that a `package-lock` file is not generated for this sample.
 4. [TODO] subscription testing (will be added with follow-up PR)
 
-## [TODO] Global cleanup
+### [TODO] Global cleanup
 
 Add a check to clean up potential orphaned records after all tests have run
 (see https://jestjs.io/docs/configuration#globalteardown-string).
 
-## [Known Issue] Jest open handles warning
+### [Known Issue] Jest open handles warning
 
 Tests that use `Hub` currently emit the following warning when running the tests:
 
@@ -42,7 +44,7 @@ TODO: Investigate and/or fix.
 
 All tests within the `sandbox` directory deploy a new sandbox for each test run.
 
-## How it works:
+### How it works:
 
 CLI commands are managed by the process controller. The process controller is
 responsible for executing CLI commands, and listening for output from the CLI.
@@ -58,7 +60,7 @@ Test setup / execution is structured as follows:
 5. Sandbox is torn down after the tests are complete. `.amplify/` and
    `amplify_outputs.json` are deleted.
 
-## CI AWS Credentials:
+### CI AWS Credentials:
 
 We follow the recommended OS approach to get AWS account credentials: https://w.amazon.com/bin/view/Open_Source/GitHub/Actions.
 Specifically, we use the [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials#assuming-a-role) GH Action to obtain credentials from
@@ -83,16 +85,27 @@ To view identify provider details in the console, see: `Console > IAM > Identity
 - https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services
 - https://github.com/aws-actions/configure-aws-credentials
 
-## How to add a new test:
+### How to add a new test:
 
 1. Add a new project directory under `amplify-backends`. Must also contain a
    basic `package.json`.
 2. Add a corresponding test file under `__tests__`.
 3. Use the utils to generate / teardown the sandbox withthe process controller.
 
-## Using Execa with Jest
+### Using Execa with Jest
 
 Execa is the primary dependency used to run CLI commands in the process
 controller, and it is a pure ES module. Unlike `amplify-backend`, which uses the
 Node test runner, our tests are written in Jest. This results in [this issue](https://github.com/sindresorhus/execa/issues/465).
 The workaround is to use Jest's [experimental support for ECMAScript Modules](https://jestjs.io/docs/ecmascript-modules). There is a backlog item to investigate other potential solutions.
+
+## General tool testing:
+
+This is the general guidance for testing module bundlers. When testing other tools in the future, please adapt these guidelines accordingly and update this file.
+
+### How to add a new e2e test sample:
+
+1. Add a sample in this directory following the structure `e2e-tests/<framework>`.
+2. Add configs for your test to the [local-integ-all.yml](../../.github/integ-config/local-integ-all.yml) file.
+3. Create an npm command for your test script and call it from [callable-local-e2e-test](../../.github/workflows/callable-local-e2e-test.yml).
+4. Update the e2e test coverage for `amplify-data` repo in [e2e test coverage table](../../e2e-TEST-COVERAGE.md).

packages/e2e-tests/webpack/.gitignore

@@ -0,0 +1,8 @@
+# amplify
+node_modules
+.amplify
+amplify_outputs*
+amplifyconfiguration*
+
+# lock file
+package-lock.json