chore(dashboard): user journey smoke tests #7124

LetItRock · 2024-11-25T13:04:27Z

What changed? Why was the change needed?

Enterprise PR: https://github.com/novuhq/packages-enterprise/pull/273

Dashboard - the core user journeys smoke tests that cover:

dashboard-created workflow
code-created workflow

Screenshots

linear · 2024-11-25T13:04:30Z

NV-4795 Write two E2E user journey as a smoke test for new Dashboard

netlify · 2024-11-25T13:04:46Z

✅ Deploy Preview for novu-stg-vite-dashboard-poc ready!

Name	Link
🔨 Latest commit	`fa860f3`
🔍 Latest deploy log	https://app.netlify.com/sites/novu-stg-vite-dashboard-poc/deploys/674d7a4f4008e900085d165d
😎 Deploy Preview	https://deploy-preview-7124.dashboard-v2.novu-staging.co
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

LetItRock · 2024-11-25T13:06:12Z

apps/api/.gitignore

@@ -121,3 +121,4 @@ backups/

 # Nest.js auto-generated metadata (https://docs.nestjs.com/recipes/swc#monorepo-and-cli-plugins)
 src/metadata.ts
+src/.env.test


removed the .env.test file from git, because the local file that I used for the tests had sensitive clerk credentials

LetItRock · 2024-11-25T13:07:48Z

apps/dashboard/package.json

@@ -5,6 +5,7 @@
  "type": "module",
  "scripts": {
    "start": "vite",
+    "start:test": "vite --mode test",


run dashboard and use .env.test file

LetItRock · 2024-11-25T13:08:21Z

apps/dashboard/package.json

+    "@novu/dal": "workspace:*",
+    "@novu/testing": "workspace:*",


used to initialize the api session

LetItRock · 2024-11-25T13:09:42Z

apps/dashboard/playwright.config.ts

+const fileName = fileURLToPath(import.meta.url);
+const dirName = dirname(fileName);
+dotenv.config({ path: path.resolve(dirName, '.env.playwright') });


Because the Dashboard project is configured with "type": "module" this is how we can import the playwright env variables.

LetItRock · 2024-11-25T13:13:06Z

apps/dashboard/playwright.config.ts

+    {
+      name: 'setup',
+      testMatch: /global\.setup\.ts/,
+    },


The global setup test file will initialize the Clerk session for all tests. Using this approach, we can say that for some tests, we don't want to have a session, and then we will be able to verify, for example, the sign-in/up flows.

LetItRock · 2024-11-25T13:22:45Z

apps/dashboard/tests/code-created-flow.spec.ts

+  bridgeServer = new BridgeServer({ secretKey: session.environment.apiKeys[0].key, apiUrl: process.env.API_URL });
+
+  const newWorkflow = workflow(workflowId, async ({ step }) => {
+    await step.inApp(
+      inAppStepId,
+      async (controls) => {
+        return {
+          subject: `Hi ${controls.name}! You've been invited to join the Novu project`,
+          body,
+        };
+      },
+      {
+        controlSchema: {
+          type: 'object',
+          properties: {
+            name: { type: 'string', default: 'John' },
+          },
+        } as const,
+      }
+    );
+  });
+  await bridgeServer.start({ workflows: [newWorkflow] });
+  await session.testAgent.post(`/v1/bridge/sync`).send({
+    bridgeUrl: bridgeServer.serverPath,
+  });


Run the bridge server and sync the workflow

LetItRock · 2024-11-25T13:23:04Z

apps/dashboard/tests/code-created-flow.spec.ts

+  await session.testAgent.post(`/v1/bridge/sync`).send({
+    bridgeUrl: bridgeServer.serverPath,
+  });
+  await page.waitForTimeout(2000);


should be removed

LetItRock · 2024-11-25T13:23:44Z

apps/dashboard/tests/global.setup.ts

+  /**
+   * TODO: Currently we are running tests with a single Clerk user and organization.
+   * This approach has a drawback that the tests are not independent from each other and write the data to the same organization.
+   * We should consider creating a new Clerk user and organization for each test using the @clerk/backend package.
+   * Then initialize the BE session with the new Clerk user and organization and update Clerk user metadata from the newly created session.
+   */


hopefully, it explains the problem well

Correct! We should leverage Clerk Backend API, create a user before every run, sign in with that user, and then delete the user at the end.

Regarding the need for Clerk webhooks during user and organization creation, I suggest connecting to Mongo directly from the test code using the response of the Clerk API to create the user and organization record the application needs to function.

On user deletion, I don't think we need to do anything in Mongo as we run in a dockerized environment.

I'll think about this, as it will require reimplementing all the current setup.

LetItRock · 2024-11-25T13:26:31Z

apps/dashboard/tests/package.json

@@ -0,0 +1,3 @@
+{
+  "type": "commonjs"


The dal and testing packages are CJS modules which can't be imported by the ESM modules.
So this is the trick (together with tsconfig.json) to say "please allow me to import" in my test files :D

@LetItRock Since we wrote dedicated E2E helpers in the Dashboard, do we need @novu/testing afterall?

the @novu/testing is not needed, I made it in a way that the tests setup is independent of it

LetItRock · 2024-11-25T13:30:28Z

libs/testing/src/ee/ee.organization.service.ts

-  private memberRepository = getEERepository<MemberRepository>('MemberRepository');
+  private memberRepository: MemberRepository;
+
+  constructor({ mockClerkClient = true }: { mockClerkClient?: boolean } = {}) {


For the Dashboard E2E tests we don't want to mock the Clerk client, because we initialize the Clerk session with a real user that has the org and metadata that is used by the Dashboard code.

pkg-pr-new · 2024-11-25T13:41:28Z

Open in Stackblitz

@novu/client

npm i https://pkg.pr.new/novuhq/novu/@novu/client@7124

@novu/framework

npm i https://pkg.pr.new/novuhq/novu/@novu/framework@7124

@novu/js

npm i https://pkg.pr.new/novuhq/novu/@novu/js@7124

@novu/nest

npm i https://pkg.pr.new/novuhq/novu/@novu/nest@7124

@novu/headless

npm i https://pkg.pr.new/novuhq/novu/@novu/headless@7124

@novu/nextjs

npm i https://pkg.pr.new/novuhq/novu/@novu/nextjs@7124

@novu/node

npm i https://pkg.pr.new/novuhq/novu/@novu/node@7124

@novu/notification-center

npm i https://pkg.pr.new/novuhq/novu/@novu/notification-center@7124

novu

npm i https://pkg.pr.new/novuhq/novu@7124

@novu/providers

npm i https://pkg.pr.new/novuhq/novu/@novu/providers@7124

@novu/react

npm i https://pkg.pr.new/novuhq/novu/@novu/react@7124

@novu/react-native

npm i https://pkg.pr.new/novuhq/novu/@novu/react-native@7124

@novu/shared

npm i https://pkg.pr.new/novuhq/novu/@novu/shared@7124

@novu/stateless

npm i https://pkg.pr.new/novuhq/novu/@novu/stateless@7124

commit: 5d0de32

LetItRock · 2025-02-20T22:42:29Z

.github/workflows/reusable-dashboard-e2e.yml

+          touch .env
+          echo VITE_LAUNCH_DARKLY_CLIENT_SIDE_ID=${{ secrets.LAUNCH_DARKLY_CLIENT_SIDE_ID }} >> .env
+          echo VITE_API_HOSTNAME=http://127.0.0.1:1336 >> .env
+          echo VITE_WEBSOCKET_HOSTNAME=http://127.0.0.1:1340 >> .env
+          echo VITE_LEGACY_DASHBOARD_URL=http://127.0.0.1:4200 >> .env
+          echo VITE_CLERK_PUBLISHABLE_KEY=${{ secrets.CLERK_E2E_PUBLISHABLE_KEY }} >> .env


.env file for the Dashboard

LetItRock · 2025-02-20T22:43:39Z

.github/workflows/reusable-dashboard-e2e.yml

+          touch .env.playwright
+          echo NOVU_ENTERPRISE=true >> .env.playwright
+          echo NEW_RELIC_ENABLED=false >> .env.playwright
+          echo NEW_RELIC_APP_NAME=Novu >> .env.playwright
+          echo MONGO_URL=mongodb://127.0.0.1:27017/novu-test >> .env.playwright
+          echo API_URL=http://127.0.0.1:1336 >> .env.playwright
+          echo CLERK_ENABLED=true >> .env.playwright
+          echo CLERK_PUBLISHABLE_KEY=${{ secrets.CLERK_E2E_PUBLISHABLE_KEY }} >> .env.playwright
+          echo CLERK_SECRET_KEY=${{ secrets.CLERK_E2E_SECRET_KEY }} >> .env.playwright
+          echo NODE_ENV=test >> .env.playwright


.env file for the playwright, used to initialize the session, create, and sync users/orgs

LetItRock · 2025-02-20T22:46:07Z

apps/dashboard/package.json

+    "@clerk/backend": "^1.6.2",
+    "@clerk/testing": "^1.3.27",


used to create Clerk user/orgs and to sign-in into the app

LetItRock · 2025-02-20T22:49:33Z

apps/dashboard/src/components/primitives/tag-input.tsx

+                  e.preventDefault();
+                  e.stopPropagation();


fix the bug - the click on remove button was causing the form submission

LetItRock · 2025-02-20T22:50:02Z

apps/dashboard/src/components/workflow-editor/base-node.tsx

@@ -80,7 +80,7 @@ export const NodeBody = ({

  return (
    <HoverCard openDelay={300}>
-      <HoverCardTrigger>
+      <HoverCardTrigger asChild>


react warning a nested in a

LetItRock · 2025-02-20T22:50:42Z

apps/dashboard/src/hooks/use-on-element-unmount.ts

-      return () => {
-        observer.disconnect();
-      };


react warning on that the callback is never called

SokratisVidros · 2025-01-24T09:44:41Z

apps/dashboard/tests/package.json

@@ -0,0 +1,3 @@
+{
+  "type": "commonjs"


SokratisVidros · 2025-01-24T09:56:05Z

apps/dashboard/tests/utils/session.ts

+   */
+  await page.addInitScript((currentSession) => {
+    window.addEventListener('DOMContentLoaded', () => {
+      localStorage.setItem('nv_auth_token', currentSession.token);


Do you know if this is necessary in Dashboard v2? If not mistaken this is V1 logic for the community auth.

it's the leftover from the old implementation, will remove it

SokratisVidros · 2025-02-21T07:31:28Z

apps/dashboard/tests/manage-workflows.e2e.ts

+  await triggerWorkflowPage.triggerWorkflowBtnClick();
+  const activityPanel = await triggerWorkflowPage.getActivityPanel();
+  await expect(activityPanel).toBeVisible();
+  await expect(activityPanel.locator('span').filter({ hasText: workflowNameUpdated })).toBeVisible();


I suggest we run more assertions on the trigger page to ensure that the status is updated while running.

I'll do that in the PR that I've opened for the activity panel fix: #7768

SokratisVidros · 2025-02-21T07:34:17Z

apps/dashboard/tests/package.json

@@ -0,0 +1,3 @@
+{
+  "type": "commonjs"


@LetItRock Since we wrote dedicated E2E helpers in the Dashboard, do we need @novu/testing afterall?

SokratisVidros · 2025-02-21T07:34:57Z

apps/dashboard/tests/manage-workflows.e2e.ts

+import { TriggerWorkflowPage } from './page-object-models/trigger-workflow-page';
+
+test('manage workflows', async ({ page }) => {
+  const workflowName = 'test-workflow';


How about starting the flow from sign-up so that we test the onboarding?

we should do it in a separate test as the test fixture is doing all the user setup logic

LetItRock added 2 commits November 25, 2024 13:48

chore(dashboard): user journey smoke tests

Loading
Loading status checks…

e12abe9

chore(dashboard): fix imports in page models

Loading
Loading status checks…

9f22ad5

LetItRock self-assigned this Nov 25, 2024

LetItRock commented Nov 25, 2024

View reviewed changes

LetItRock requested review from SokratisVidros, BiswaViraj, ChmaraX and desiprisg November 25, 2024 13:31

Merge branch 'next' into nv-4795-dashboard-user-journey-smoke-tests

Loading
Loading status checks…

d0a3333

LetItRock temporarily deployed to Linting November 25, 2024 13:33 — with GitHub Actions Inactive

LetItRock had a problem deploying to Linting November 25, 2024 13:33 — with GitHub Actions Failure

chore(dashboard): remove timeout

Loading
Loading status checks…

ceabff2

LetItRock temporarily deployed to Linting November 25, 2024 13:37 — with GitHub Actions Inactive

LetItRock had a problem deploying to Linting November 25, 2024 13:37 — with GitHub Actions Failure

chore(api): revert .env.test

Loading
Loading status checks…

8ad2e31

LetItRock temporarily deployed to Linting November 25, 2024 14:23 — with GitHub Actions Inactive

LetItRock had a problem deploying to Linting November 25, 2024 14:23 — with GitHub Actions Failure

chore(dashboard): fix spell check and adjust pipelines

Loading
Loading status checks…

461275c

LetItRock requested review from a team and scopsy and removed request for a team November 25, 2024 15:52

LetItRock temporarily deployed to Linting November 25, 2024 15:52 — with GitHub Actions Inactive

LetItRock had a problem deploying to Linting February 20, 2025 16:56 — with GitHub Actions Failure

chore(dashboard): fix failing e2e

Loading
Loading status checks…

8570e9d

LetItRock temporarily deployed to Linting February 20, 2025 17:39 — with GitHub Actions Inactive

chore(libs): revert old changes

Loading
Loading status checks…

8ec0afa

LetItRock temporarily deployed to Linting February 20, 2025 23:02 — with GitHub Actions Inactive

LetItRock commented Feb 20, 2025

View reviewed changes

SokratisVidros approved these changes Feb 21, 2025

View reviewed changes

Merge branch 'next' into nv-4795-dashboard-user-journey-smoke-tests

Loading
Loading status checks…

34888fe

LetItRock temporarily deployed to Linting February 24, 2025 16:55 — with GitHub Actions Inactive

fix(testing): failing api e2e

Loading
Loading status checks…

94e10bc

LetItRock temporarily deployed to Linting February 24, 2025 17:28 — with GitHub Actions Inactive

fix(testing): failing api e2e

Loading
Loading status checks…

63b597e

LetItRock temporarily deployed to Linting February 24, 2025 17:50 — with GitHub Actions Inactive

fix(dashboard): update ee source

Loading
Loading status checks…

d595298

LetItRock temporarily deployed to Linting February 24, 2025 18:12 — with GitHub Actions Inactive

LetItRock merged commit ae191d8 into next Feb 24, 2025
16 checks passed

LetItRock deleted the nv-4795-dashboard-user-journey-smoke-tests branch February 24, 2025 18:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(dashboard): user journey smoke tests #7124

chore(dashboard): user journey smoke tests #7124

LetItRock commented Nov 25, 2024 •

edited

Loading

linear bot commented Nov 25, 2024

netlify bot commented Nov 25, 2024 •

edited

Loading

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

SokratisVidros Dec 10, 2024

LetItRock Dec 31, 2024

LetItRock Nov 25, 2024

SokratisVidros Jan 24, 2025

SokratisVidros Feb 21, 2025

LetItRock Feb 24, 2025

LetItRock Nov 25, 2024

pkg-pr-new bot commented Nov 25, 2024 •

edited

Loading

LetItRock Feb 20, 2025

LetItRock Feb 20, 2025

LetItRock Feb 20, 2025

LetItRock Feb 20, 2025

LetItRock Feb 20, 2025

LetItRock Feb 20, 2025

SokratisVidros Jan 24, 2025

SokratisVidros Jan 24, 2025

LetItRock Feb 24, 2025

SokratisVidros Feb 21, 2025

LetItRock Feb 24, 2025

SokratisVidros Feb 21, 2025

SokratisVidros Feb 21, 2025 •

edited

Loading

LetItRock Feb 24, 2025

chore(dashboard): user journey smoke tests #7124

chore(dashboard): user journey smoke tests #7124

Conversation

LetItRock commented Nov 25, 2024 • edited Loading

What changed? Why was the change needed?

Screenshots

linear bot commented Nov 25, 2024

netlify bot commented Nov 25, 2024 • edited Loading

✅ Deploy Preview for novu-stg-vite-dashboard-poc ready!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pkg-pr-new bot commented Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SokratisVidros Feb 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LetItRock commented Nov 25, 2024 •

edited

Loading

netlify bot commented Nov 25, 2024 •

edited

Loading

pkg-pr-new bot commented Nov 25, 2024 •

edited

Loading

SokratisVidros Feb 21, 2025 •

edited

Loading