Screenshot/video is not creating folder structure from tests

[PippoRaimondi]

Current behavior

When we have a test folder structure like this:

cypress
    --> specs
        --> some-test.cy.js
        --> some-folder
            --> some-other-test.js

And screenshots are taken within the above tests, we get screenshots added the following way:

cypress
    --> screenshots
         --> screenshot-some-test
         --> screenshot-some-other-test

Desired behavior

How it used to work and how all visual regression tools need it to work:

cypress
    --> specs
         --> some-test.cy.js
         --> some-folder
             --> some-other-test.js

And screenshots are taken within the above tests, we get screenshots added the following way:

cypress
    --> screenshots
        --> screenshot-some-test
        --> some-folder
            --> screenshot-some-other-test

If it's too much work to fix the above, can you please include specName back to the after:screenshot details object?

Cypress 9.x after:screenshot details object:

{
  name: 'folder-structure-test/folder-structure-spec-wholePage',
  specName: 'folder-structure-test/folder-structure-spec.js',
  path: '/Users/filipporaimondi/code/cypress-image-diff/cypress/screenshots/folder-structure-test/folder-structure-spec.js/folder-structure-test/folder-structure-spec-wholePage (1).png',
  ...
}

Cypress 10.x after:screenshot details object:

{
  name: 'folder-structure.cy-wholePage',
  path: '/Users/filipporaimondi/code/cypress-image-diff/cypress/screenshots/folder-structure.cy-wholePage (1).png',
  ...
}

Test code to reproduce

Just take any screenshot within a folder structure above to replicate...

Cypress Version

10.x

Node version

16.x

Operating System

mac

Debug Logs

No response

Other

No response

[gabrielgagne]

Can confirm that I ran into the same problem.
I get the expected result when I run cypress with
specPattern: 'src/integration/**/*.spec.{js,jsx,ts,tsx}',

The path of the screenshot will be the correct
PROJECT_PATH\src\fixtures\screenshots\button\button.component.spec.ts\some_screenshot.png
(see button\button.component.spec.ts)

But if I run cypress with a more targeted config with
specPattern: src/integration/button/**/*.spec.{js,jsx,ts,tsx}

in 'after:screenshot' I have
PROJECT_PATH\src\fixtures\screenshots\button.component.spec.ts\some_screenshot.png
(see the missing \button before \button.component.spec.ts)

And if I run cypress in dev mode with the GUI, I chose the 'button.component.spec.ts' file, I get
PROJECT_PATH\src\fixtures\screenshots\some_screenshot.png
(missing \button AND \button.component.spec.ts)

Didn't have this problem in 9.7.0, but I do now in 10.10.0

[clickcell]

This was a fantastic feature in Cypress 9. I hope it can be fixed soon.

[erikmom]

We are running into the same sort of problems when we try to collect all screenshots in the test:after:run callback function. In that callback function we usually added the screenshots and videos as context to a report, but we are now having a hard time to determine the location of those files as it seems dependent on what tests we are running and in what mode.

Besides that, we've also noticed that the video of a failed test is stored in a different directory than the screenshots of the failed test.

In our opinion it would be best to store screenshots and videos in the same directory structure as is defined for the test files itself (as suggested by the original poster).

[Waterstraal]

Any update on this?

[uhlstefa]

Still no update or fix? I am also having issue with this bug

[emilyrohrbough]

This was an intentional change in Cypress 10.0.0 - see Migration Guide - Generated Files.

That being said, I will surface to the team that the original behavior is preferred. This won't guarantee this behavior will change.

[marktnoonan]

Example details object with missing specName value (empty string) logged to terminal from setupNodeEvents.

{
  testAttemptIndex: 0,
  size: 319081,
  takenAt: '2023-03-15T20:03:35.940Z',
  dimensions: { width: 1000, height: 4913 },
  multipart: true,
  pixelRatio: 1,
  specName: '',
  path: '/Users/marknoonan/Sites/examples/cy-validators-example/cypress/screenshots/the homepage -- renders error if a query param is passed.png',
  scaled: false,
  blackout: [],
  duration: 1713
}

test code:

    it('renders error if a query param is passed', () => {
        cy.visit('https://example.cypress.io')
        cy.screenshot()
    })

cypress.config.js

const { defineConfig } = require("cypress");

module.exports = defineConfig({
    'e2e': {
        setupNodeEvents(on) {
            on('after:screenshot', (details) => {
              console.log(details)
            })
    }
});

[emilyrohrbough]

@GoushiRam provided an in-test workaround to this issue and the bug with the after:screenshot event details. Please see #24090 (comment).

[rkrisztian]

@erikmom said:

We are running into the same sort of problems when we try to collect all screenshots in the test:after:run callback function. In that callback function we usually added the screenshots and videos as context to a report, but we are now having a hard time to determine the location of those files as it seems dependent on what tests we are running and in what mode.

I agree, I have the same problem in #18543.

[s0me0nelv]

Any update regarding the issue?

[jennifer-shehane]

@s0me0nelv No updates on this issue, sorry.