Addon A11y: Introduce parameters.a11y.test

[valentinpalkovic]

Closes #

What I did

I have introduced a new parameter parameters.a11y.test to control how the @storybook/addon-a11y addon behaves in combination with @storybook/experimental-addon-test.

The parameter controls how and whether accessibility violations should be displayed and shown in Storybook's UI and in a Vitest standalone run:

	off	todo (default)	error
Storybook Test UI	Skip a11y checks	Run a11y checks but display failures as warnings in sidebar (yellow triangle)	Run a11y checks and display failures as errors in sidebar (red circle)
Vitest CLI	Skip a11y checks	Skip a11y checks	Run a11y checks and error on violations

To support this new behaviour, the following work was done:

• adjusted automigrations to generate parameters.a11y.test: 'todo' into the user's .storybook/preview.ts file per default if it is not set yet when installing @storybook/addon-a11y or @storybook/experimental-addon-test (only if @storybook/addon-a11y is installed)
• adjusted the reporting of a11y violations to match the behaviour mentioned in the aforementioned table
• removed the old undocumented a11y-test tag and its functionalities

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

• stories
• unit tests
• integration tests
• end-to-end tests

Manual testing

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

Documentation

• Add or update documentation reflecting your changes
• If you are deprecating/removing a feature, make sure to update
  MIGRATION.MD

Checklist for Maintainers

• When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

• Make sure this PR contains one of the labels below:

  Available labels

  • bug: Internal changes that fixes incorrect behavior.
  • maintenance: User-facing maintenance tasks.
  • dependencies: Upgrading (sometimes downgrading) dependencies.
  • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
  • cleanup: Minor cleanup style change. Will not show up in release changelog.
  • documentation: Documentation only changes. Will not show up in release changelog.
  • feature request: Introducing a new feature.
  • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
  • other: Changes that don't fit in the above categories.

🦋 Canary release

This pull request has been released as version 0.0.0-pr-30516-sha-086477d9. Try it out in a new sandbox by running npx [email protected] sandbox or in an existing project with npx [email protected] upgrade.

More information

Published version	0.0.0-pr-30516-sha-086477d9
Triggered by	@valentinpalkovic
Repository	storybookjs/storybook
Branch	valentin/a11y-parameter-test
Commit	086477d9
Datetime	Tue Feb 11 15:08:22 UTC 2025 (1739286502)
Workflow run	13266015775

To request a new release of this pull request, mention the @storybookjs/core team.

core team members can create a new canary release here or locally with gh workflow run --repo storybookjs/storybook canary-release-pr.yml --field pr=30516

name	before	after	diff	z	%
createSize	0 B	0 B	0 B	-	-
generateSize	80.5 MB	80.5 MB	0 B	0.45	0%
initSize	80.5 MB	80.5 MB	0 B	0.45	0%
diffSize	97 B	97 B	0 B	-	0%
buildSize	7.31 MB	7.31 MB	-13 B	2.57	0%
buildSbAddonsSize	1.9 MB	1.9 MB	0 B	2.51	0%
buildSbCommonSize	195 kB	195 kB	0 B	-	0%
buildSbManagerSize	1.88 MB	1.88 MB	0 B	-0.47	0%
buildSbPreviewSize	0 B	0 B	0 B	-	-
buildStaticSize	0 B	0 B	0 B	-	-
buildPrebuildSize	3.97 MB	3.97 MB	0 B	2.45	0%
buildPreviewSize	3.34 MB	3.34 MB	-13 B	2.29	0%
testBuildSize	0 B	0 B	0 B	-	-
testBuildSbAddonsSize	0 B	0 B	0 B	-	-
testBuildSbCommonSize	0 B	0 B	0 B	-	-
testBuildSbManagerSize	0 B	0 B	0 B	-	-
testBuildSbPreviewSize	0 B	0 B	0 B	-	-
testBuildStaticSize	0 B	0 B	0 B	-	-
testBuildPrebuildSize	0 B	0 B	0 B	-	-
testBuildPreviewSize	0 B	0 B	0 B	-	-

name	before	after	diff	z	%
createTime	7.2s	24.6s	17.3s	0.3	70.6%
generateTime	18.6s	18.1s	-492ms	-1.09	-2.7%
initTime	4.3s	4.1s	-225ms	-1.55	🔰-5.4%
buildTime	8.7s	8.4s	-242ms	-0.9	-2.9%
testBuildTime	0ms	0ms	0ms	-	-
devPreviewResponsive	5.5s	6s	523ms	0.63	8.6%
devManagerResponsive	3.9s	4.5s	656ms	0.79	14.3%
devManagerHeaderVisible	701ms	932ms	231ms	0.94	24.8%
devManagerIndexVisible	708ms	972ms	264ms	0.96	27.2%
devStoryVisibleUncached	4.8s	4.5s	-287ms	1.41	🔰-6.2%
devStoryVisible	792ms	973ms	181ms	0.67	18.6%
devAutodocsVisible	719ms	1s	334ms	2.42	🔺31.7%
devMDXVisible	783ms	862ms	79ms	0.87	9.2%
buildManagerHeaderVisible	826ms	719ms	-107ms	-0.42	-14.9%
buildManagerIndexVisible	931ms	729ms	-202ms	-0.6	-27.7%
buildStoryVisible	812ms	702ms	-110ms	-0.37	-15.7%
buildAutodocsVisible	597ms	597ms	0ms	-0.27	0%
buildMDXVisible	860ms	588ms	-272ms	-0.56	-46.3%

Greptile Summary

This PR introduces a new parameters.a11y.test parameter to control how accessibility violations are handled in both Storybook's UI and Vitest test runs, replacing the previous tag-based approach.

• Added new A11yTest type in code/addons/a11y/src/params.ts with values 'off', 'todo', or 'error' to control test behavior
• Modified experimental_afterEach in code/addons/a11y/src/preview.tsx to handle test status based on parameter value instead of tags
• Added automigration in code/lib/cli-storybook/src/automigrate/fixes/addon-a11y-addon-test.ts to help users transition to the new parameter system
• Updated TestProviderRender component to support new parameter-based accessibility testing configuration
• Removed A11Y_TEST_TAG constant and related tag-based functionality across multiple files

[nx-cloud]

View your CI Pipeline Execution ↗ for commit 2e9408b.

Command	Status	Duration	Result
nx run-many -t build --parallel=3	✅ Succeeded	2m 1s	View ↗

---

☁️ Nx Cloud last updated this comment at 2025-02-12 14:49:51 UTC

[storybook-pr-benchmarking]

Package Benchmarks

^{Commit: 2e9408b, ran on 12 February 2025 at 14:54:56 UTC}

No significant changes detected, all good. 👏

[greptile-apps]

_{13 file(s) reviewed, 3 comment(s)
Edit PR Review Bot Settings | Greptile}