Skip to content

fix(browserstack-service): run a11y scans for jasmine framework (SDK-7190) - #123

Open
osho-20 wants to merge 3 commits into
mainfrom
fix/sdk-7190-jasmine-a11y
Open

fix(browserstack-service): run a11y scans for jasmine framework (SDK-7190)#123
osho-20 wants to merge 3 commits into
mainfrom
fix/sdk-7190-jasmine-a11y

Conversation

@osho-20

@osho-20 osho-20 commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

What is this about?

Web Accessibility Automation produced no report for WDIO projects using the jasmine framework, with no warning. The customer's build showed the scan as provisioned on the A11y side, but zero scan commands were ever sent.

Root cause: the launcher provisions the a11y build framework-agnostically (build registered, accessibility: true stamped into caps, extension injected), but AccessibilityHandler.beforeTest/afterTest were hard-gated to this._framework !== 'mocha'. For jasmine both hooks returned immediately, so the scan-session map was never armed (no auto-scans) and saveTestResults never ran — the session was half-enabled and failed silently.

Fix:

  • beforeTest/afterTest gates now admit ['mocha', 'jasmine'] — WDIO's jasmine adapter emits the exact same service hooks as mocha (cucumber keeps its beforeScenario/afterScenario path).
  • The scan include/exclude tag filter reads the spec name from test.title ?? test.description, since jasmine test objects carry the name in description.
  • getIdentifier already handled jasmine (test.fullName); this also un-breaks browser.startA11yScanning() for jasmine because _testIdentifier is now set.

Verification (real Automate sessions, same spec/config, only the service build differs):

Run Session A11y activity in service log
Bugged (pre-fix) bugged session a11y provisioned (success: true), then 0 scans / 0 saved results — matches customer log exactly
Fixed fixed session scan started → performed → Accessibility Automation Scanner saved the test case results. {success: true}

A mocha control run of the same spec produced the identical a11y outcome as the fixed jasmine run.

Tests: rewrote the jasmine unit test that previously asserted the buggy skip behaviour; added jasmine beforeTest orchestration, scan-map arming, jasmine afterTest stop-event, and cucumber-negative cases. tests/accessibility-handler.test.ts: 44/44 passing; full package suite shows no new failures vs main.

Related Jira task/s

SDK-7190

Release (mandatory for every PR — required for the ready-for-review label)

Version bump: (required — tick exactly one)

  • minor (backwards-compatible feature)
  • patch (bug fix or other small change)

Release notes type: (optional)

  • New Feature
  • Bug Fix
  • Other Improvement

Release notes (customer-facing): (optional but encouraged)

  • Fixed Accessibility Automation producing no report for WebdriverIO suites running on the jasmine framework. Accessibility scans now run for jasmine specs, as they already did for mocha.

Release notes (internal): (required — engineer-facing; what actually changed / why)

  • SDK-7190: AccessibilityHandler.beforeTest/afterTest were hard-gated to this._framework !== 'mocha', so jasmine sessions were provisioned a11y-side (build registered, caps stamped, extension injected) but never armed the scan-session map and never ran saveTestResults — silently producing no report. Both gates now admit ['mocha', 'jasmine']; cucumber keeps its beforeScenario/afterScenario path.
  • The scan include/exclude tag filter reads the spec name from test.title ?? test.description, because jasmine carries the name in description. Knock-on: browser.startA11yScanning() now works for jasmine, since _testIdentifier is set.

Checklist

  • Ready to review
  • Has it been tested locally?

PR Validations

Run Tests: Comment RUN_TESTS to trigger sanity tests.

🤖 Generated with Claude Code

…7190)

Web Accessibility Automation silently produced no report on jasmine:
the launcher provisions the a11y build framework-agnostically, but the
accessibility handler's beforeTest/afterTest were hard-gated to mocha,
so no scan was ever fired and saveTestResults never ran. WDIO's jasmine
adapter emits the same beforeTest/afterTest service hooks as mocha, so
admit jasmine through the gate and read the spec name from jasmine's
`description` field for the include/exclude tag filter.

Verified on Automate: identical jasmine spec produced no scan activity
before the fix and a saved a11y report after it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@osho-20
osho-20 requested a review from a team as a code owner August 6, 2026 07:14
Comment thread .changeset/sdk-7190-jasmine-a11y.md Outdated
Comment on lines +5 to +7
fix: run Web Accessibility Automation scans for the jasmine framework (SDK-7190)

The accessibility handler's beforeTest/afterTest hooks were hard-gated to mocha, so jasmine sessions with `accessibility: true` were provisioned on the A11y side (build registered, extension injected) but never orchestrated a scan or saved results — silently producing no report. WDIO's jasmine adapter emits the same beforeTest/afterTest service hooks as mocha, so the gate now admits jasmine, and the scan include/exclude filter reads the spec name from jasmine's `description` field.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How exactly will this be used? Do we want to add all the details here, including jira id?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both fair — and digging into it, the file shouldn't have been hand-written at all.

How it gets used: on merge to main, changeset version inlines the changeset body verbatim as a bullet in packages/browserstack-service/CHANGELOG.md under the new version, and at publish release.yml (createGithubReleases: true) turns those entries into the GitHub release notes. So it's customer-facing release copy, not an engineering note — the Jira ID and the RCA paragraph don't belong there.

Why it existed: changeset-from-pr.yml already derives .changeset/pr-<number>.md from the PR body's ## Release section (preferring the customer-facing bullets), which is why the template says you don't need to run npx changeset. This PR had no ## Release section at all, so generate no-op'd and check-ready was failing — the hand-written file was quietly standing in for it. Had it merged as-is we'd have shipped two CHANGELOG bullets for one fix.

Fixed both ends:

  • Dropped .changeset/sdk-7190-jasmine-a11y.md (5fc8751).
  • Filled the PR body per the repo template, so the generator produced .changeset/pr-123.md:
---
"@wdio/browserstack-service": patch
---

- Fixed Accessibility Automation producing no report for WebdriverIO suites running on the jasmine framework. Accessibility scans now run for jasmine specs, as they already did for mocha.

One customer-facing line; SDK-7190 and the beforeTest/afterTest gate detail now live in the internal release notes in the PR body instead.

…e generation

changeset-from-pr.yml derives .changeset/pr-<number>.md from the PR body's
"## Release" section, so a hand-written file both duplicates the CHANGELOG
bullet and puts internal RCA detail in customer-facing release notes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

🔴 SDK PR Review gate is red. Pending:

  • The SDK PR Review Agent has not reviewed the current head commit yet — run the SDK PR Review Agent.

It turns green once the latest SDK PR Review Agent run reports GTG on the current head commit. A native reviewer approval is separately required by branch protection before merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants