Skip to content

Security changes - #359

Merged
gaurav-singh-9227 merged 5 commits into
browserstack:mainfrom
SavioBS629:security-changes
Aug 3, 2026
Merged

Security changes#359
gaurav-singh-9227 merged 5 commits into
browserstack:mainfrom
SavioBS629:security-changes

Conversation

@SavioBS629

Copy link
Copy Markdown
Collaborator

No description provided.

SavioBS629 and others added 2 commits August 3, 2026 14:24
…esponses

SDK setup tools (setupBrowserStackAutomateTests and the Percy SDK flows)
interpolated the caller's real BROWSERSTACK_USERNAME/ACCESS_KEY into the
browserstack.yml and setup instructions returned to the LLM, so keys could
land in chat history, logs, or committed config files. Emit env-var
references (${BROWSERSTACK_USERNAME}/${BROWSERSTACK_ACCESS_KEY}) for
shell/yml and <your_browserstack_*> placeholders for JSON configs instead,
and stop threading the real credentials through the instruction call chain.

Also redact the stored site password in getAccessibilityAuthConfig's
response (mirrors the existing '***' redaction on the create path).

Adds regression tests asserting env-var references are emitted (never the
real credentials) and that the accessibility auth-config password is
redacted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
setupBrowserStackAppAutomateTests interpolated the caller's real
BROWSERSTACK_USERNAME/ACCESS_KEY into the generated browserstack.yml,
env exports, SDK setup commands, and language config files (Java, Python,
C#, Node, Ruby) returned to the LLM — same exposure class as the web
Automate setup tools. Emit <your_browserstack_*> placeholders instead and
stop threading real credentials through the instruction generators. The
credentials are still passed to uploadApp, which needs them for the actual
app-upload API call (that path returns only a bs:// URL, no secrets).

Adds a regression test covering every supported language/framework combo
that asserts placeholders are emitted and real credentials never appear.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@SavioBS629 SavioBS629 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review (automated) — 9 inline finding(s). Full report in the PR comment below. Verdict: Failed - see PR comment.

Comment thread src/tools/appautomate-utils/appium-sdk/formatter.ts Outdated
Comment thread src/tools/sdk-utils/bstack/constants.ts Outdated
Comment thread src/tools/sdk-utils/bstack/constants.ts Outdated
Comment thread src/tools/sdk-utils/bstack/constants.ts Outdated
Comment thread src/tools/sdk-utils/bstack/constants.ts Outdated
Comment thread src/tools/sdk-utils/bstack/constants.ts Outdated
Comment thread src/tools/sdk-utils/bstack/commands.ts Outdated
Comment thread src/tools/appautomate-utils/appium-sdk/languages/ruby.ts Outdated
Comment thread tests/tools/sdk-utils-commands.test.ts Outdated
@SavioBS629

Copy link
Copy Markdown
Collaborator Author

Claude Code PR Review

PR: #359Head: 01a8c56Reviewers: stack:code-review

Summary

Removes live BrowserStack credentials from generated SDK setup instructions across the Automate/App Automate/Percy flows — dropping username/accessKey parameters from ~14 generator signatures and replacing interpolated values with placeholders — and redacts the password field in the getAccessibilityAuthConfig tool response.

The security intent is right and the signature refactor is mechanically complete (every call site updated; npm run build is green: eslint clean, prettier clean, 256 tests pass, tsc --noEmit exit 0). But the replacement placeholders are not functionally valid in most of the shell blocks they were substituted into, and two language flows now convey credentials by no route at all.

Review Table

Priority Category Check Status Notes
High Security No hardcoded secrets or credentials Fail Goal largely met for SDK instructions, but the sibling executeCreateAuthConfig path (accessibility.ts:291) still echoes result.data unredacted while the GET path next to it was fixed.
High Security Authentication/authorization checks present Pass No authz surface changed. Note: removing getBrowserStackAuth() drops an implicit fail-fast on missing credentials (see INFO below) — a behavior change, not a bypass.
High Security Input validation and sanitization N/A No user input paths touched.
High Security No IDOR — resource ownership validated N/A No resource lookups changed.
High Security No SQL injection (parameterized queries) N/A No SQL in this codebase.
High Correctness Logic is correct, handles edge cases Fail Findings 1–4: emitted shell blocks are syntactically invalid in bash/PowerShell/cmd; Python and C# flows reference env vars nothing sets; setup command emitted before the export step it depends on.
High Correctness Error handling is explicit, no swallowed exceptions Pass No error paths altered.
High Correctness No race conditions or concurrency issues N/A Pure string generation; the refactor actually reduces shared state by dropping config reads.
Medium Testing New code has corresponding tests Pass One new test file plus a rewritten one; both cover the changed surface.
Medium Testing Error paths and edge cases tested Fail Finding 10: the decoy-env assertions can only fail if a generator reads process.env (none ever did); the 17-combo loop asserts nothing positive and would pass green on empty output. No test covers the actual regression (re-introducing a getBrowserStackAuth-fed parameter).
Medium Testing Existing tests still pass (no regressions) Pass 28 files / 256 tests pass on head SHA.
Medium Performance No N+1 queries or unbounded data fetching N/A No I/O added.
Medium Performance Long-running tasks use background jobs N/A Not applicable.
Medium Quality Follows existing codebase patterns Fail Finding 5: two mutually incompatible placeholder conventions (${BROWSERSTACK_USERNAME} vs <your_browserstack_username>) now coexist for the same artifacts across sdk-utils and appium-sdk.
Medium Quality Changes are focused (single concern) Pass All changes serve the credential-removal goal.
Low Quality Meaningful names, no dead code Fail Finding 7: config: BrowserStackConfig in sdkHandler.ts:17 is now unreferenced (eslint misses it because args: "after-used" and a later param is still used).
Low Quality Comments explain why, not what Fail Finding 8: the ruby.ts comment describes the TDZ hazard as pre-existing when this PR introduced it, and "read lazily" is inaccurate for module constants.
Low Quality No unnecessary dependencies added Pass No dependency changes.

Findings

1. export VAR=<placeholder> is a bash syntax error, not a placeholder

  • File: src/tools/appautomate-utils/appium-sdk/formatter.ts:48
  • Severity: High
  • Reviewer: stack:code-review
  • Issue: Bash parses < as input redirection, so every emitted block of this shape fails to run rather than prompting the user to substitute a value. Verified: bash -n on export BROWSERSTACK_USERNAME=<your_browserstack_username>syntax error near unexpected token 'newline', exit 2. This is the highest-blast-radius instance: formatEnvCommands feeds createEnvStep, which the App Automate java, python, ruby and csharp flows all consume (java.ts:159, python.ts:70 and :94, ruby.ts:86, csharp.ts:63). The same class of bug appears at nodejs.ts:80-81, 155-156, 243-244, 262-263, 281-282 and sdk-utils/bstack/constants.ts:103-104, 313-314, 347-348, 535-536.
  • Suggestion: Quote the placeholder: export BROWSERSTACK_USERNAME="<your_browserstack_username>". (formatter.ts:43-44's setx ... "<...>" form is already quoted and correct — mirror it.)

2. PowerShell reserves < — the PowerShell block is a parse error

  • File: src/tools/sdk-utils/bstack/constants.ts:353
  • Severity: High
  • Reviewer: stack:code-review
  • Issue: $env:BROWSERSTACK_USERNAME=<your_browserstack_username> fails with "The '<' operator is reserved for future use."
  • Suggestion: $env:BROWSERSTACK_USERNAME="<your_browserstack_username>".

3. cmd treats < as redirection — the cmd block is a parse error

  • File: src/tools/sdk-utils/bstack/constants.ts:541
  • Severity: High
  • Reviewer: stack:code-review
  • Issue: set BROWSERSTACK_USERNAME=<your_browserstack_username> is invalid in cmd.exe for the same reason as finding 1.
  • Suggestion: set "BROWSERSTACK_USERNAME=<your_browserstack_username>".

4. Python and C# SDK flows now deliver credentials by no route at all

  • File: src/tools/sdk-utils/bstack/constants.ts:49 (also :149, :218)
  • Severity: High
  • Reviewer: stack:code-review
  • Issue: These emit ${BROWSERSTACK_USERNAME} / ${BROWSERSTACK_ACCESS_KEY}, but neither instruction set contains a step telling the user to export those variables, and getSDKPrefixCommand returns "" for both languages (commands.ts:89-100 handles only nodejs and java). The generated browserstack.yml (configUtils.ts:25-26) defers to the same unset vars. Verified by reading constants.ts:36-51: the Python setup string is pip-install followed directly by browserstack-sdk setup --username "${BROWSERSTACK_USERNAME}" with no export step anywhere before it. Net result for a Python user: browserstack-sdk setup --framework "pytest" --username "" --key "", then an unresolvable browserstack.yml. Before this PR both carried real values.
  • Suggestion: Add a properly-quoted env-export step at the head of generatePythonFrameworkInstructions, csharpCommonInstructions, and csharpPlaywrightCommonInstructions.

5. Ordering: the ${VAR}-consuming command is emitted before the step that sets VAR

  • File: src/tools/sdk-utils/bstack/sdkHandler.ts:86
  • Severity: High
  • Reviewer: stack:code-review
  • Issue: sdkSetupCommand ("Install BrowserStack SDK") is pushed at :86-97, before frameworkInstructions.setup at :99-113 — but the setup command is the consumer of the env vars (nodejs: commands.ts:25; java Maven: commands.ts:51-52, 62-63), while the export step lives in constants.ts:310-314 / :101-105. A user following the steps in order runs the credential-consuming command before setting the credentials. percy-bstack/handler.ts:51-95 has the same inversion.
  • Suggestion: Fold the export step into getSDKPrefixCommand's output, or reorder the pushes so the export step precedes the setup command.

6. ${BROWSERSTACK_USERNAME} does not expand on Windows

  • File: src/tools/sdk-utils/bstack/commands.ts:51
  • Severity: Medium
  • Reviewer: stack:code-review
  • Issue: In getMavenCommandForWindows, ${VAR} is literal text in cmd.exe (which needs %VAR%) and is PowerShell variable syntax — not env — in PowerShell, resolving to $null so Maven receives -DBROWSERSTACK_USERNAME="". The whole purpose of branching on isWindows here is defeated.
  • Suggestion: Emit %BROWSERSTACK_USERNAME% / %BROWSERSTACK_ACCESS_KEY% in the cmd branch, or switch this branch to the literal-placeholder convention.

7. The accessibility redaction is one-sided — the create path still leaks the password

  • File: src/tools/accessibility.ts:291
  • Severity: Medium
  • Reviewer: stack:code-review
  • Issue: The PR redacts password on the GET path (:328-334) but executeCreateAuthConfig still does JSON.stringify(result.data, null, 2) on the same AuthConfigResponse.data shape with the same optional password field (accessiblity-utils/auth-config.ts:5-16). If the create endpoint echoes the field back, this is the identical leak the PR set out to close — and the line immediately above it already redacts the request args, so the hazard was clearly in mind. Verified by reading accessibility.ts:280-292.
  • Suggestion: Extract a redactAuthConfig(data) helper and apply it on both paths. Adjacent (outside this diff, same goal): accessiblity-utils/auth-config.ts:118 logs the whole response and :122-124 logs err.response.data — both can carry the site password into logs, against .claude/rules/security.md "Never log credentials".

8. Two mutually incompatible placeholder conventions for the same artifact

  • File: src/tools/sdk-utils/bstack/configUtils.ts:25

  • Severity: Medium

  • Reviewer: stack:code-review

  • Issue: The PR introduces both styles with no stated rule, and they disagree on the same generated file:

    Artifact sdk-utils (web) appium-sdk (app)
    browserstack.yml configUtils.ts:25${BROWSERSTACK_USERNAME} config-generator.ts:38<your_browserstack_username>
    SDK setup CLI constants.ts:49,149,218${...} csharp.ts:69,73, python.ts:86<your_...>
    Maven -D flags commands.ts:51${...} java.ts:89,119<your_...>

    One of the two is wrong for any given consumer.

  • Suggestion: Pick one convention and centralise it in the new USERNAME_PLACEHOLDER / ACCESS_KEY_PLACEHOLDER constants. Since the appium flows do emit export steps via createEnvStep, ${BROWSERSTACK_USERNAME} is the coherent choice there and matches the SDK's documented browserstack.yml interpolation.

9. Dead parameter left behind

  • File: src/tools/sdk-utils/bstack/sdkHandler.ts:17
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: config: BrowserStackConfig is no longer referenced anywhere in runBstackSDKOnly's body — its only former use was the deleted getBrowserStackAuth call. ESLint misses it because no-unused-vars defaults to args: "after-used" and the later isPercyAutomate param is still used. Its sibling runPercyWithBrowserstackSDK did drop the param, so the two handlers are now inconsistent.
  • Suggestion: Drop it and update handler.ts:34 and handler.ts:129, or add a comment stating it is retained for signature stability.

10. The ruby.ts comment is misleading, and the import cycle should just be broken

  • File: src/tools/appautomate-utils/appium-sdk/languages/ruby.ts:14
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: The TDZ hazard is real but newly introduced by this PR — the old top-level const username = "${process.env.BROWSERSTACK_USERNAME}" was a plain string literal with no dependency on the barrel. The cycle is languages/ruby.ts../index.js./instructions.js (index.ts:2) → ./languages/ruby.js (instructions.ts:11), with export * from "./constants.js" at index.ts:9 sitting after the instructions subgraph. Also, "Read lazily" is inaccurate — nothing is read; these are module constants.
  • Suggestion: Import directly from ../constants.js instead of the barrel. That removes the cycle entirely and lets the consts return to module scope. csharp.ts:7-8, java.ts:7-8, python.ts:9-10 route through the barrel too and are safe only because they're referenced inside function bodies — a latent trap for the next editor.

11. Test env mutation without restore

  • File: tests/tools/sdk-utils-commands.test.ts:17
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: beforeEach sets process.env.BROWSERSTACK_USERNAME / BROWSERSTACK_ACCESS_KEY to decoys and never restores them — the existing afterEach (:7-11) only restores process.platform. The prior version deleted the vars (also unrestored, but benign); this version leaves fake credentials in the worker env for whatever runs after it.
  • Suggestion: Mirror the afterEach restore in tests/tools/appautomate-sdk-credentials.test.ts:23-35, which gets this right.

12. Both new/rewritten test files under-assert

  • File: tests/tools/appautomate-sdk-credentials.test.ts:39
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: The not.toContain(DECOY_*) guards can only fail if an emitter reads process.env directly, which none of them ever did. The regression these tests should catch is re-introducing a username/accessKey parameter fed from getBrowserStackAuth(config) — nothing covers that. Worse, the 17-combo loop asserts nothing about content, so it would pass green if every generator returned "".
  • Suggestion: Add a positive assertion inside the loop (toContain(USERNAME_PLACEHOLDER) under the chosen convention). All 17 combos return non-empty text today (prefix commands 203–2650 chars, project instructions 66–1292 chars), so this is safe to add.

Informational (non-gating)

  • Removed auth validation is a real behavior change. get-auth.ts:6-8 throws when credentials are absent. Dropping getBrowserStackAuth(config) from bstack/sdkHandler.ts and percy-bstack/handler.ts removes an implicit fail-fast, so runTestsOnBrowserStack / setUpPercyHandler (AUTOMATE branch) will now render a full setup guide on a server with no credentials configured instead of erroring. Probably intended — but the PR is titled "Security changes" with an empty body, which also misses the conventional-commit and PR guidance in .claude/rules/commit-conventions.md. Please state this in the description.
  • src/tools/testmanagement-utils/rich-text.ts:1 is trailing-whitespace-only inside a comment. Confirmed inert — the TM_ALLOWED_TAG regex and all logic are byte-identical.
  • The accessibility redaction is type-safe. AuthConfigResponse.data is a flat object (auth-config.ts:5-16), not the array the old test mocked, so {...result.data} cannot degrade into numeric keys. The updated mock in tests/tools/accessibility.test.ts:41-47 now matches the declared type — an improvement.
  • Env-var discipline holds. The .claude/rules/security.md carve-out for src/tools/*/appium-sdk/languages/*.ts emitting literal process.env.BROWSERSTACK_* still applies to nodejs.ts:96-97; no new violation.
  • Pre-existing, untouched but inside a modified hunk: appium-sdk/languages/csharp.ts:60 uses console.warn instead of logger.

Verdict: FAIL — the credential-removal goal is right, but the substituted placeholders produce shell blocks that do not parse, and the Python/C# flows now supply no credentials at all.

…-auth password

Follow-up to the credential-removal changes, resolving the automated PR
review findings:

- Use one consistent, quoted placeholder convention (<your_browserstack_*>)
  everywhere. The previous mix of ${BROWSERSTACK_USERNAME} env-refs and bare
  <placeholder> values produced shell that did not parse: `export VAR=<x>` is
  a redirection error in bash, `<` is reserved in PowerShell, and cmd treats
  it as redirection. All export/set/$env/CLI/yml/Maven spots now emit quoted
  placeholders the user fills in — valid across bash, cmd, and PowerShell, and
  removing the Python/C# flows' reliance on env vars nothing set.
- Redact the site password on the accessibility CREATE path too (not just
  GET), plus the internal response logging, via a shared
  redactAuthConfigResponse() helper.
- Break the appium-sdk import cycle at the source: constants.ts imports enums
  from types.js instead of the index barrel, so ruby.ts can read the
  placeholder constants at module scope again (removes the TDZ workaround).
- Document the retained (now-unused) config param in runBstackSDKOnly.
- Tests: restore mutated env in afterEach; add positive assertions so the
  suites cannot pass green on empty output; cover the create-path redaction.

Verified: extracted and parsed every emitted bash block (63) — no syntax
errors from credential lines; 0 unquoted env-set placeholders remain.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@SavioBS629 SavioBS629 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review (automated) — 8 inline finding(s). Full report in the PR comment below. Verdict: Passed.

Run the following command to setup browserstack sdk:
\`\`\`bash
npx browserstack-node-sdk setup --username ${username} --key ${accessKey}
npx browserstack-node-sdk setup --username "<your_browserstack_username>" --key "<your_browserstack_access_key>"

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] Nothing tells the user to substitute the placeholder

Across all 72 rendered credential-bearing blocks there is no substitution guidance — confirmed by grep, the only "Replace" text in the emitted output is configUtils.ts:28's TODO about projectName/buildName.

The consumer here is usually a coding agent, and these steps are prefixed "DO NOT SKIP ANY STEP … Each step is compulsory". The agent will run this line verbatim — and browserstack-node-sdk setup writes those values into browserstack.yml, baking the placeholder into project config. Tests then 401 at run time with nothing pointing at the cause. That trades the previous round's loud syntax error for a silent misconfiguration.

Suggestion: either append one line to each env/setup step — "Replace <your_browserstack_username> and <your_browserstack_access_key> with the values from https://www.browserstack.com/accounts/profile/details" — or put the export step first (properly fixing the earlier ordering finding) and reference "$BROWSERSTACK_USERNAME" / %BROWSERSTACK_USERNAME% / $env:BROWSERSTACK_USERNAME per shell.

Reviewer: stack:code-review

* Redacts the stored site password before an auth-config response is returned
* to the LLM/tool caller or written to logs. Never expose the raw password.
*/
export function redactAuthConfigResponse(

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] Redaction is shallow and single-key while callers stringify the whole object

This helper rewrites only the top-level data.password, but both callers (accessibility.ts:294-298 and :337-341) JSON.stringify the entire data object. The declared AuthConfigResponse.data includes username — the customer's site login — alongside password, so the site username is still echoed to the MCP client, and any key the API returns beyond the nine declared ones is dumped with no TypeScript warning.

The create request nests site credentials under authData: { username, password, ... } (:89-93, :149-153). If the response echoes that shape, the early return if (!response?.data?.password) never fires and the raw password passes straight through.

Suggestion: stop dumping the object — emit an explicit allowlist ({ id, name, type, url }), which also satisfies rules/tool-design.md § "Response payloads: trim before returning, never echo". If a generic helper is preferred, make it recurse, match on key name (/pass(word)?|secret|token|accesskey/i), and handle arrays and null explicitly.

Reviewer: stack:code-review

`\`\`\`bash
export BROWSERSTACK_USERNAME=${username}
export BROWSERSTACK_ACCESS_KEY=${accessKey}
export BROWSERSTACK_USERNAME="<your_browserstack_username>"

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] Placeholder duplicated as a bare literal

USERNAME_PLACEHOLDER / ACCESS_KEY_PLACEHOLDER exist in appium-sdk/constants.ts:28-29, but this file hardcodes the literal string at :80,81,96,97,155,156,243,244,262,263,281,282, and all 25 occurrences under sdk-utils/bstack/ are bare literals too — ~34 sites total.

The values agree today, which is exactly how the convention divergence flagged in the previous review arose in the first place.

Suggestion: promote the two constants to a module both trees import (e.g. src/tools/sdk-utils/common/) and reference them everywhere.

Reviewer: stack:code-review

Comment thread tests/tools/appautomate-sdk-credentials.test.ts Outdated
Comment thread tests/tools/sdk-utils-commands.test.ts Outdated
if (originalPlatform) {
Object.defineProperty(process, "platform", originalPlatform);
}
process.env.BROWSERSTACK_USERNAME = originalUser;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] Env restore writes the string "undefined"

Same issue as tests/tools/appautomate-sdk-credentials.test.ts:33 — assigning an undefined original coerces to the string "undefined" rather than removing the key, so teardown leaves a bogus value behind.

Suggestion: guard the restore, or use vi.stubEnv() / vi.unstubAllEnvs().

Reviewer: stack:code-review

} from "../../src/tools/appautomate-utils/appium-sdk/index";

// Every supported App Automate language/testing-framework combination.
const COMBOS: Array<[string, string[]]> = [

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] Test omits the C# branch and the access-key placeholder

COMBOS covers java/python/nodejs/ruby but not csharp, so getCSharpSDKCommand() — one of the branches that embeds the placeholder into a dotnet browserstack-sdk setup command (csharp.ts:66,71) — is entirely uncovered by this suite.

The loop also asserts only <your_browserstack_username>; the standalone yml test at :82-96 correctly asserts both placeholders.

Suggestion: add ["csharp", ["nunit", "mstest", "xunit", "specflow", "reqnroll"]] to COMBOS, and assert <your_browserstack_access_key> inside the loop as well.

Reviewer: stack:code-review

id: "auth-1",
name: "test",
username: "site-user",
password: "super-secret-site-password",

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] GET mock reshaped from array to object — confirm the real payload

This mock changed from data: [{ id, name }] to a flat object. That matches the declared AuthConfigResponse.data type and is defensible — but the previous mock asserted a list shape.

If the live endpoint can return a list, response.data.password is undefined, redactAuthConfigResponse returns the payload untouched, and this test passes green while production leaks the password.

Suggestion: confirm the endpoint's actual response shape. If a list is possible, handle it — the allowlist approach suggested on auth-config.ts:24 covers this case for free.

Reviewer: stack:code-review

config: BrowserStackConfig,
isPercyAutomate = false,
): Promise<RunTestsInstructionResult> {
void config;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] Signature asymmetry, and breaking exports for the downstream wrapper

void config; keeps the unused param alive here, while the sibling runPercyWithBrowserstackSDK (percy-bstack/handler.ts:17) deleted its own config param outright — two conventions for the same situation.

More consequentially: per CLAUDE.md this package is consumed as a library by the remote MCP wrapper, and this PR removes positional params from exported symbols (getSDKPrefixCommand, getAppSDKPrefixCommand, generateBrowserStackYMLInstructions, generateAppBrowserStackYMLInstructions, getInstructionsForProjectConfiguration, formatEnvCommands, createEnvStep, runPercyWithBrowserstackSDK) plus the exported ConfigMapping.instructions type (common/types.ts:64). Because the removals are positional, a stale caller can silently pass username where appPath is now expected instead of failing to compile. In-repo callers are updated and tsc is green, but the wrapper builds separately.

Suggestion: pick one convention for the unused param, and call the export changes out in the PR body so the wrapper is updated in lockstep.

Reviewer: stack:code-review

@SavioBS629

Copy link
Copy Markdown
Collaborator Author

Claude Code PR Review

PR: #359Head: 04a143dReviewers: stack:code-review

Continues the previous review — changes since 01a8c56 (FULL re-review: 13 files / 219 lines changed, above the delta threshold).

Summary

Removes live BrowserStack credentials from generated SDK setup instructions across the Automate/App Automate/Percy flows and redacts the stored site password in the accessibility auth-config tool responses. The new commit 04a143d addresses the previous review: it quotes every placeholder, redacts the create path, breaks the barrel import cycle properly, and strengthens the tests.

All six High findings from the previous round are genuinely fixed — verified mechanically, not by inspection. npm run build passes at this head (lint, format, 28 files / 256 tests, tsc). Two Medium items remain worth landing before merge.

Review Table

Priority Category Check Status Notes
High Security No hardcoded secrets or credentials Pass The PR's goal is met: no live credential reaches generated instructions, and the auth-config password is now redacted on both the create and get paths. Residual Medium (N1): callers still JSON.stringify the whole data object, so the stored site username and any undeclared response key are echoed.
High Security Authentication/authorization checks present Pass No authz surface changed. Removing getBrowserStackAuth() drops an implicit fail-fast — see the Info item.
High Security Input validation and sanitization N/A No user input paths touched.
High Security No IDOR — resource ownership validated N/A No resource lookups changed.
High Security No SQL injection (parameterized queries) N/A No SQL in this codebase.
High Correctness Logic is correct, handles edge cases Pass The previous round's six shell-syntax and empty-credential failures are resolved. Verified: 40 credential-bearing bash blocks all pass bash -n (0 failures); zero ${BROWSERSTACK_ references remain in emitted command text; the cmd and PowerShell forms use each shell's correct quoting idiom.
High Correctness Error handling is explicit, no swallowed exceptions Pass No error paths altered.
High Correctness No race conditions or concurrency issues N/A Pure string generation; the refactor reduces shared state by dropping per-request config reads.
Medium Testing New code has corresponding tests Pass One new test file plus two updated; both now carry positive assertions.
Medium Testing Error paths and edge cases tested Fail N6: COMBOS omits the csharp branch entirely, leaving getCSharpSDKCommand() — one of the placeholder-embedding paths — uncovered; the loop asserts only the username placeholder, not the access key.
Medium Testing Existing tests still pass (no regressions) Pass 28 files / 256 tests pass at 04a143d.
Medium Performance No N+1 queries or unbounded data fetching N/A No I/O added.
Medium Performance Long-running tasks use background jobs N/A Not applicable.
Medium Quality Follows existing codebase patterns Pass The convention divergence flagged last round is resolved — one placeholder string is now used at all ~40 sites and no third variant was introduced. Hygiene residue in N4.
Medium Quality Changes are focused (single concern) Pass All substantive changes serve the credential-removal goal (N9 notes one unrelated whitespace edit).
Low Quality Meaningful names, no dead code Fail N8: config in sdkHandler.ts:19-22 is still unused, now with void config; to silence it, while its sibling runPercyWithBrowserstackSDK deleted the param outright.
Low Quality Comments explain why, not what Pass The misleading ruby.ts TDZ comment was corrected, and the underlying barrel cycle was genuinely broken rather than papered over.
Low Quality No unnecessary dependencies added Pass No dependency changes.

Findings

Resolved since 01a8c56

  • src/tools/appautomate-utils/appium-sdk/formatter.ts:48 High — export VAR=<placeholder> is a bash syntax error Resolved — now export BROWSERSTACK_USERNAME="<your_browserstack_username>". Re-verified: bash -n exit 0, and all 40 credential-bearing bash blocks across every language/framework combination parse cleanly. < inside double quotes is literal in bash, and the placeholder contains no $, so it neither redirects nor expands.
  • src/tools/sdk-utils/bstack/constants.ts:103 High — same bash syntax error (web SDK) Resolved — quoted at :103, :313, :347, :535.
  • src/tools/sdk-utils/bstack/constants.ts:353 High — PowerShell reserves < Resolved — now $env:BROWSERSTACK_USERNAME="<your_browserstack_username>". The reserved-operator parse error fires only on the bare operator; inside a double-quoted string < is an ordinary character.
  • src/tools/sdk-utils/bstack/constants.ts:541 High — cmd treats < as redirection Resolved — now set "BROWSERSTACK_USERNAME=<your_browserstack_username>", which is the correct cmd idiom (the quotes are consumed, not stored).
  • src/tools/sdk-utils/bstack/constants.ts:49 High — Python flow references env vars nothing sets Resolved by design change — the flow now emits a literal quoted placeholder, so no env expansion is load-bearing and the missing export step is no longer a defect. (Residual consequence tracked as N2.)
  • src/tools/sdk-utils/bstack/constants.ts:149 High — same gap in both C# flows Resolved:149 and :218 both emit literal quoted placeholders.
  • src/tools/sdk-utils/bstack/sdkHandler.ts:86 High — consuming command emitted before the step that sets the var Resolved indirectly — the ordering is unchanged (sdkHandler.ts:89 and percy-bstack/handler.ts:87 still push the setup command first), but since no emitted command reads an env var any more, the inversion no longer produces empty credentials. Noted rather than pruned silently: if the N2 fix takes the env-var route, this ordering must be corrected first.
  • src/tools/sdk-utils/bstack/commands.ts:51 Medium — ${VAR} does not expand on Windows Resolved — zero ${BROWSERSTACK_ occurrences remain in any emitted command text; confirmed by grep across src/.
  • src/tools/accessibility.ts:291 Medium — create path echoed the password unredacted Resolvedaccessibility.ts:294-298 now routes through redactAuthConfigResponse, matching the get path at :337-341; the fix also redacted the previously-unredacted logger.info at auth-config.ts:110-114. (Completeness caveat: N1.)
  • src/tools/sdk-utils/bstack/configUtils.ts:25 Medium — two incompatible placeholder conventions Resolved on value — one string is now used everywhere, no third variant introduced. Hygiene residue tracked as N4.
  • src/tools/appautomate-utils/appium-sdk/languages/ruby.ts:14 Low — misleading TDZ comment Resolved properly — the comment now states the real reason, and appium-sdk/constants.ts imports its enums from ./types.js instead of the ./index.js barrel, genuinely breaking the cycle. types.ts has no imports of its own, so ruby.ts's module-scope constants at :12-13 are safe.
  • tests/tools/appautomate-sdk-credentials.test.ts:39 Low — 17-combo loop asserted nothing positive Resolved — the loop now asserts rendered.length > 0 and toContain("<your_browserstack_username>"), so an all-empty regression fails. Residual gaps in N6.

Still open

N2. Nothing tells the user to substitute the placeholder

  • File: src/tools/sdk-utils/bstack/commands.ts:25 (also constants.ts:49,149,218, appium-sdk/languages/python.ts:86, csharp.ts:66,71)
  • Severity: Medium
  • Reviewer: stack:code-review
  • Issue: Across all 72 rendered credential-bearing blocks there is no substitution guidance. Confirmed by grep: the only "Replace" text in the entire emitted output is configUtils.ts:28's # TODO: Replace these sample values with your actual project details, which refers to projectName/buildName. The consumer of these instructions is usually a coding agent, and the steps are prefixed "DO NOT SKIP ANY STEP … Each step is compulsory" — so the agent runs npx browserstack-node-sdk setup --username "<your_browserstack_username>" ... verbatim. That CLI writes the value into browserstack.yml, baking the placeholder into project config; tests then 401 at run time with nothing pointing at the cause. This trades the previous round's loud syntax error for a silent misconfiguration.
  • Suggestion: Either (a) append one line to each env/setup step — "Replace <your_browserstack_username> and <your_browserstack_access_key> with the values from https://www.browserstack.com/accounts/profile/details"; or (b) put the export step first (properly fixing the prior ordering finding) and have setup commands reference "$BROWSERSTACK_USERNAME" / %BROWSERSTACK_USERNAME% / $env:BROWSERSTACK_USERNAME per shell.
  • Note on severity: rated Medium, not High. A self-describing <your_...> placeholder is the conventional idiom for setup docs, and nothing here fails to parse or leaks a secret — but the agent-consumer path makes it more than cosmetic.

N1. Redaction is shallow and single-key while callers stringify the whole object

  • File: src/tools/accessiblity-utils/auth-config.ts:24
  • Severity: Medium
  • Reviewer: stack:code-review
  • Issue: redactAuthConfigResponse rewrites only the top-level data.password, but its callers JSON.stringify the entire data object. The declared AuthConfigResponse.data includes username (the customer's site login) alongside password — so the site username is still echoed to the MCP client, and any key the API returns beyond the nine declared ones is dumped too, with no TypeScript warning because the type doesn't model them. The create request nests site credentials under authData: { username, password, ... } (auth-config.ts:89-93, 149-153); if the response echoes that shape, the raw password goes straight through — the early return if (!response?.data?.password) would not even fire.
  • Suggestion: Stop dumping the object — emit an explicit allowlist ({ id, name, type, url }), which also satisfies rules/tool-design.md § "Response payloads: trim before returning, never echo". If a generic helper is preferred, make it recurse, match on key name (/pass(word)?|secret|token|accesskey/i), and handle arrays and null explicitly.

N3. Error path logs the unredacted response body

  • File: src/tools/accessiblity-utils/auth-config.ts:122
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: logger.error(\Error creating form auth config: ${JSON.stringify(err?.response?.data)}`). The fix commit redacted the success-path logger.infobut left this one. A 4xx that echoes the submittedauthDatawrites the customer's site password to the log, against.claude/rules/security.md"Never log credentials". Only the form path has such a log —createBasicAuthConfigandgetAuthConfig` have none — so this is a one-line fix.
  • Suggestion: Route it through the same redaction, or log only err?.response?.status plus a message.
  • (Not posted inline: this line falls outside the diff hunks, so GitHub cannot anchor a comment to it.)

N4. Placeholder duplicated as a bare literal at ~34 sites

  • File: src/tools/appautomate-utils/appium-sdk/languages/nodejs.ts:80
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: USERNAME_PLACEHOLDER / ACCESS_KEY_PLACEHOLDER exist only in appium-sdk/constants.ts:28-29, and even within that module nodejs.ts hardcodes the literal at :80,81,96,97,155,156,243,244,262,263,281,282. All 25 occurrences under sdk-utils/bstack/ are bare literals too. The values agree today — which is exactly how the divergence flagged last round arose in the first place.
  • Suggestion: Promote the two constants to a module both trees import (e.g. src/tools/sdk-utils/common/) and use them everywhere.

N5. Env restore writes the string "undefined"

  • File: tests/tools/appautomate-sdk-credentials.test.ts:33 (also tests/tools/sdk-utils-commands.test.ts:25)
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: afterEach now restores — an improvement on last round — but process.env.BROWSERSTACK_USERNAME = originalUser coerces when originalUser is undefined. Verified: node -e 'process.env.FOO=undefined' yields the string "undefined", and tests/setup.ts sets no env, so this is the CI path. Teardown therefore leaves a bogus value where the key should be absent. Nothing breaks today (get-auth.ts doesn't read env), but it is a live trap for any future truthiness check.
  • Suggestion: if (originalUser === undefined) delete process.env.BROWSERSTACK_USERNAME; else process.env.BROWSERSTACK_USERNAME = originalUser; — or use vi.stubEnv() / vi.unstubAllEnvs().

N6. New test omits the C# branch and the access-key placeholder

  • File: tests/tools/appautomate-sdk-credentials.test.ts:9
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: COMBOS covers java/python/nodejs/ruby but not csharp, so getCSharpSDKCommand() — one of the branches that embeds the placeholder into a dotnet browserstack-sdk setup command (csharp.ts:66,71) — is entirely uncovered. The loop also asserts only <your_browserstack_username>; the standalone yml test at :82-96 correctly asserts both.
  • Suggestion: Add ["csharp", ["nunit","mstest","xunit","specflow","reqnroll"]] and assert the access-key placeholder in the loop too.

N7. GET mock reshaped from array to object — confirm the real payload

  • File: tests/tools/accessibility.test.ts:45
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: The getAuthConfig mock changed from data: [{ id, name }] to data: { id, name, username, password }. That matches the declared object type and is defensible — but the previous mock asserted a list shape. If the live endpoint can return a list, response.data.password is undefined, redactAuthConfigResponse returns the payload untouched, and the test passes while production leaks.
  • Suggestion: Confirm the endpoint's actual shape; if a list is possible, handle it — the allowlist approach in N1 solves this for free.

N8. Signature asymmetry, and breaking exports for the downstream wrapper

  • File: src/tools/sdk-utils/bstack/sdkHandler.ts:22
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: runBstackSDKOnly keeps its config param and adds void config; "for signature stability" while runPercyWithBrowserstackSDK (percy-bstack/handler.ts:17) deleted its own — two conventions for the same situation. More consequentially: per CLAUDE.md this package is consumed as a library by the remote MCP wrapper, and this PR removes positional params from exported symbols (getSDKPrefixCommand, getAppSDKPrefixCommand, generateBrowserStackYMLInstructions, generateAppBrowserStackYMLInstructions, getInstructionsForProjectConfiguration, formatEnvCommands, createEnvStep, runPercyWithBrowserstackSDK) and changes the exported ConfigMapping.instructions type (common/types.ts:64). Because the removals are positional, a stale caller can silently pass username where appPath is now expected rather than failing to compile. In-repo callers are all updated (tsc is green), but the wrapper builds separately.
  • Suggestion: Pick one convention for the unused param, and call the export changes out in the PR body so the wrapper is updated in lockstep.

Informational (non-gating)

  • PR body is still empty (carried forward, unresolved). Confirmed: gh pr view 359 returns title Security changes with no body. Two things belong there: (1) removing getBrowserStackAuth() drops an implicit fail-fast, so setupBrowserStackAutomateTests now returns full setup instructions on an unauthenticated server while setupBrowserStackAppAutomateTests still 401s via appium-sdk/handler.ts:32 — an intentional-looking divergence that should be stated; (2) the breaking export changes in N8. This also misses the conventional-commit and PR guidance in .claude/rules/commit-conventions.md.
  • src/tools/testmanagement-utils/rich-text.ts:1 remains a trailing-whitespace-only comment edit, unrelated to the security theme. Harmless, just noise in the diff.

Verified clean

  • No new process.env reads. The occurrences at nodejs.ts:96-97 and constants.ts:375-376 are literal template text emitted into the user's own project — the explicit carve-out in rules/security.md.
  • YAML validity. configUtils.ts:25-26 and config-generator.ts:38-39 emit unquoted userName: <your_browserstack_username>. Safe: < is not in YAML's c-indicator set, so it parses as a plain scalar.
  • setx BROWSERSTACK_USERNAME "<...>" (formatter.ts:43-44) is valid cmd. Pre-existing caveat, not introduced here: setx affects only future shells.
  • getAppUploadInstruction (appium-sdk/utils.ts:46-69) still receives real credentials — correctly, since it performs an actual uploadApp API call and emits only the resulting bs:// URL.
  • Instrumentation (trackMCP on both paths), isError on error returns, and .describe() coverage are untouched.

Verdict: PASS — every High from the previous round is fixed and mechanically verified; the two remaining Mediums (N2 substitution guidance, N1 redaction completeness) should land before merge but do not gate.

…stitution guidance

Follow-up to the previous review round (verdict PASS; these resolve the
remaining Medium/Low items):

- Auth-config responses now return an explicit allowlist ({id, name, type,
  url}) instead of stringifying the whole data object, so the stored site
  username (and any undeclared response key) is no longer echoed to the
  caller — not just the password (N1). The internal redactor now masks
  username too, and the form-auth error log no longer dumps the response
  body (N3).
- Generated setup instructions now begin with an explicit step telling the
  reader to replace <your_browserstack_*> with real credentials, so a coding
  agent running steps verbatim doesn't bake the placeholder into
  browserstack.yml and hit a silent 401 (N2).
- Placeholders now have a single source of truth in
  sdk-utils/common/credentials.ts; appium-sdk/constants.ts re-exports from it
  so the two trees cannot drift (N4).
- Tests: restore env with delete-when-undefined instead of the string
  "undefined" (N5); cover the C# SDK branch and assert the access-key
  placeholder too (N6).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@SavioBS629 SavioBS629 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review (automated) — 7 inline finding(s). Full report in the PR comment below. Verdict: Passed.

const data = response.data;
logger.info(`The data returned from the API is: ${JSON.stringify(data)}`);
logger.info(
`The data returned from the API is: ${JSON.stringify(

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] logger.info still dumps the nested response unredacted

The error path was converted to status-only and the tool responses to a proper allowlist, but this info path still stringifies the whole response through redactAuthConfigResponse — which masks only top-level data.password / data.username (:24-34).

The request body nests credentials as authData: { username, password, ...selectors } (:107-111) — exactly the echo shape flagged last round. If the API echoes it, data.authData.password is logged verbatim, as are the selectors at any depth and any undeclared key.

Failure scenario: createFormAuthConfig("login", { username: "qa@acme.com", password: "S3cret!", ... }); endpoint returns {success:true, data:{id:7, name:"login", type:"form", authData:{username:"qa@acme.com", password:"S3cret!"}}} → the log line contains S3cret!, against .claude/rules/security.md "Never log credentials".

Suggestion: reuse the allowlist here — logger.info(\Auth config created: ${JSON.stringify(safeAuthConfigData(data))}`)— or logdata.idalone.redactAuthConfigResponse` then has no callers and can be deleted.

Reviewer: stack:code-review

`\`\`\`bash
export BROWSERSTACK_USERNAME=${username}
export BROWSERSTACK_ACCESS_KEY=${accessKey}
export BROWSERSTACK_USERNAME="<your_browserstack_username>"

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] Bare placeholder literals survive the shared-module fix

src/tools/sdk-utils/common/credentials.ts is the right fix in the right place, but adoption stopped at the three handlers. Counted by grep: bstack/constants.ts 19, this file 12, bstack/commands.ts 5, bstack/configUtils.ts 2 — 38 bare literals, and nothing under sdk-utils/bstack/ imports the new module at all.

The drift risk that produced the round-1 convention split is unchanged for those sites.

Suggestion: import USERNAME_PLACEHOLDER / ACCESS_KEY_PLACEHOLDER from the shared module across bstack/** and this file.

Reviewer: stack:code-review

export const STEP_DELIMITER = "---STEP---";

export {
USERNAME_PLACEHOLDER,

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] Pass-through re-export shim with a now-false comment

These constants are re-exported from the shared module and are no longer in this file's local scope. It compiles only because nothing in constants.ts references them — but the file header still says it lets other modules "safely read the placeholder constants below at module scope", which is now misleading, and a future edit inside this file that references them will fail to resolve.

Separately, appium-sdk/index.ts does export * from "./constants.js", so the appium barrel now transitively re-exports symbols owned by the sdk-utils tree.

Suggestion: have the ~6 appium consumers (formatter.ts, languages/*.ts) import from ../../sdk-utils/common/credentials.js directly and drop the shim; at minimum fix the comment.

Reviewer: stack:code-review

export const CREDENTIALS_SUBSTITUTION_NOTE =
`Replace ${USERNAME_PLACEHOLDER} and ${ACCESS_KEY_PLACEHOLDER} in the steps below ` +
`with your BrowserStack credentials from https://www.browserstack.com/accounts/profile/details ` +
`(or export them as BROWSERSTACK_USERNAME / BROWSERSTACK_ACCESS_KEY). Do not commit real credentials.`;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] The "or export" branch conflicts with the literal yml placeholders

This offers "or export them as BROWSERSTACK_USERNAME / BROWSERSTACK_ACCESS_KEY", but the yml emitters write literal userName: <your_browserstack_username> into browserstack.yml (bstack/configUtils.ts:25-26, bstack/constants.ts:480-481). An agent that takes the export branch still commits a file containing placeholder text — it works only because the SDK's env vars out-rank the yml.

Suggestion: narrow to "…export them instead of editing browserstack.yml", or have the yml step state that the placeholders may be replaced with ${BROWSERSTACK_USERNAME} / ${BROWSERSTACK_ACCESS_KEY}.

Reviewer: stack:code-review

["python", ["pytest", "robot", "behave", "lettuce"]],
["nodejs", ["jest", "mocha", "cucumberJs", "webdriverio", "nightwatch"]],
["ruby", ["cucumberRuby"]],
["csharp", ["nunit", "mstest", "xunit", "specflow", "reqnroll"]],

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] These rows exercise code unreachable in production, and the comment is false

The access-key assertion was added (:72) and this row does pass — getCSharpSDKCommand() genuinely emits both placeholders, so it isn't vacuous. But SUPPORTED_CONFIGURATIONS.appium.csharp is [] (appium-sdk/types.ts:73), so validateSupportforAppAutomate rejects every appium+csharp request before these generators run. The test reaches them only by calling the generators directly.

The framework names are real enum members, but the header comment "Every supported App Automate language/testing-framework combination" (:8) is untrue, and nothing guards COMBOS against drifting from SUPPORTED_CONFIGURATIONS.

Suggestion: derive COMBOS from SUPPORTED_CONFIGURATIONS.appium and keep the csharp rows in a separate, explicitly-labelled "unsupported but must not leak" block — self-maintaining, and honest about reachability.

Reviewer: stack:code-review

expect(serialized).not.toContain("super-secret-site-password");
expect(serialized).not.toContain("site-user");
// Safe identifying fields are still returned.
expect(serialized).toContain("auth-1");

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] This assertion cannot fail

toContain("auth-1") is meant to prove the allowlisted payload came through, but the response also carries a separate ID: ${result.data?.id} line (accessibility.ts:292) with the same value. The assertion passes even if safeAuthConfigData returned undefined, so only the get-path assertion is load-bearing.

Suggestion: assert on the allowlist shape instead — e.g. that the payload contains "type" and does not contain usernameSelector — so it fails if the allowlist stops emitting.

Reviewer: stack:code-review

const steps: RunTestsStep[] = [];
const authString = getBrowserStackAuth(config);
const [username, accessKey] = authString.split(":");
void config;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] void config asymmetry, and breaking exports still undocumented

The explanatory comment is an improvement, but the asymmetry stands: the sibling runPercyWithBrowserstackSDK took the opposite decision and dropped its config param entirely.

More consequentially, the positional-parameter removals on eight exported symbols (getSDKPrefixCommand, getAppSDKPrefixCommand, generateBrowserStackYMLInstructions, generateAppBrowserStackYMLInstructions, getInstructionsForProjectConfiguration, formatEnvCommands, createEnvStep, runPercyWithBrowserstackSDK) plus the ConfigMapping.instructions type change are still documented nowhere. Per CLAUDE.md this package is consumed as a library by the separately-built remote MCP wrapper — and because the removals are positional, a stale caller silently passes username where appPath is now expected instead of failing to compile.

Suggestion: pick one convention for the unused param, and list the export changes in the PR body so the wrapper is updated in lockstep.

Reviewer: stack:code-review

@SavioBS629

Copy link
Copy Markdown
Collaborator Author

Claude Code PR Review

PR: #359Head: 2e14c0dReviewers: stack:code-review

Continues the previous review — changes since 04a143d (FULL re-review: 10 files / 116 lines changed, above the delta threshold).

Summary

Removes live BrowserStack credentials from generated SDK setup instructions across the Automate/App Automate/Percy flows and locks down the accessibility auth-config tool responses. Commit 2e14c0d addresses the second review: it replaces the auth-config redaction with a true allowlist, adds a credential-substitution step to every emitting entry point, introduces a shared credentials.ts module, and fixes the test teardown.

Both Mediums from the previous round are fixed. npm run build passes at this head (lint, format, 263 tests / 29 files, tsc). One new Medium surfaced: the fix converted the error-log path but left the info-log path dumping the nested response.

Review Table

Priority Category Check Status Notes
High Security No hardcoded secrets or credentials Pass No live credential reaches generated instructions. Tool responses now use a true allowlist ({id, name, type, url?}) on both auth-config paths, dropping the site username, password, all selectors, and any undeclared key. One residual log-path gap at Medium (NEW-1).
High Security Authentication/authorization checks present Pass No authz surface changed. The fail-fast removal remains a documented-nowhere behavior change — see the Info item.
High Security Input validation and sanitization N/A No user input paths touched.
High Security No IDOR — resource ownership validated N/A No resource lookups changed.
High Security No SQL injection (parameterized queries) N/A No SQL in this codebase.
High Correctness Logic is correct, handles edge cases Pass The six shell-syntax/empty-credential failures from round 1 remain fixed; no shell block was modified this round, so that verification still holds. The allowlist is undefined-safe and fail-closed on an unexpected array response.
High Correctness Error handling is explicit, no swallowed exceptions Pass The error path now logs status only and still rethrows a useful message.
High Correctness No race conditions or concurrency issues N/A Pure string generation; credentials.ts introduces no module-level mutable state, so it is multi-tenant safe.
Medium Testing New code has corresponding tests Pass 263 tests across 29 files; the credential-leak suite now carries positive assertions for both placeholders.
Medium Testing Error paths and edge cases tested Fail Two gaps: the csharp combos exercise code unreachable in production (appium.csharp: []), and the create-path assertion at accessibility.test.ts:123 cannot fail.
Medium Testing Existing tests still pass (no regressions) Pass Full suite green at 2e14c0d.
Medium Performance No N+1 queries or unbounded data fetching N/A No I/O added.
Medium Performance Long-running tasks use background jobs N/A Not applicable.
Medium Quality Follows existing codebase patterns Pass A single shared placeholder module now exists in the right architectural home (sdk-utils/common/), with no import cycle. Adoption is incomplete — Low (N4).
Medium Quality Changes are focused (single concern) Pass All substantive changes serve the credential-removal goal; one unrelated whitespace edit remains.
Low Quality Meaningful names, no dead code Fail void config persists in sdkHandler.ts:23; and once the info-log path moves to the allowlist, redactAuthConfigResponse becomes dead (it has no other caller).
Low Quality Comments explain why, not what Fail Two now-false comments: appium-sdk/constants.ts still says the constants are "below" when they are re-exported, and the test's "Every supported … combination" is untrue of the csharp row.
Low Quality No unnecessary dependencies added Pass No dependency changes.

Findings

Resolved since 04a143d

  • src/tools/sdk-utils/bstack/commands.ts:25 Medium — nothing tells the user to substitute the placeholder Resolved. CREDENTIALS_SUBSTITUTION_NOTE is now step 1 of all three credential-emitting entry points: bstack/sdkHandler.ts:24-30, percy-bstack/handler.ts:21-27, appium-sdk/handler.ts:37-44. Verified the coverage argument holds: every remaining placeholder site lives under sdk-utils/bstack/** or appautomate-utils/appium-sdk/**, and those trees are reachable only through getSDKPrefixCommand / getInstructionsForProjectConfiguration / getAppSDKPrefixCommand / getAppInstructionsForProjectConfiguration, whose every caller is one of the three patched handlers. The runPercyAutomateOnly fallback inherits the note via sdkResult.steps. The note also names the exact credentials page, and no existing fenced block or template literal was disturbed.
  • src/tools/accessiblity-utils/auth-config.ts:24 Medium — redaction was shallow and single-key Resolved. safeAuthConfigData (:41-52) returns only {id, name, type, url?}, and both callers use it — accessibility.ts:295 (create) and :338 (get). Site username, site password, all three selectors, and any undeclared key are dropped rather than masked. It returns undefined for missing data, and on an unexpected array response data.id is undefined, so it fails closed.
  • src/tools/accessiblity-utils/auth-config.ts:122 Low — error path logged the unredacted response body Resolved at the site flagged. :140-142 now logs only status ${err?.response?.status ?? "unknown"}. The sibling info path was left behind — reopened as NEW-1 below rather than treated as closed.
  • tests/tools/appautomate-sdk-credentials.test.ts:33 / tests/tools/sdk-utils-commands.test.ts:25 Low — env restore wrote the string "undefined" Resolved. Both files now branch on === undefined and delete the key, with a comment explaining why.
  • tests/tools/accessibility.test.ts:45 Low — GET mock reshaped from array to object Resolved as a risk, not as a test. The mock is still flat and the list-shaped response is still untested, but the allowlist is fail-closed, so the scenario can no longer become a production leak. The residual test weakness is tracked separately below.

Still open

NEW-1. logger.info still dumps the nested response unredacted

  • File: src/tools/accessiblity-utils/auth-config.ts:129
  • Severity: Medium
  • Reviewer: stack:code-review
  • Issue: The fix converted the error path to status-only and the tool responses to an allowlist, but this info path still does JSON.stringify(redactAuthConfigResponse(data)) over the whole response — and redactAuthConfigResponse (:24-34) masks only top-level data.password / data.username. The request body nests credentials as authData: { username, password, ...selectors } (:107-111), which is exactly the echo shape the previous round flagged. If the API echoes it, data.authData.password is written verbatim to logs, as are the selectors at any depth and any undeclared key. Verified by reading both the helper and the call site.
  • Failure scenario: createFormAuthConfig("login", { username: "qa@acme.com", password: "S3cret!", ... }); the endpoint returns { success: true, data: { id: 7, name: "login", type: "form", authData: { username: "qa@acme.com", password: "S3cret!" } } } → the log line contains S3cret!, against .claude/rules/security.md "Never log credentials".
  • Suggestion: Reuse the allowlist here — logger.info(\Auth config created: ${JSON.stringify(safeAuthConfigData(data))}`)— or logdata.idalone. Either wayredactAuthConfigResponse` then has no callers and should be deleted.

N4. 38 bare placeholder literals survive; the bstack tree never imports the shared module

  • File: src/tools/appautomate-utils/appium-sdk/languages/nodejs.ts:80
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: src/tools/sdk-utils/common/credentials.ts is the right fix in the right place, but adoption stopped at the three handlers. Counted by grep: bstack/constants.ts 19, appium-sdk/languages/nodejs.ts 12, bstack/commands.ts 5, bstack/configUtils.ts 2 — 38 bare literals, and nothing under sdk-utils/bstack/ imports the new module at all. The drift risk that produced the round-1 convention split is unchanged for those sites. (No import cycle: credentials.ts imports nothing.)
  • Suggestion: Import USERNAME_PLACEHOLDER / ACCESS_KEY_PLACEHOLDER from ../common/credentials.js across bstack/** and appium-sdk/languages/nodejs.ts.

NEW-2. Pass-through re-export shim with a now-false comment

  • File: src/tools/appautomate-utils/appium-sdk/constants.ts:29
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: USERNAME_PLACEHOLDER / ACCESS_KEY_PLACEHOLDER are re-exported from the shared module and are no longer in this file's local scope. It compiles only because nothing in constants.ts references them — but the file header still says it lets other modules "safely read the placeholder constants below at module scope", which is now misleading, and any future edit inside constants.ts that references them will fail to resolve. Separately, appium-sdk/index.ts does export * from "./constants.js", so the appium barrel now transitively re-exports symbols owned by the sdk-utils tree.
  • Suggestion: Have the ~6 appium consumers (formatter.ts, languages/*.ts) import from ../../sdk-utils/common/credentials.js directly and drop the shim; at minimum correct the comment.

NEW-3. The note's "or export" branch conflicts with the literal yml placeholders

  • File: src/tools/sdk-utils/common/credentials.ts:14
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: The note offers "or export them as BROWSERSTACK_USERNAME / BROWSERSTACK_ACCESS_KEY", but the yml emitters write literal userName: <your_browserstack_username> into browserstack.yml (bstack/configUtils.ts:25-26, bstack/constants.ts:480-481). An agent that takes the export branch still commits a file containing placeholder text; it happens to work only because the SDK's env vars out-rank the yml.
  • Suggestion: Narrow the wording to "…export them instead of editing browserstack.yml", or have the yml step say the placeholders may be replaced with ${BROWSERSTACK_USERNAME} / ${BROWSERSTACK_ACCESS_KEY}.

N6. The csharp combos exercise unreachable paths, and the comment is false

  • File: tests/tools/appautomate-sdk-credentials.test.ts:17
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: The access-key assertion was added (:72) and the csharp row does pass — so getCSharpSDKCommand() genuinely emits both placeholders, not vacuously. But SUPPORTED_CONFIGURATIONS.appium.csharp is [] (appium-sdk/types.ts:73, confirmed), so validateSupportforAppAutomate rejects every appium+csharp request before those generators run. The test reaches them only by calling the generators directly. The frameworks named are real enum members, but the header comment "Every supported App Automate language/testing-framework combination" (:8) is untrue, and nothing guards COMBOS against drifting from SUPPORTED_CONFIGURATIONS.
  • Suggestion: Derive COMBOS from SUPPORTED_CONFIGURATIONS.appium and keep the csharp rows in a separate, explicitly-labelled "unsupported but must not leak" block. That makes the test self-maintaining and honest about reachability.

NEW-4. The create-path assertion cannot fail

  • File: tests/tools/accessibility.test.ts:123
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: expect(serialized).toContain("auth-1") is intended to prove the allowlisted payload came through, but the response also carries a separate ID: ${result.data?.id} line (accessibility.ts:292) containing the same value. The assertion therefore passes even if safeAuthConfigData returned undefined. Only the get-path assertion is load-bearing.
  • Suggestion: Assert on the serialized allowlist shape instead — e.g. that the payload contains "type" and does not contain usernameSelector — so the assertion fails if the allowlist stops emitting.

N8. void config asymmetry, and breaking exports still undocumented

  • File: src/tools/sdk-utils/bstack/sdkHandler.ts:23
  • Severity: Low
  • Reviewer: stack:code-review
  • Issue: void config; now carries an explanatory comment — an improvement — but the asymmetry stands: the sibling runPercyWithBrowserstackSDK took the opposite decision and dropped its config param entirely. More consequentially, the positional-parameter removals on eight exported symbols (getSDKPrefixCommand, getAppSDKPrefixCommand, generateBrowserStackYMLInstructions, generateAppBrowserStackYMLInstructions, getInstructionsForProjectConfiguration, formatEnvCommands, createEnvStep, runPercyWithBrowserstackSDK) plus the ConfigMapping.instructions type change are still documented nowhere. Per CLAUDE.md this package is consumed as a library by the separately-built remote MCP wrapper; because the removals are positional, a stale caller silently passes username where appPath is now expected instead of failing to compile.
  • Suggestion: Pick one convention for the unused param, and list the export changes in the PR body so the wrapper is updated in lockstep.

Informational (non-gating)

  • PR body is still empty (carried forward across all three rounds). Confirmed at this head: title Security changes, body length 0. Two things belong there — the fail-fast removal (setupBrowserStackAutomateTests now returns full setup instructions on an unauthenticated server while setupBrowserStackAppAutomateTests still 401s via appium-sdk/handler.ts:32), and the breaking export changes in N8. This also misses the conventional-commit and PR guidance in .claude/rules/commit-conventions.md.
  • src/tools/testmanagement-utils/rich-text.ts:1 remains a trailing-whitespace-only comment edit, unrelated to the security theme.

Verified clean this round

  • No shell block was modified by 2e14c0d, so round 1's mechanical parse verification (40 bash blocks via bash -n, plus the cmd and PowerShell quoting idioms) still holds.
  • No instruction text anywhere cross-references a hardcoded step number, so the +1 shift from prepending the credentials step breaks nothing.
  • credentials.ts imports nothing — no cycle — and holds no mutable module state, so it is multi-tenant safe per rules/multi-tenant-safety.md.
  • Every import added this round is used; AppSDKInstruction.type: "setup" and RunTestsStep.type: "instruction" are valid members; createStep is genuinely exported through the appium barrel.

Verdict: PASS — both prior Mediums are fixed and verified; one new Medium (the info-log path) and a set of Lows remain, none of which gate.

…ential leak)

The success-path logger.info still stringified the whole API response through
redactAuthConfigResponse, which only masked top-level password/username — so a
nested authData.{password,selectors} echo would write the customer's site
password to logs. Switch it to the safeAuthConfigData allowlist ({id, name,
type, url}), matching the tool responses, and delete the now-unused
redactAuthConfigResponse helper.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@SavioBS629

Copy link
Copy Markdown
Collaborator Author

Claude Code PR Review

PR: #359Head: 91ccfb0Reviewers: fallback inline checklist

Continues the previous review — changes since 2e14c0d (delta: 1 file / 20 lines).

How this run was produced. This is a targeted verification of the single new commit, not a fresh full-reviewer dispatch — no reviewer subagent was invoked for 91ccfb0. The delta is one file and one logical change, and it was checked directly. The carried-forward Low findings below were established by stack:code-review in the previous rounds and are reproduced unchanged; they were not re-derived here.

Summary

91ccfb0 closes the one Medium left open at 2e14c0d: the accessibility auth-config success-path logger.info now routes through the safeAuthConfigData allowlist instead of the shallow top-level mask, and the superseded redactAuthConfigResponse helper is deleted. 1 insertion, 19 deletions.

Review Table

Priority Category Check Status Notes
High Security No hardcoded secrets or credentials Pass The last known credential-exposure path is closed. The info log now emits only {id, name, type, url?}, matching the tool responses, so a nested authData.{password,selectors} echo can no longer reach the logs.
High Security Authentication/authorization checks present Pass Unchanged by this commit. The fail-fast removal remains undocumented — see the Info item.
High Security Input validation and sanitization N/A Not touched.
High Security No IDOR — resource ownership validated N/A Not touched.
High Security No SQL injection (parameterized queries) N/A No SQL in this codebase.
High Correctness Logic is correct, handles edge cases Pass safeAuthConfigData was already verified undefined-safe and fail-closed on an unexpected array response; this commit only changes which function the log line calls.
High Correctness Error handling is explicit, no swallowed exceptions Pass The if (!data.success) throw guard below the log is untouched.
High Correctness No race conditions or concurrency issues N/A Pure logging change.
Medium Testing New code has corresponding tests Pass No new behavior to test; the existing redaction assertions in tests/tools/accessibility.test.ts still cover the tool-response paths.
Medium Testing Error paths and edge cases tested Fail Unchanged from the previous round: the csharp combos exercise code unreachable in production, and the create-path assertion at accessibility.test.ts:123 cannot fail.
Medium Testing Existing tests still pass (no regressions) Pass 263 tests / 29 files pass; tsc --noEmit exit 0.
Medium Performance No N+1 queries or unbounded data fetching N/A No I/O change.
Medium Performance Long-running tasks use background jobs N/A Not applicable.
Medium Quality Follows existing codebase patterns Pass The log path now uses the same allowlist as the tool responses — one helper, one contract. Placeholder-constant adoption is still incomplete (Low).
Medium Quality Changes are focused (single concern) Pass One file, one logical change.
Low Quality Meaningful names, no dead code Fail Improved — redactAuthConfigResponse was deleted rather than left orphaned, and zero references remain anywhere in src/ or tests/. Still open: void config in sdkHandler.ts:23.
Low Quality Comments explain why, not what Fail Unchanged: the appium-sdk/constants.ts header still claims the placeholder constants are "below", and the test's "Every supported … combination" comment is untrue of the csharp row.
Low Quality No unnecessary dependencies added Pass No dependency changes.

Findings

Resolved since 2e14c0d

  • src/tools/accessiblity-utils/auth-config.ts:129 Medium — logger.info still dumped the nested response unredacted Resolved. The line is now logger.info(\Auth config API returned: ${JSON.stringify(safeAuthConfigData(data))}`), so the log emits only the allowlisted {id, name, type, url?}and a nestedauthData.{password,selectors}echo can no longer be written. Verified three ways:redactAuthConfigResponseis **fully deleted** (grep acrosssrc/andtests/returns zero references, so it wasn't merely orphaned); this is the onlyJSON.stringifyleft in the file, so no sibling path still dumps a raw payload; and the build is clean (263 tests / 29 files,tsc --noEmit` exit 0). The commit is 1 insertion / 19 deletions — the fix made the file smaller.

Still open (all carried forward unchanged from 2e14c0d)

None of the following are affected by this commit; they are reproduced from the previous review for continuity, with their original attribution.

  • src/tools/appautomate-utils/appium-sdk/languages/nodejs.ts:80 Low38 bare placeholder literals survive the shared-module fix (bstack/constants.ts 19, nodejs.ts 12, bstack/commands.ts 5, bstack/configUtils.ts 2), and nothing under sdk-utils/bstack/ imports sdk-utils/common/credentials.ts at all. The drift risk that produced the original convention split is unchanged for those sites. (stack:code-review)
  • src/tools/appautomate-utils/appium-sdk/constants.ts:29 Low — pass-through re-export shim whose file header still says the constants are "below at module scope". Compiles only because nothing in the file references them; a future in-file reference will fail to resolve. The appium barrel also now transitively re-exports symbols owned by the sdk-utils tree. (stack:code-review)
  • src/tools/sdk-utils/common/credentials.ts:14 Low — the note's "or export them as BROWSERSTACK_USERNAME / BROWSERSTACK_ACCESS_KEY" branch conflicts with the yml emitters, which write literal userName: <your_browserstack_username> into browserstack.yml. An agent taking that branch still commits placeholder text. (stack:code-review)
  • tests/tools/appautomate-sdk-credentials.test.ts:17 Low — the csharp rows pass and do assert real output, but SUPPORTED_CONFIGURATIONS.appium.csharp is [], so production rejects those combos before the generators run. The header comment claiming to enumerate supported combinations is untrue, and nothing guards COMBOS against drift. (stack:code-review)
  • tests/tools/accessibility.test.ts:123 LowtoContain("auth-1") also matches the separate ID: ${result.data?.id} line, so the create-path assertion passes even if safeAuthConfigData returned undefined. Only the get-path assertion is load-bearing. (stack:code-review)
  • src/tools/sdk-utils/bstack/sdkHandler.ts:23 Lowvoid config asymmetry against runPercyWithBrowserstackSDK, which dropped its config param entirely; plus the positional parameter removals on eight exported symbols and the ConfigMapping.instructions type change, consumed by the separately-built remote MCP wrapper, where a stale caller silently passes username where appPath is now expected. (stack:code-review)
  • Info — the PR body is still empty (confirmed at this head: title Security changes, body length 0). Four rounds in, the two items that need documenting are unchanged: the fail-fast removal (setupBrowserStackAutomateTests now returns full setup instructions on an unauthenticated server while setupBrowserStackAppAutomateTests still 401s), and the breaking export changes above. Also misses the PR guidance in .claude/rules/commit-conventions.md. (stack:code-review)

Verdict: PASS — the last Medium is closed and verified; only Lows and the empty PR body remain, none of which gate.

@gaurav-singh-9227
gaurav-singh-9227 merged commit ef337a5 into browserstack:main Aug 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants