Skip to content

[#1103] Add upgrade step syncing missing ScriptingService sub-configurations - #1104

Merged
vharseko merged 3 commits into
OpenIdentityPlatform:masterfrom
vharseko:fix/1103-scripting-service-upgrade
Aug 10, 2026
Merged

[#1103] Add upgrade step syncing missing ScriptingService sub-configurations#1104
vharseko merged 3 commits into
OpenIdentityPlatform:masterfrom
vharseko:fix/1103-scripting-service-upgrade

Conversation

@vharseko

@vharseko vharseko commented Aug 7, 2026

Copy link
Copy Markdown
Member

Summary

Fixes #1103.

PR #1034 (16.1.1) introduced the OAUTH2_ACCESS_TOKEN_MODIFICATION script context and its default global script in the <Configuration> section of scripting.xml. The SMS only registers that section when the whole service is new, and the upgrade framework (UpgradeServiceSchemaStepServiceSchemaModifications) diffs <Schema> only. As a result, instances upgraded from a version that already contained the Scripting Service (e.g. 16.0.6 → 16.1.1) are left without:

  • the scriptContext=OAUTH2_ACCESS_TOKEN_MODIFICATION sub-configuration (incl. its engineConfiguration whitelist),
  • the globalScripts/d22f9a0c-426a-4466-b95e-d0f125b0d5fa default global script,
  • the new context choice values in the ScriptingService schema (attribute modifications are only applied when an UpgradeHelper is registered for the service, and ScriptingService had none).

Meanwhile the new OAuth2 Provider attribute forgerock-oauth2-provider-access-token-modification-script is added on upgrade (via OAuth2ProviderUpgradeHelper) with its default pointing at the missing script. Reading /json/global-config/services?_action=nextdescendents then fails single_choice validation (ScriptChoiceValues finds no script of that context) and the whole Global Services page returns 500.

Changes

  • UpgradeScriptingSubConfigsStep (new, depends on UpgradeServiceSchemaStep): reads the tag-swapped bundled scripting.xml via UpgradeServiceUtils.getServiceDefinitions, navigates to Service[@name='ScriptingService']/Configuration/GlobalConfiguration, recursively compares its sub-configurations with the Scripting Service global config in the data store, and creates any missing ones. Presence is determined via SMSUtils.serviceExists, not a null check — for an absent entry whose name matches its sub-schema name (engineConfiguration, globalScripts) ServiceConfig.getSubConfig returns a non-null config wrapping a non-existent SMSEntry, so a partially applied manual workaround (e.g. ssoadm create-sub-cfg for the context without its engine configuration) is also healed. Entries under globalScripts are created before the script contexts that reference them via defaultScript, so a failure part-way through cannot leave the store in the broken Upgrades to versions >= 16.1.1 do not apply the ScriptingService configuration introduced in 16.1.1 (#1034) #1103 shape. perform() guards against missing parents, re-checks existence before creating (the missing list is captured in initialize()), and reports failure on unchecked exceptions too; initialize() wraps unexpected exceptions into a descriptive UpgradeException instead of aborting the whole upgrade with a bare NPE. The upgrade report counts and lists all created sub-configurations, including descendants of the missing nodes. Existing sub-configurations are left untouched (user-tuned engine whitelists are not overwritten). The generic diff also self-heals installations already upgraded to 16.1.1/16.1.2 on their next upgrade, and automatically covers any script contexts added in future versions. Follows the UpgradeEntitlementSubConfigsStep pattern.
  • ScriptingServiceHelper (new, registered in serverupgrade.properties): syncs the script context choice values of the global defaultScriptContext attribute and the realm-level scriptConfiguration.context attribute with the service definition. The comparison reads the raw <ChoiceValues> of the schema nodes, because AttributeSchemaImpl does not parse them for the type="single" attribute scriptConfiguration.context — a getChoiceValues()-based comparison could never detect that change. The administrator's configured default is preserved (for Global attributes the configured value is persisted as the schema default, so it is grafted onto the new node via updateDefaultValues, same pattern as LoggingUpgradeHelper) — unless it is no longer a valid choice, in which case the bundled default is restored to avoid re-creating the very single_choice 500 this PR fixes.
  • UpgradeServiceUtils.getServiceDefinitions widened to public so the step in the steps.scripting sub-package can reuse it.
  • New report keys in amUpgrade.properties.
  • Removed two dead lines from serverupgrade.properties (an orphaned RestSecurity=...UserSelfServiceHelper entry outside the upgrade.helper continuation, referencing a class that no longer exists, and a duplicate services.to.delete key).

Creation order is safe: the defaultScript attribute of scriptContext validates against the ScriptConstants.GlobalScript enum (GlobalOnly=true), not against the data store.

Known limitations (documented in the step javadoc): attributes of existing sub-configurations are never reconciled with the service definition, and a missing single-instance sub-configuration (such as the whole globalScripts node) cannot be created under a parent that already has other children — unreachable on supported upgrade paths, since all such nodes have existed since OpenAM 13.

Testing

mvn -o -pl openam-upgrade test
# Tests run: 136, Failures: 0, Errors: 0, Skipped: 0 — BUILD SUCCESS
  • UpgradeScriptingSubConfigsStepTest (+ test-scripting.xml resource, mirroring the real service definition) verifies that the missing script context and default global script are created with the expected attributes in the right order (script before the context that references it), that the engine configuration whitelist — including org.forgerock.openam.oauth2.ScriptableAccessToken — is carried through, and that a phantom engineConfiguration under an existing context (non-null getSubConfig result with exists() == false) is detected and created. The id = name and empty-attribute fallbacks are exercised by a clearly marked synthetic fixture node. Also covered: the perform-time skip of an entry created after initialize(), a parent disappearing before perform(), both initialize() early exits, and the not-applicable case when everything is already configured.
  • ScriptingServiceHelperTest verifies that new choice values are added while the administrator's configured default is preserved, that the bundled default is used when none is configured or the configured one is no longer a valid choice, that the type="single" realm context attribute is synced via the raw schema nodes, and that unchanged choice values produce no modification.
  • The generated upgradesteps.properties orders the new step after UpgradeServiceSchemaStep and ScriptingSchemaStep.

…gService sub-configurations

Script contexts and default global scripts defined in the scripting.xml
<Configuration> section are only registered by the SMS when the whole
service is new, so instances upgraded from a version that already
contained the Scripting Service were left without the
OAUTH2_ACCESS_TOKEN_MODIFICATION context and its default global script
introduced in 16.1.1 (OpenIdentityPlatform#1034). The dangling default of
forgerock-oauth2-provider-access-token-modification-script then failed
single_choice validation and broke the Global Services page with a 500.

- UpgradeScriptingSubConfigsStep adds any global sub-configurations
  present in the bundled scripting.xml but missing from the config store
  (script contexts with engine configurations, default global scripts)
- ScriptingServiceHelper keeps the script context choice values of the
  ScriptingService schema in sync with the service definition

@maximthomas maximthomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the fix end to end against a 16.0.6 → 16.1.x store, and it is correct: the step records exactly two entries in document order (context, then script) and writes them through the same code a fresh install uses (CreateServiceConfig.getAttributeValuePairscreateSubConfigEntry), so nothing is dropped relative to a fresh install. Creating defaultScript before the script exists is safe because AttributeValidator short-circuits under installTime=true, and tag swapping resolves @GlobalAccessTokenModificationScriptId@ on the upgrade path. Step ordering, the single-instance addSubConfig guard, and the cost of re-calling getServiceDefinitions were all checked and are non-issues.

Two things to address before merge: the existence check misses two node types, and the new helper silently reverts an administrator setting.

Existence check is unsound for engineConfiguration and globalScripts (major)

openam-upgrade/src/main/java/org/forgerock/openam/upgrade/steps/scripting/UpgradeScriptingSubConfigsStep.java:113

ServiceConfig.getSubConfig(name) does not return null for every missing sub-config. For an absent entry ServiceConfigImpl.getInstance finds no ATTR_SERVICE_ID and derives the sub-schema id from the sub-config name; it returns null only when parentSS.getSubSchema(name) also fails. Where the name equals its sub-schema name — globalScripts (scripting.xml:426) and engineConfiguration (scripting.xml:252,263,326,380) — it returns a non-null phantom wrapping an absent SMSEntry, so the step records nothing.

Concrete case: an operator who applied the manual ssoadm create-sub-cfg workaround for OAUTH2_ACCESS_TOKEN_MODIFICATION but not its engineConfiguration child upgrades — the script whitelist is never created and the report says nothing is missing. The happy path passes only because context names and script UUIDs differ from their sub-schema names, and the test's mock returns null outright.

SMSUtils is already imported:

            ServiceConfig existingConfig = parentConfig.getSubConfig(name);
            if (!SMSUtils.serviceExists(existingConfig)) {

Helper reverts the administrator's global Default Script Context (minor)

openam-upgrade/src/main/java/org/forgerock/openam/upgrade/helpers/ScriptingServiceHelper.java:49-51

Returning attributeFromNewSchema wholesale makes ServiceSchema.replaceChildNode swap the entire <AttributeSchema> node, including <DefaultValues><Value>POLICY_CONDITION</Value></DefaultValues>. For Global attributes the administrator's configured value is the schema default — SmsGlobalSingletonProvider round-trips it through setAttributeDefaults/getAttributeDefaults. Anyone who changed Global Services → Scripting → Default Script Context has it silently reset, with nothing in the upgrade report. This is the only place the PR regresses existing behaviour.

LoggingUpgradeHelper:52-55 does the same job — adding a choice value to a Global attribute that carries a meaningful default — and preserves it. The non-empty guard matters: updateDefaultValues with an empty set removes the element entirely.

    @Override
    public AttributeSchemaImpl upgradeAttribute(AttributeSchemaImpl attributeToUpgrade,
            AttributeSchemaImpl attributeFromNewSchema) throws UpgradeException {

        if (asSet(attributeToUpgrade.getChoiceValues()).equals(asSet(attributeFromNewSchema.getChoiceValues()))) {
            return null;
        }
        // For Global attributes the administrator's configured value is persisted as <DefaultValues>,
        // so carry it over instead of reverting to the value bundled in the WAR.
        Set<String> existingDefaults = attributeToUpgrade.getDefaultValues();
        if (existingDefaults.isEmpty()) {
            return attributeFromNewSchema;
        }
        return updateDefaultValues(attributeFromNewSchema, existingDefaults);
    }

perform() replays stale state and leaks unchecked exceptions (minor)

openam-upgrade/src/main/java/org/forgerock/openam/upgrade/steps/scripting/UpgradeScriptingSubConfigsStep.java:133-146

UpgradeServices runs every step's initialize() before any perform(), so this list is ~29 steps stale when replayed. If an entry appeared meanwhile, addSubConfig throws ServiceAlreadyExistsException (an SMSException) and the upgrade aborts partway with no rollback. Not reachable today, but the re-check is free. Separately, getSubConfig returning null on line 137 NPEs past the SMSException | SSOException handler, bypassing reportEnd("upgrade.failed").

            for (MissingSubConfig missing : missingSubConfigs) {
                UpgradeProgress.reportStart(AUDIT_NEW_SUB_CONFIG_START, missing.name);
                ServiceConfig parentConfig = globalConfig;
                for (String parentName : missing.parentPath) {
                    parentConfig = parentConfig.getSubConfig(parentName);
                    if (parentConfig == null) {
                        throw new UpgradeException("Missing parent configuration for " + missing.getDisplayName());
                    }
                }
                // The list was captured in initialize(); another step may have created the entry since.
                if (SMSUtils.serviceExists(parentConfig.getSubConfig(missing.name))) {
                    DEBUG.message("Scripting Service configuration {} already exists, skipping", missing.name);
                } else {
                    addSubConfig(parentConfig, missing.node);
                }
                UpgradeProgress.reportEnd("upgrade.success");
            }
        } catch (UpgradeException e) {
            UpgradeProgress.reportEnd("upgrade.failed");
            throw e;
        } catch (Exception e) {
            UpgradeProgress.reportEnd("upgrade.failed");
            DEBUG.error("An error occurred while adding missing Scripting Service configurations", e);
            throw new UpgradeException("Unable to add missing Scripting Service configurations", e);
        }

Nits

  • Dead context registration: openam-upgrade/src/main/java/org/forgerock/openam/upgrade/helpers/ScriptingServiceHelper.java:35,42 — both context declarations (scripting.xml:180, :223) are type="single", and AttributeSchemaImpl:620-622 only parses <ChoiceValues> for SINGLE_CHOICE/MULTIPLE_CHOICE/LIST, so getChoiceValues() is null on both sides and the helper can never fire for it. Drop SCRIPT_CONTEXT and trim the class javadoc — only defaultScriptContext does real work.
  • Document-wide <GlobalConfiguration> lookup: .../UpgradeScriptingSubConfigsStep.java:105getElementsByTagName(...).item(0) takes the first match anywhere in the file instead of navigating Service[@name='ScriptingService']/Configuration/GlobalConfiguration. Harmless today, silently wrong if scripting.xml ever holds a second <Service>.
  • Report undercounts: .../UpgradeScriptingSubConfigsStep.java:174,186 — only top-level missing nodes are counted, so engineConfiguration children created by the recursion are never listed; UpgradeScriptingSubConfigsStepTest.java:93 asserts 2 while verifying 3 addSubConfig calls. Needs a separate reporting-only list — adding descendants to missingSubConfigs would double-create, and the report is also rendered pre-upgrade so a perform-time counter will not work.
  • ScriptingServiceHelper is untested: there is no openam-upgrade/src/test/java/org/forgerock/openam/upgrade/helpers/ directory at all.
  • Fixture misses the fallback branches: openam-upgrade/src/test/resources/test-scripting.xml gives every <SubConfiguration> an id and no priority, so the id = name, priority == null and attributes == null paths are never exercised — the real scripting.xml:252 has an empty <SubConfiguration name="engineConfiguration" id="engineConfiguration"/>, which makes getAttributeValuePairs return null.
  • Scope caveat worth a javadoc line: the step only creates missing sub-configurations and never reconciles attributes on existing ones, so a later release adding e.g. a whiteList entry to an existing context's engineConfiguration will not reach upgraded instances.
  • Dead lines in a file already being edited: openam-server-only/src/main/webapp/WEB-INF/template/sms/serverupgrade.properties:49-50 — line 47 ends the upgrade.helper continuation, so RestSecurity=...UserSelfServiceHelper parses as a standalone key nothing reads, and services.to.delete is then redefined identically. Delete both lines; do not repair the continuation — UserSelfServiceHelper does not exist in the tree, so re-joining it would raise ClassNotFoundException in populateUpgradeHelpers and assertInitialized() would abort every upgrade.

…eserve admin defaults

- Detect missing sub-configurations via SMSUtils.serviceExists: for an
  absent entry whose name matches its sub-schema name (engineConfiguration,
  globalScripts) getSubConfig returns a non-null phantom, not null.
- ScriptingServiceHelper: carry over the administrator's configured
  defaultScriptContext default instead of reverting it to the bundled one;
  drop the ineffective "context" attribute registration.
- perform(): guard against missing parents, re-check existence before
  creating, and fail the progress report on unchecked exceptions too.
- Navigate to the ScriptingService GlobalConfiguration node instead of
  taking the first match in the document; count created descendants in the
  upgrade report.
- Tests: stub ServiceConfig#exists, cover the phantom engineConfiguration
  case and ScriptingServiceHelper; exercise the empty sub-configuration
  fallbacks in the fixture.
- Remove dead lines from serverupgrade.properties.
@vharseko

vharseko commented Aug 8, 2026

Copy link
Copy Markdown
Member Author

Thanks for the thorough review — all points addressed in 2e314a7.

Existence check (major): captureMissingSubConfigs now uses !SMSUtils.serviceExists(...), so the non-null phantoms returned for absent entries whose name matches a sub-schema name (engineConfiguration, globalScripts) are treated as missing. The new addsEngineConfigurationWhenContextExistsWithoutIt test covers exactly the manual ssoadm workaround scenario: an existing context whose engineConfiguration resolves to a phantom (exists() == false) gets its engine configuration (incl. the whitelist) created. The step test mocks now stub ServiceConfig#exists to match the real SMS behaviour.

Helper reverts the admin default (minor): ScriptingServiceHelper now carries over attributeToUpgrade.getDefaultValues() via updateDefaultValues — same pattern as LoggingUpgradeHelper — with the non-empty guard, so the bundled default is used only when nothing was configured. Covered by the new ScriptingServiceHelperTest.

perform() stale state / unchecked exceptions (minor): added the null-parent guard (fails with a descriptive UpgradeException), a serviceExists re-check before addSubConfig, and separate catch (UpgradeException) / catch (Exception) handlers so unchecked exceptions also hit reportEnd("upgrade.failed").

Nits:

  • dropped the dead context registration and trimmed the helper javadoc to defaultScriptContext;
  • getGlobalConfigurationNode now navigates Service[@name='ScriptingService']/Configuration/GlobalConfiguration instead of taking the first match in the document;
  • the report is built from a separate reporting-only list that includes descendants of the missing nodes, while perform() keeps creating them by recursion — the step test now expects 3 entries and asserts the OAUTH2_ACCESS_TOKEN_MODIFICATION/engineConfiguration path;
  • ScriptingServiceHelper is now tested (choice-value sync preserving the configured default, bundled default when none is configured, null on no change);
  • test-scripting.xml declares the new context's engine configuration as an empty <SubConfiguration name="engineConfiguration"/> without an id, exercising the id = name and attributes == null fallbacks like the real scripting.xml;
  • added the scope caveat (existing sub-configurations are never reconciled) to the step javadoc;
  • deleted the two dead lines from serverupgrade.properties without re-joining the continuation, since UserSelfServiceHelper no longer exists in the tree.

mvn -o -pl openam-upgrade test: Tests run: 130, Failures: 0, Errors: 0.

@vharseko
vharseko requested a review from maximthomas August 8, 2026 06:05

@maximthomas maximthomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach is right and I verified the load-bearing claims against the source rather than the description: the phantom-ServiceConfig reasoning holds (ServiceConfigImpl.getInstance falls back to the sub-config name as subConfigId, so getSubConfig("engineConfiguration") returns a non-null config over a non-existent entry — a null check would have silently skipped it), and creating globalScripts/d22f9a0c-… is exactly what makes ScriptChoiceValues resolve the OAuth2 attribute and closes the 500. Order safety holds twice over: GlobalOnly=true predates #1034 so it's true of the persisted 16.0.6 schema too, and Upgrade.doUpgrade() sets SYS_PROPERTY_INSTALL_TIME=true, which short-circuits single_choice validation for the whole run. Removal of the two dead serverupgrade.properties lines checks out — UserSelfServiceHelper exists nowhere in the tree and the two services.to.delete lines were byte-identical.

No blockers — the production logic is sound. Two majors to resolve before merge.

Test fixture diverges from the real service definition (major)

openam-upgrade/src/test/resources/test-scripting.xml:38 declares the new context's engine config as empty:

<SubConfiguration name="engineConfiguration"/>

The real node in openam-scripting/src/main/resources/scripting.xml:380 has an id and a 40-value whitelist:

<SubConfiguration name="engineConfiguration" id="engineConfiguration">
    <AttributeValuePair>
        <Attribute name="whiteList"/>
        <Value>org.forgerock.openam.oauth2.ScriptableAccessToken</Value>
        ...

So UpgradeScriptingSubConfigsStepTest.java:109-114 asserts the opposite of production behaviour:

assertThat(engineAttributes.getValue()).isEmpty();

Creating that whitelist is the most consequential thing this step does. If nested-sub-config attribute copying ever regressed, this test stays green and the deployment fails at runtime with SecurityException: Access to Java class "org.forgerock.openam.oauth2.ScriptableAccessToken" is prohibited. (addsEngineConfigurationWhenContextExistsWithoutIt does cover whitelist copying — but for OIDC_CLAIMS, not for the context this PR exists to create.)

Also, the comment at :109 and the PR description both state the real definition declares this element without an id. That isn't true — grep '<SubConfiguration' openam-scripting/src/main/resources/scripting.xml | grep -v 'id=' is empty, i.e. the id = name fallback is dead code in production. The claim was read off the fixture, not the source.

Fix: give the fixture node id="engineConfiguration" and at least org.forgerock.openam.oauth2.ScriptableAccessToken in its whitelist, and assert it is carried through. Keep id-fallback coverage on a separate synthetic node if you want it.

Choice-value sync is only half applied (major)

openam-upgrade/src/main/java/org/forgerock/openam/upgrade/helpers/ScriptingServiceHelper.java:35,41 registers one attribute:

attributes.add("defaultScriptContext");

But #1034 added the OAUTH2_ACCESS_TOKEN_MODIFICATION choice value in two places — the <Global> defaultScriptContext (scripting.xml:37) and the <Organization> sub-schema attribute scriptConfiguration.context (scripting.xml:229). ServiceSchemaModifications.getAttributesModified skips any attribute the helper doesn't name, and getServiceModificationsRecursive applies the same filter when recursing into sub-schemas, so the realm-level choice list stays stale after upgrade.

Nothing throws — that attribute is type="single", and AttributeValidator only enforces choice values for SINGLE_CHOICE/MULTIPLE_CHOICE. But the realm New Script context dropdown won't offer the new context on upgraded instances, which is the same fresh-vs-upgraded divergence #1103 reports, and the PR description claims schema choice values are covered.

Fix:

attributes.add("context");

Safe: the helper matches by attribute name across schemas, so it also fires for globalScript.context (scripting.xml:180) — but that attribute has no <ChoiceValues>, so asSet(null).equals(asSet(null)) short-circuits to return null at ScriptingServiceHelper.java:48 and nothing happens. If this was a deliberate scope decision, please say so rather than silently leaving it.

Nits

  • globalScripts can abort the whole upgrade: globalScripts (scripting.xml:176) and engineConfiguration (:69) are single-instance sub-schemas, and ServiceConfig.addSubConfig (openam-core/src/main/java/com/sun/identity/sm/ServiceConfig.java:341-347) rejects an add when the parent already has any child. An installation missing the whole globalScripts node would fail with a cryptic sms-invalid-add-sub-config. Unreachable on 16.0.6→16.1.x (it's existed since 13.0), but worth a pre-check with a clear message or a javadoc note.
  • Tag swap is untested: the fixture hardcodes UUIDs where the real file uses @GlobalAccessTokenModificationScriptId@, and the tests override getScriptingServiceXML() to bypass UpgradeServiceUtils.getServiceDefinitions entirely. If tag swap ever failed, the step would create a sub-config literally named @GlobalAccessTokenModificationScriptId@ with nothing to catch it.
  • initialize() catch is too narrow: it catches SMSException | SSOException, but Integer.parseInt(priority) (UpgradeScriptingSubConfigsStep.java:199) and a nameless <SubConfiguration> throw NumberFormatException/NPE — which would abort the entire upgrade from the UpgradeServices constructor with no useful message. perform()'s catch (Exception) doesn't cover initialize().
  • Missing null check: UpgradeScriptingSubConfigsStep.java:205 uses createdConfig unchecked; ServiceConfigImpl.getInstance can return null, which would surface as a bare NPE swallowed by catch (Exception) at :184 with no indication of which node failed. Two lines: throw new SMSException("Unable to read newly created " + name).
  • Referrer created before referent: perform() iterates in document order, so the context (with defaultScript=d22f9a0c-…) is created before globalScripts/d22f9a0c-…. Fine today, but a failure between the two leaves the store in exactly the #1103 broken shape. Sorting globalScripts descendants first is free insurance.
  • Fragile mock: UpgradeScriptingSubConfigsStepTest.java:92 relies on createdContextConfig.exists() being unstubbed (Mockito default false) for the perform-time re-check to fall through to creation — the comment at :90 doesn't say that. Stub it explicitly, and add a dedicated test for the skip path (:174-175), which is currently uncovered along with the missing-parent throw and both initialize() early-exit branches.
  • Report over-counts: reportedSubConfigs is frozen in initialize() and includes descendants, while perform() emits UpgradeProgress.reportStart only for top-level nodes — the report claims 3 where the progress log shows 2. Same as UpgradeEntitlementSubConfigsStep, so consistent; just noting it.
  • Helper can persist an invalid default: upgradeAttribute copies the admin's configured default without checking membership in the new schema's choice values. If a future release removes a choice an admin had selected, it writes a default that is no longer valid — reproducing the exact single_choice 500 this PR fixes.
  • Redundant service-definition load: UpgradeServiceUtils.getServiceDefinitions re-reads, tag-swaps and DOM-parses all ~104 service XMLs and re-decrypts the DS bind password; UpgradeServiceSchemaStep.initialize() already did this moments earlier. Precedented (UpgradeAgentServiceStep does the same), so acceptable — but caching it would avoid needing the visibility widening at all.

… full choice-value sync

- test-scripting.xml now mirrors the real service definition: the new
  context's engineConfiguration carries its id and a whitelist including
  ScriptableAccessToken, and the test asserts it is carried through; the
  id = name and empty-attribute fallbacks moved to a marked synthetic node.
- ScriptingServiceHelper: register the context attribute and compare the
  raw <ChoiceValues> of the schema nodes — AttributeSchemaImpl does not
  parse them for type="single", so getChoiceValues()-based comparison can
  never detect the scriptConfiguration.context change; keep the configured
  default only while it remains a valid choice.
- UpgradeScriptingSubConfigsStep: create globalScripts entries before the
  contexts that reference them, broaden the initialize() catch, and fail
  with a clear error when a just-created sub-config cannot be read back;
  document the single-instance sub-schema limitation.
- Tests: cover the perform-time skip path, a parent disappearing between
  initialize() and perform(), both initialize() early exits, the single-
  typed context attribute sync and the no-longer-valid default fallback.
@vharseko

Copy link
Copy Markdown
Member Author

Thanks for the second pass — both majors and the nits are addressed in aa079ff. mvn -o -pl openam-upgrade test: Tests run: 136, Failures: 0, Errors: 0.

Fixture divergence (major): entirely my misreading — "empty" in the first review meant "no attribute pairs", not "no id", and I rewrote the fixture (and the test comment, and the PR description) off that wrong reading. Fixed: the fixture's engineConfiguration again carries id="engineConfiguration" and a whitelist including org.forgerock.openam.oauth2.ScriptableAccessToken, and the test now asserts the whitelist is carried through — so a regression in nested attribute copying fails the build instead of surfacing as a runtime SecurityException. The id = name / empty-attribute fallbacks moved to a separate synthetic node that is explicitly marked as not present in the real service definition. The false "without an id" claim is removed from both the test comment and the PR description.

Half-applied choice-value sync (major): agreed on the problem — getAttributesModified skips attributes the helper doesn't name, and the same filter applies in getServiceModificationsRecursive, so the realm-level list stays stale. One correction on the fix, though: attributes.add("context") alone is a no-op. As the first review itself established when flagging the registration as dead code, AttributeSchemaImpl only parses <ChoiceValues> for SINGLE_CHOICE/MULTIPLE_CHOICE/LIST, so for the type="single" attribute getChoiceValues() returns null on both sides, the comparison sees them as equal and upgradeAttribute returns null — nothing is ever applied. So the helper now registers context and compares the choice values read from the raw schema nodes (getAttributeSchemaNode()), which is also what the apply side persists (modifyAttributesInExistingSchema writes attrs.getAttributeSchemaNode() and recurses into sub-schemas, reaching scriptConfigurations/scriptConfiguration). globalScript.context has no <ChoiceValues> on either side, so it short-circuits to null exactly as you predicted. Covered by syncsChoiceValuesOfSingleTypedRealmContextAttribute, which first asserts getChoiceValues() is null for both schemas to pin down why the raw comparison is required.

Nits:

  • globalScripts abort: documented in the step javadoc as an unreachable-on-supported-paths limitation (single-instance sub-schema under a non-empty parent is rejected by the SMS; all such nodes have existed since 13.0).
  • Tag swap untested: accepted limitation — exercising it would need the full ServerConfiguration bootstrap behind UpgradeServiceUtils.getServiceDefinitions; the same call is exercised by UpgradeServiceSchemaStep on every real upgrade, so a tag-swap failure would not be specific to this step.
  • initialize() catch: broadened to rethrow UpgradeException and wrap any other Exception, so a malformed definition aborts with a useful message instead of an NPE out of the UpgradeServices constructor.
  • createdConfig null check: added — throw new SMSException("Unable to read newly created Scripting Service configuration " + name).
  • Referrer before referent: initialize() now moves globalScripts entries to the front of the list, so default scripts are created before the contexts that reference them; the test verifies the order with InOrder.
  • Fragile mock / uncovered branches: createdContextConfig.exists() is stubbed explicitly with a comment; new tests cover the perform-time skip path (skipsEntryCreatedBetweenInitializeAndPerform), a parent disappearing between initialize() and perform() (failsWhenParentConfigurationDisappearsBeforePerform), and both initialize() early exits.
  • Report vs progress log: left as-is per your note (consistent with UpgradeEntitlementSubConfigsStep).
  • Invalid default: the helper now carries the configured default over only while it is still among the new choice values, otherwise it falls back to the bundled default (revertsToBundledDefaultWhenConfiguredDefaultIsNoLongerAValidChoice).
  • Redundant service-definition load: left as-is per your note (precedented by UpgradeAgentServiceStep).

@vharseko
vharseko requested a review from maximthomas August 10, 2026 06:58
@vharseko
vharseko merged commit 7fb46c2 into OpenIdentityPlatform:master Aug 10, 2026
15 checks passed
@vharseko
vharseko deleted the fix/1103-scripting-service-upgrade branch August 10, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug java Pull requests that update java code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upgrades to versions >= 16.1.1 do not apply the ScriptingService configuration introduced in 16.1.1 (#1034)

2 participants