Add Azure Linux support to MongoDB/YCSB workload (runtime package-manager detection)

[moprashant]

Summary

Adds Azure Linux 3 (Mariner) support to the MongoDB/YCSB workload, which previously ran on Ubuntu only. Also fixes two product bugs in MongoDBServerExecutor that prevented the server from starting on any RPM-based distribution.

Note for reviewers: this is one of two alternative implementations. The companion PR (mongodb-azurelinux-distro-filter) achieves the same outcome using declarative SupportedLinuxDistributions filtering. Only one should be merged — they conflict in PERF-MONGODB-YCSB.json. Both share the identical C# fixes. This PR is the lower-footprint option: the profile's scenario list stays byte-identical to main.

Product fixes (MongoDBServerExecutor.cs)

These are genuine bugs, not Azure-Linux-specific workarounds:

1. Hardcoded mongodb service account. The data directory was chowned to a hardcoded mongodb user. RPM packages create the account as mongod, so mongod could not write to its own dbPath and failed to start. Now resolved at runtime per package family (mongodb on Debian/Ubuntu, mongod on RPM).

2. MongoDB port never opened on the local firewall. Ubuntu's default iptables INPUT policy is ACCEPT, so this was silently unnecessary there. Azure Linux defaults to DROP, so the client could never reach the server. InitializeAsync now opens the port via the existing IFirewallManager.

Profile changes (PERF-MONGODB-YCSB.json)

• Repository setup steps detect the available package manager (apt / dnf) at runtime, so the scenario list is unchanged from main.
• Repository setup is split into atomic dependency steps (key download, dearmor, repo registration, package-list refresh) rather than one combined step, so failures point at the exact operation.
• Adds lshw and gnupg2, which Ubuntu images preinstall but Azure Linux does not.
• Adds -p mongodb.upsert=true to the Read_Latest and Short_Range_Scan scenarios (see below).

Fixes pre-existing duplicate-key insert failures

Read_Latest (workloadd) and Short_Range_Scan (workloade) both perform 5% run-phase inserts. YCSB seeds both insert sequences at recordcount, so whichever runs second collides on every insert:

E11000 duplicate key error collection: ycsb.usertable index: _id_

The error count matched INSERT-FAILED-Operations exactly (3445/3445). This is pre-existing on main and reproduces identically on Ubuntu — it is not introduced or caused by Azure Linux. mongodb.upsert=true is the documented remedy for a partially loaded data set.

Behavioural note: colliding inserts now become updates, so INSERT latency is measured where it previously reported 0 successful ops. RecordCount, metric names, the operation mix, and scenario names are unchanged.

Testing

Unit tests (new, in MongoDBServerExecutorTests.cs):

• MongoDBServerExecutor_ConfigureDisk_ResolvesTheMongoDBServiceUserForThePlatform
• MongoDBServerExecutor_InitializeAsync_OpensTheMongoDBPortOnTheLocalFirewall

92/92 MongoDB tests pass against current main.

End-to-end, two-VM client/server (Standard_D8s_v5):

Distro	Result
Azure Linux 3	Exit Code 0 — 7/7 scenarios, 247 metric rows, 0 failed operations
Ubuntu 24.04	Exit Code 0 — 7/7 scenarios, 0 failed operations

Azure Linux throughput (ops/sec): Initialize 4439.4 · Update_Heavy 2629.7 · Read_Mostly 3501.0 · Read_Only 4205.2 · Read_Latest 7115.9 · Short_Range_Scan 71.4 · Read_Modify_Write 2386.7

Ubuntu regression risk was explicitly exercised, not just reasoned about: the firewall rule was flushed before the Ubuntu run so the new code path genuinely executed, and it completed without throwing. Measured data-directory ownership confirmed mongodb on Ubuntu and mongod on Azure Linux.

Unsupported package manager fallback

The else branch that fails fast when no supported package manager is present has been verified explicitly.

Each step's Command string was extracted from the committed profile and executed against a sandbox PATH containing every normal system tool (1,585 executables — bash, curl, tee, grep, etc. all present) but with apt-get, apt, dnf, tdnf, yum and zypper removed. The sandbox asserts both halves of that condition before running anything, so the test cannot silently pass for the wrong reason.

All six steps exit 1 and emit the intended diagnostic on stderr:

PASS  MongoGPGKey            (exit=1)  Unsupported package manager. MongoDB repository setup requires apt-get or dnf.
PASS  MongoGPGDearmor        (exit=1)  Unsupported package manager. MongoDB repository setup requires apt-get or dnf.
PASS  MongoAddRepository     (exit=1)  Unsupported package manager. MongoDB repository setup requires apt-get or dnf.
PASS  MongoUpdatePackageList (exit=1)  Unsupported package manager. MongoDB repository setup requires apt-get or dnf.
PASS  MongoInstallServer     (exit=1)  Unsupported package manager. MongoDB installation requires apt-get or dnf.
PASS  MongoInstallClient     (exit=1)  Unsupported package manager. MongoDB installation requires apt-get or dnf.

PASSED: 6   FAILED: 0

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[moprashant]

Alternative implementation of the same feature: #784 (declarative \SupportedLinuxDistributions\ filtering). These two PRs conflict in \PERF-MONGODB-YCSB.json\ — only one should be merged. Happy to close whichever the team doesn't prefer.

[moprashant]

moprashant please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

• (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

• (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree company="Microsoft"

[moprashant]

Closing out the one open item from the description: the else branch that fails fast when neither apt-get nor dnf/tdnf is present is now verified rather than assumed.

Method: each step's Command string was read straight out of the committed PERF-MONGODB-YCSB.json (not retyped) and executed against a sandbox PATH built by symlinking every executable on the real PATH except the package managers. The harness asserts up front that bash/echo/curl/tee/grep are still reachable and that apt-get/dnf/tdnf are not, so it aborts rather than passing for the wrong reason.

Result - all 6 steps exit 1 with the intended stderr message:

Sandbox contains 1585 executables.
Verified: bash/echo/curl/tee/grep present; apt-get/dnf/tdnf absent.

PASS  MongoGPGKey            (exit=1)
PASS  MongoGPGDearmor        (exit=1)
PASS  MongoAddRepository     (exit=1)
PASS  MongoUpdatePackageList (exit=1)
PASS  MongoInstallServer     (exit=1)
PASS  MongoInstallClient     (exit=1)

PASSED: 6   FAILED: 0

The PR description has been updated accordingly. Happy to contribute the harness as a test asset if the team wants it in-repo - I left it out because the repo's test suite is NUnit and these are profile JSON strings rather than C#.

[Bryan DeYoung (brdeyo)]

Thank you for offering the different options. Per the conversation in Teams, I like your second option #784, the declarative approach. I have approved that PR. If you are ok with it, let's abandon this one and go with that one.

Thanks again for the great work here to try to align the platform for versatility and reusability.

[moprashant]

Closing this in favour of #784.

Per review feedback on this PR, the team selected the declarative approach in #784 (a generic SupportedLinuxDistributions component filter on VirtualClientComponent) over the separate PERF-MONGODB-YCSB-AZURELINUX.json profile proposed here. That avoids maintaining two profiles that must be kept in sync, and the filter is reusable by any component in any profile.

The documentation change from this PR (adding AzureLinux to the supported operating systems in website/docs/workloads/mongodb/mongodb-profiles.md) was the one piece not already covered by #784, so it has been carried over there.

The two product fixes validated here are also in #784:

• MongoDBServerExecutor resolves the mongod service user at runtime (mongod on RPM distros, mongodb on Debian) instead of hardcoding mongodb
• Inbound port 27017 is opened via IFirewallManager, which Azure Linux requires since it default-denies inbound