HDDS-15244. Add secure-cluster /logLevel integration tests - #11024
HDDS-15244. Add secure-cluster /logLevel integration tests#11024yandrey321 wants to merge 5 commits into
Conversation
|
@jojochuang @smengcl please take a look |
jojochuang
left a comment
There was a problem hiding this comment.
Good, didn't realize it's already working
| } | ||
|
|
||
| static String getLogLevel(String address, String logger) throws Exception { | ||
| HttpURLConnection connection = |
There was a problem hiding this comment.
we need to make sure the socket connections are closed when end.
|
Do we need a test for authenticated but not admin user? |
| <div class="alert alert-info" role="alert"> | ||
|
|
||
| Under the hood `ozone insight` uses HTTP endpoints to retrieve the required information (`/conf`, `/prom` and `/logLevel` endpoints). It's not yet supported in secure environment. | ||
| Under the hood `ozone insight` uses HTTP endpoints to retrieve the required information (`/conf`, `/prom` and `/logLevel` endpoints). In a secure (Kerberos) cluster these endpoints are protected by SPNEGO: run `kinit` first, and the requesting user must be an Ozone administrator (`ozone.administrators`). `ozone insight` negotiates SPNEGO automatically. |
There was a problem hiding this comment.
that's not exactly right. Admin rights are required only by /logLevel. /conf requires an additional hadoop.security.instrumentation.requires.admin set to true to be that. And /prom accepts any auhenticated user.
| <div class="alert alert-info" role="alert"> | ||
|
|
||
| Under the hood `ozone insight` uses HTTP endpoints to retrieve the required information (`/conf`, `/prom` and `/logLevel` endpoints). It's not yet supported in secure environment. | ||
| Under the hood `ozone insight` uses HTTP endpoints to retrieve the required information (`/conf`, `/prom` and `/logLevel` endpoints). In a secure (Kerberos) cluster these endpoints are protected by SPNEGO: run `kinit` first, and the requesting user must be an Ozone administrator (`ozone.administrators`). `ozone insight` negotiates SPNEGO automatically. |
There was a problem hiding this comment.
It looks like ozone insight config does not support SPNEGO right now. It should use InsightHttpUtils.openConnection just like ozone insight metrics and ozone insight log already do to allow auth with SPNEGO.
There was a problem hiding this comment.
should we document this behavior for 'ozone insight config' and open a separate JIRA to fix it?
There was a problem hiding this comment.
Yes, let's file a JIRA to fix that.
What changes were proposed in this pull request?
HDDS-15244 reported that the
/logLevelHTTP endpoint is inaccessible in secureclusters (permanent 403 / SPNEGO replay). Investigation showed the endpoint is
not broken on master:
ozone.http.filter.initializers=org.apache.hadoop.security.AuthenticationFilterInitializer(see
compose/ozonesecureandSecuringOzoneHTTP.md). That initializer registersthe SPNEGO
AuthenticationFilterasauthentication.addDefaultServlets()registers/logLevelviaaddServlet(...), which callsaddFilterPathMapping("/logLevel", …)— mapping the auth filter onto/logLevelexactly once.
/logLevelchallenges unauthenticated requests (401) and returns 200 to anauthenticated Ozone admin, gated by
hasAdministratorAccess.This PR:
and secure (SPNEGO/Kerberos + MiniKdc) clusters, covering OM, SCM and DN:
TestLogLevelEndpointInsecure— GET/SET/logLevelreturn 200 and reflect the level change.TestLogLevelEndpointSecure— authenticated admin GET/SET succeed over SPNEGO;unauthenticated requests are rejected (401/403). The secure test sets
ozone.http.filter.initializers=AuthenticationFilterInitializer, mirroring a realsecure cluster, and restores global login-user /
jdk.http.auth.*system statearound each request.
LogLevelEndpointTestUtil— shared helpers for both suites.Observability.md/Observability.zh.md), whichstated
/logLevelis "not yet supported in secure environment". These endpoints workover SPNEGO: run
kinitfirst, the requesting user must be an Ozone administrator, andozone insightnegotiates SPNEGO automatically (since HDDS-13883).No change to
ozone insight— SPNEGO client support already exists.What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15244
How was this patch tested?
CI: https://github.com/yandrey321/ozone/actions/runs/31817735986/job/94826812059
New integration tests, run locally:
TestLogLevelEndpointInsecure— 6/6 pass (GET/SET on OM, SCM, DN).TestLogLevelEndpointSecure— 9/9 pass (admin GET/SET over SPNEGO on OM, SCM, DN;unauthenticated requests rejected).
checkstyle:checkclean on the touched modules.