fix(audit): resolve the client IP from the trusted proxy chain, not the leftmost header

apps/docs/content/docs/en/platform/self-hosting/authentication.mdx

@@ -109,13 +109,21 @@ Both SSO flags are needed: the server-side one grants access, and the `NEXT_PUBL
 
 ## Behind a load balancer
 
-Tell Better Auth which forwarding hops to trust when resolving the client IP:
+Tell Sim which forwarding hops to trust when resolving the client IP:
 
 ```bash
 AUTH_TRUSTED_PROXIES=10.0.0.0/24,192.0.2.10
 ```
 
-Better Auth walks `X-Forwarded-For` right to left, skips these hops, and uses the first untrusted address as the client IP for session records and its own IP-based checks. Use your proxies' actual addresses — a broad private range that also covers client traffic defeats the purpose. See [Security](/platform/self-hosting/security).
+The forwarded chain is walked right to left, skipping these hops, and the first untrusted address becomes the client IP. Use your proxies' actual addresses — a broad private range that also covers client traffic defeats the purpose, and `0.0.0.0/0` matches every hop, so the walk finds no client at all.
+
+This one variable governs every IP-derived behavior in Sim: session records, per-IP rate limits on public endpoints, the `ipAddress` column on audit log entries, and per-webhook IP allowlists.
+
+<Callout type="warn">
+  Leave it unset behind a proxy and Sim resolves no client IP at all. Nothing breaks outright — per-IP rate limits fall back to one shared bucket per endpoint (stricter, not weaker), audit rows record a blank IP, and any webhook configured with an IP allowlist rejects every delivery. Sim will not read the leftmost `X-Forwarded-For` value as a fallback, because a client can set that header itself.
+</Callout>
+
+Sim reads `X-Forwarded-For` first and falls back to `X-Real-IP`. If your ingress sets only `X-Real-IP` (nginx's `proxy_set_header X-Real-IP $remote_addr`), that is enough — but make sure the header is *overwritten* at the edge rather than passed through from the client. See [Security](/platform/self-hosting/security).
 
 ## Disabling authentication entirely
 

apps/docs/content/docs/en/platform/self-hosting/environment-variables.mdx

@@ -30,7 +30,7 @@ import { Callout } from 'fumadocs-ui/components/callout'
 | `REDIS_TLS_SERVERNAME` | TLS SNI override. Required when `REDIS_URL` uses `rediss://` with a bare IP, or the app throws at startup |
 | `NEXT_PUBLIC_SOCKET_URL` | WebSocket URL — defaults to the page origin; set only if realtime is on a separate host |
 | `TRUSTED_ORIGINS` | Comma-separated additional origins to trust for auth (apex + `www`, alias domains) |
-| `AUTH_TRUSTED_PROXIES` | Comma-separated reverse-proxy IPs/CIDRs so the client IP cannot be forged through `X-Forwarded-For` |
+| `AUTH_TRUSTED_PROXIES` | Comma-separated reverse-proxy IPs/CIDRs so the client IP cannot be forged through `X-Forwarded-For`. Governs sessions, per-IP rate limits, audit log IPs, and webhook IP allowlists — behind a proxy, leaving it unset means no client IP resolves at all |
 | `INTERNAL_API_BASE_URL` | Internal URL for server-side self-calls, e.g. `http://sim-app.simstudio.svc.cluster.local:3000`. Required for PII log redaction; defaults to `NEXT_PUBLIC_APP_URL` |
 | `DATABASE_REPLICA_URL` | Read-replica connection string for log listing, audit logs, and dashboard aggregations. Falls back to the primary when unset |
 

apps/docs/content/docs/en/platform/self-hosting/security.mdx

@@ -148,7 +148,22 @@ ALLOW_PRIVATE_DATABASE_HOSTS=true
 
 ## Client IP and forwarded headers
 
-Behind a load balancer, `X-Forwarded-For` is client-controllable. Set `AUTH_TRUSTED_PROXIES` to your proxies' actual addresses so Better Auth resolves the real client IP, and `TRUSTED_ORIGINS` if users reach Sim from more than one origin. Both are covered in [Authentication](/platform/self-hosting/authentication#behind-a-load-balancer).
+`X-Forwarded-For` is appended to by each hop, and the leftmost value is written by the client — so behind a load balancer it is attacker-controlled. Sim never reads it directly. It walks the chain right to left, skips the hops you declare in `AUTH_TRUSTED_PROXIES`, and takes the first address it does not recognize as a proxy. A chain it cannot verify resolves to no IP at all rather than to a guess.
+
+Set `AUTH_TRUSTED_PROXIES` to your proxies' actual addresses, and `TRUSTED_ORIGINS` if users reach Sim from more than one origin. Both are covered in [Authentication](/platform/self-hosting/authentication#behind-a-load-balancer).
+
+What depends on it:
+
+| Surface | With it set | Without it, behind a proxy |
+|---|---|---|
+| Session records | Real client IP on the session row | Blank |
+| Per-IP rate limits on public endpoints | One bucket per client | One shared bucket per endpoint — stricter, and not amplifiable |
+| Audit log `ipAddress` | Real client IP | Blank |
+| Per-webhook IP allowlists | Enforced against the real client | Every delivery rejected |
+
+<Callout type="warn">
+  Do not set it to `0.0.0.0/0` (or `::/0`) to "just make it work". Every hop then counts as a trusted proxy, the walk runs past the end of the chain, and *no* request resolves an IP. Sim logs an error at startup if you do, and likewise if an entry is not a valid IP or CIDR — a typo silently drops that entry.
+</Callout>
 
 ## Restricting who can use the instance
 
@@ -179,7 +194,7 @@ The service bundles ~2.2 GB of spaCy models, so first start takes around three m
 - [ ] Images pinned to an explicit tag or digest on app, realtime, and migrations
 - [ ] TLS terminating at the ingress; HTTP redirected or disabled
 - [ ] `NEXT_PUBLIC_APP_URL` and `BETTER_AUTH_URL` set to the real public origin
-- [ ] `AUTH_TRUSTED_PROXIES` set if behind a load balancer
+- [ ] `AUTH_TRUSTED_PROXIES` set to the ingress addresses if behind a load balancer (not `0.0.0.0/0`)
 - [ ] Signup restricted (`DISABLE_REGISTRATION` or `ALLOWED_LOGIN_DOMAINS`)
 - [ ] `DISABLE_AUTH` **not** set
 - [ ] NetworkPolicy enabled and `ingressFrom` scoped to the ingress controller

apps/sim/.env.example

@@ -19,7 +19,7 @@ BETTER_AUTH_URL=http://localhost:3000
 NEXT_PUBLIC_APP_URL=http://localhost:3000
 # INTERNAL_API_BASE_URL=http://sim-app.default.svc.cluster.local:3000  # Optional: internal URL for server-side /api self-calls; defaults to NEXT_PUBLIC_APP_URL
 # TRUSTED_ORIGINS=https://www.example.com,https://app.example.com  # Optional: comma-separated additional public origins to trust for auth (apex+www, alias domains). Merged into Better Auth trustedOrigins.
-# AUTH_TRUSTED_PROXIES=10.0.0.0/24,192.0.2.10  # Optional: reverse-proxy IPs/CIDRs in front of the app. Better Auth walks x-forwarded-for right to left, skips these hops, and uses the first untrusted address as the client IP (prevents forwarded-header spoofing). Use your proxies' actual addresses, not broad private ranges that also cover clients.
+# AUTH_TRUSTED_PROXIES=10.0.0.0/24,192.0.2.10  # Optional: reverse-proxy IPs/CIDRs in front of the app. Sim walks x-forwarded-for right to left, skips these hops, and uses the first untrusted address as the client IP (prevents forwarded-header spoofing). Governs session IPs, per-IP rate limits, audit log IPs, and webhook IP allowlists. Use your proxies' actual addresses — not broad private ranges that also cover clients, and never 0.0.0.0/0 (matches every hop, so nothing resolves).
 
 # Chat (Optional)
 # COPILOT_API_KEY=                      # Mint one at https://sim.ai. Without it the Sim Chat block, prompt jobs, and Inbox cannot run

apps/sim/app/api/chat/[identifier]/otp/route.test.ts

@@ -201,7 +201,7 @@ describe('Chat OTP API Route', () => {
     }))
 
     requestUtilsMockFns.mockGenerateRequestId.mockReturnValue('req-123')
-    requestUtilsMockFns.mockGetClientIp.mockReturnValue('1.2.3.4')
+    requestUtilsMockFns.mockResolveClientIp.mockReturnValue('1.2.3.4')
 
     mockCheckRateLimitDirect.mockResolvedValue({
       allowed: true,
@@ -342,8 +342,8 @@ describe('Chat OTP API Route', () => {
       expect(headerSet).toHaveBeenCalledWith('Retry-After', '900')
     })
 
-    it('folds spoofed `unknown` client IPs into a single shared bucket', async () => {
-      requestUtilsMockFns.mockGetClientIp.mockReturnValueOnce('unknown')
+    it('folds untrustworthy client IPs into a single shared bucket', async () => {
+      requestUtilsMockFns.mockResolveClientIp.mockReturnValueOnce(null)
       queueDeployment(emailDeployment)
 
       const request = new NextRequest('http://localhost:3000/api/chat/test/otp', {
@@ -355,7 +355,7 @@ describe('Chat OTP API Route', () => {
 
       expect(mockCheckRateLimitDirect).toHaveBeenCalledTimes(2)
       expect(mockCheckRateLimitDirect).toHaveBeenCalledWith(
-        expect.stringMatching(/^chat-otp:ip:.*:unknown$/),
+        expect.stringMatching(/^chat-otp:ip:.*:unresolved$/),
         expect.any(Object)
       )
       expect(mockCheckRateLimitDirect).toHaveBeenCalledWith(

apps/sim/app/api/chat/[identifier]/otp/route.ts

@@ -19,7 +19,7 @@ import {
   OTP_IP_RATE_LIMIT,
   storeOTP,
 } from '@/lib/core/security/otp'
-import { generateRequestId, getClientIp } from '@/lib/core/utils/request'
+import { generateRequestId, getRateLimitIpKey } from '@/lib/core/utils/request'
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
 import { sendEmail } from '@/lib/messaging/email/mailer'
 import { setChatAuthCookie } from '@/app/api/chat/utils'
@@ -35,7 +35,7 @@ export const POST = withRouteHandler(
     const requestId = generateRequestId()
 
     try {
-      const ip = getClientIp(request)
+      const ip = getRateLimitIpKey(request)
       const ipRateLimit = await rateLimiter.checkRateLimitDirect(
         `chat-otp:ip:${identifier}:${ip}`,
         OTP_IP_RATE_LIMIT

apps/sim/app/api/chat/[identifier]/sso/route.ts

@@ -8,7 +8,7 @@ import { parseRequest } from '@/lib/api/server'
 import type { TokenBucketConfig } from '@/lib/core/rate-limiter'
 import { RateLimiter } from '@/lib/core/rate-limiter'
 import { isEmailAllowed } from '@/lib/core/security/deployment'
-import { generateRequestId, getClientIp } from '@/lib/core/utils/request'
+import { generateRequestId, getRateLimitIpKey } from '@/lib/core/utils/request'
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
 import { createErrorResponse, createSuccessResponse } from '@/app/api/workflows/utils'
 
@@ -29,7 +29,7 @@ export const POST = withRouteHandler(
   async (request: NextRequest, context: { params: Promise<{ identifier: string }> }) => {
     const requestId = generateRequestId()
 
-    const ip = getClientIp(request)
+    const ip = getRateLimitIpKey(request)
     const ipRateLimit = await rateLimiter.checkRateLimitDirect(
       `chat-sso:ip:${ip}`,
       SSO_IP_RATE_LIMIT

apps/sim/app/api/chat/utils.test.ts

@@ -197,6 +197,7 @@ describe('Chat API Utils', () => {
 
       const mockRequest = {
         method: 'POST',
+        headers: new Headers(),
         cookies: {
           get: vi.fn().mockReturnValue(null),
         },
@@ -221,6 +222,7 @@ describe('Chat API Utils', () => {
 
       const mockRequest = {
         method: 'POST',
+        headers: new Headers(),
         cookies: {
           get: vi.fn().mockReturnValue(null),
         },
@@ -247,6 +249,7 @@ describe('Chat API Utils', () => {
 
       const mockRequest = {
         method: 'POST',
+        headers: new Headers(),
         cookies: {
           get: vi.fn().mockReturnValue(null),
         },
@@ -291,6 +294,7 @@ describe('Chat API Utils', () => {
 
       const mockRequest = {
         method: 'POST',
+        headers: new Headers(),
         cookies: {
           get: vi.fn().mockReturnValue(null),
         },

apps/sim/app/api/contact/route.ts

@@ -11,7 +11,7 @@ import { env } from '@/lib/core/config/env'
 import type { TokenBucketConfig } from '@/lib/core/rate-limiter'
 import { RateLimiter } from '@/lib/core/rate-limiter'
 import { isTurnstileConfigured, verifyTurnstileToken } from '@/lib/core/security/turnstile'
-import { generateRequestId, getClientIp } from '@/lib/core/utils/request'
+import { generateRequestId, getRateLimitIpKey } from '@/lib/core/utils/request'
 import { getEmailDomain } from '@/lib/core/utils/urls'
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
 import { sendEmail } from '@/lib/messaging/email/mailer'
@@ -53,7 +53,7 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
   const requestId = generateRequestId()
 
   try {
-    const ip = getClientIp(req)
+    const ip = getRateLimitIpKey(req)
     const storageKey = `public:contact:${ip}`
 
     const { allowed, remaining, resetAt } = await rateLimiter.checkRateLimitDirect(

apps/sim/app/api/demo-requests/route.ts

@@ -8,7 +8,7 @@ import { parseRequest } from '@/lib/api/server'
 import { env } from '@/lib/core/config/env'
 import type { TokenBucketConfig } from '@/lib/core/rate-limiter'
 import { RateLimiter } from '@/lib/core/rate-limiter'
-import { generateRequestId, getClientIp } from '@/lib/core/utils/request'
+import { generateRequestId, getRateLimitIpKey } from '@/lib/core/utils/request'
 import { getEmailDomain } from '@/lib/core/utils/urls'
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
 import { sendEmail } from '@/lib/messaging/email/mailer'
@@ -27,7 +27,7 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
   const requestId = generateRequestId()
 
   try {
-    const ip = getClientIp(req)
+    const ip = getRateLimitIpKey(req)
     const storageKey = `public:demo-request:${ip}`
 
     const { allowed, remaining, resetAt } = await rateLimiter.checkRateLimitDirect(

apps/sim/app/api/files/public/[token]/otp/route.ts

@@ -21,7 +21,7 @@ import {
   OTP_IP_RATE_LIMIT,
   storeOTP,
 } from '@/lib/core/security/otp'
-import { generateRequestId, getClientIp } from '@/lib/core/utils/request'
+import { generateRequestId, getRateLimitIpKey } from '@/lib/core/utils/request'
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
 import { sendEmail } from '@/lib/messaging/email/mailer'
 import { resolveActiveShareByToken } from '@/lib/public-shares/share-manager'
@@ -57,7 +57,7 @@ export const POST = withRouteHandler(
     const requestId = generateRequestId()
 
     try {
-      const ip = getClientIp(request)
+      const ip = getRateLimitIpKey(request)
       const ipRateLimit = await rateLimiter.checkRateLimitDirect(
         `file-otp:ip:${ip}`,
         OTP_IP_RATE_LIMIT

apps/sim/app/api/files/public/[token]/sso/route.ts

@@ -7,7 +7,7 @@ import { parseRequest } from '@/lib/api/server'
 import type { TokenBucketConfig } from '@/lib/core/rate-limiter'
 import { RateLimiter } from '@/lib/core/rate-limiter'
 import { isEmailAllowed } from '@/lib/core/security/deployment'
-import { generateRequestId, getClientIp } from '@/lib/core/utils/request'
+import { generateRequestId, getRateLimitIpKey } from '@/lib/core/utils/request'
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
 import { resolveActiveShareByToken } from '@/lib/public-shares/share-manager'
 
@@ -33,7 +33,7 @@ export const POST = withRouteHandler(
   async (request: NextRequest, context: { params: Promise<{ token: string }> }) => {
     const requestId = generateRequestId()
 
-    const ip = getClientIp(request)
+    const ip = getRateLimitIpKey(request)
     const ipRateLimit = await rateLimiter.checkRateLimitDirect(
       `file-sso:ip:${ip}`,
       SSO_IP_RATE_LIMIT

apps/sim/app/api/help/integration-request/route.ts

@@ -5,7 +5,7 @@ import { parseRequest, validationErrorResponse } from '@/lib/api/server'
 import { env } from '@/lib/core/config/env'
 import type { TokenBucketConfig } from '@/lib/core/rate-limiter'
 import { RateLimiter } from '@/lib/core/rate-limiter'
-import { generateRequestId, getClientIp } from '@/lib/core/utils/request'
+import { generateRequestId, getRateLimitIpKey } from '@/lib/core/utils/request'
 import { getEmailDomain } from '@/lib/core/utils/urls'
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
 import { sendEmail } from '@/lib/messaging/email/mailer'
@@ -25,7 +25,7 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
   const requestId = generateRequestId()
 
   try {
-    const ip = getClientIp(req)
+    const ip = getRateLimitIpKey(req)
     const storageKey = `public:integration-request:${ip}`
 
     const { allowed, remaining, resetAt } = await rateLimiter.checkRateLimitDirect(

apps/sim/lib/analytics/profound.ts

@@ -8,7 +8,7 @@
 import { createLogger } from '@sim/logger'
 import { env } from '@/lib/core/config/env'
 import { isHosted } from '@/lib/core/config/env-flags'
-import { getClientIp } from '@/lib/core/utils/request'
+import { resolveClientIp } from '@/lib/core/utils/request'
 import { getBaseDomain } from '@/lib/core/utils/urls'
 
 const logger = createLogger('ProfoundAnalytics')
@@ -102,10 +102,7 @@ export function sendToProfound(request: Request, statusCode: number): void {
       host: getBaseDomain(),
       path: url.pathname,
       status_code: statusCode,
-      ip: (() => {
-        const resolved = getClientIp(request)
-        return resolved === 'unknown' ? '0.0.0.0' : resolved
-      })(),
+      ip: resolveClientIp(request) ?? '0.0.0.0',
       user_agent: request.headers.get('user-agent') || '',
       ...(Object.keys(queryParams).length > 0 && { query_params: queryParams }),
       ...(request.headers.get('referer') && { referer: request.headers.get('referer')! }),

apps/sim/lib/auth/auth.ts

@@ -4,6 +4,12 @@ import { stripe } from '@better-auth/stripe'
 import { db } from '@sim/db'
 import * as schema from '@sim/db/schema'
 import { createLogger } from '@sim/logger'
+import {
+  CLIENT_IP_HEADERS,
+  findMalformedTrustedProxies,
+  isAllTrustingProxyEntry,
+  parseTrustedProxies,
+} from '@sim/security/client-ip'
 import { toError } from '@sim/utils/errors'
 import { type BetterAuthOptions, betterAuth, type User } from 'better-auth'
 import { drizzleAdapter } from 'better-auth/adapters/drizzle'
@@ -141,15 +147,31 @@ if (validStripeKey) {
 }
 
 /**
- * Reverse-proxy hops trusted for forwarded-IP resolution. When configured,
- * Better Auth walks the x-forwarded-for chain right to left, skips these
- * hops, and records the first untrusted address as the session client IP —
- * preventing header spoofing behind multi-hop proxies.
+ * Reverse-proxy hops trusted for forwarded-IP resolution: the chain is walked
+ * right to left, these hops are skipped, and the first untrusted address is the
+ * client. Parsed with the same helper `resolveClientIp` uses so the session's
+ * recorded IP and the one every other caller resolves cannot diverge.
+ */
+const trustedProxies = parseTrustedProxies(env.AUTH_TRUSTED_PROXIES)
+
+/**
+ * Both misconfigurations below leave every request resolving no IP at all, and
+ * neither throws — an operator can only discover them if we say so.
  */
-const trustedProxies = (env.AUTH_TRUSTED_PROXIES ?? '')
-  .split(',')
-  .map((entry) => entry.trim())
-  .filter(Boolean)
+const malformedTrustedProxies = findMalformedTrustedProxies(trustedProxies)
+if (malformedTrustedProxies.length > 0) {
+  logger.error('AUTH_TRUSTED_PROXIES contains entries that are not an IP or CIDR range', {
+    malformedTrustedProxies,
+  })
+}
+
+const allTrustingProxies = trustedProxies.filter(isAllTrustingProxyEntry)
+if (allTrustingProxies.length > 0) {
+  logger.error(
+    'AUTH_TRUSTED_PROXIES trusts every hop, so no request will resolve a client IP — scope it to your ingress ranges',
+    { allTrustingProxies }
+  )
+}
 
 export const auth = betterAuth({
   baseURL: getBaseUrl(),
@@ -198,6 +220,9 @@ export const auth = betterAuth({
   },
   advanced: {
     ipAddress: {
+      // The same header list and proxy set `resolveClientIp` uses, so the
+      // address recorded on a session row is the one every other caller sees.
+      ipAddressHeaders: [...CLIENT_IP_HEADERS],
       ...(trustedProxies.length > 0 ? { trustedProxies } : {}),
     },
   },