Resolved load balancer IP in access logs?

[carestad]

Hi

I have tried figuring out how to log a users real IP in the access logs to no avail. I have tried enabling the proxy_ip_parser middleware and setting http.access_logs=true as well. Unfortunately, all IPs seem to resolve to the internal 10.0.0.0/8 IP that the load balancer has and not the actual client IP.

Is this somehow doable? Can I access selected headers in the access logs somehow for instance so I could parse this myself?

[rustatian]

Hey @carestad 👋🏻
Could you please share a header with a real IP address (how it looks like, you may replace real ip with any fake ip).

[carestad]

I don't really have the raw request headers available I'm afraid. I am testing this via Laravel Octane and I am currently writing my own adaption of some of Octane's methods so I can enable proper access logs (and format them the way I want to). My current test is in Kubernetes in Amazon EKS.

But typically EKS sets the usual X-Forwarded-* headers. But the access logs' remote_address always seems to be a private IP in the 10.0.0.0/8 range. If I run it locally, it is `127.0.0.1.

Below is an example of the "raw" data I am able to see that is outputted from Roadrunner when access logs are enabled:

{
  "level": "info",
  "ts": 1738235545293353049,
  "logger": "http",
  "msg": "http access log",
  "read_bytes": 0,
  "write_bytes": 123,
  "status": 200,
  "method": "POST",
  "URI": "/test",
  "URL": "/test",
  "remote_address": "10.128.9.40:49108",
  "query": "foo=bar",
  "content_len": 0,
  "host": "my.domain",
  "user_agent": "PostmanRuntime/7.43.0",
  "referer": "",
  "time_local": "30/Jan/25:11:12:25 +0000",
  "request_time": 1738235545293351269,
  "start": 1738235545079611898,
  "elapsed": 213
}

What I am looking for is just a way to get X-Forwarded-For or similar into the access logs somehow

[rustatian]

proxy_ip_parser handles XFF, X-Forwarded-For and similar headers. If not, we would need to have this header to check, at least for tests to verify where is the problem. So, to resolve this issue, header is required, unfortunately.

[carestad]

That makes sense. Those headers are not available to the access log module?

I also see now that I get this warning when starting the application though:

{
  "level": "warn",
  "ts": 1738249826088444357,
  "logger": "http",
  "msg": "requested middleware does not exist",
  "requested": "proxy_ip_parser"
}

Using the latest 2024.3.2 version of roadrunner though. Is there anything in particular I need to be aware of to be able to use it?

Currently my http.middleware looks like this: middleware: ["proxy_ip_parser", "static"]

[rustatian]

You may check them directly from the PHP, since RR passes all the headers to PHP worker.

Currently my http.middleware looks like this: middleware: ["proxy_ip_parser", "static"]

You don't have configuration for it: https://docs.roadrunner.dev/docs/http/proxy
You should add trusted_subnets configuration to the HTTP plugin.

[carestad]

Aha! I thought I tried setting those previously, but apparently not. It seems to work swell now. The actual x-forwarded-for IP is also shown and used in the access logs now 😃 Thanks!

[rustatian]

You're welcome, enjoy RR 😃

[carestad]

Hello again @rustatian! Thanks so much for the help previously.

I Just wanted to follow up on this in regards to hosting this behind a Cloudfront distribution. Cloudfront seems to set a X-Original-Forwarded-For header that contains multiple IPs, comma separated, and where the first IP in that list usually is the client IP, not the last which one might expect. And Cloudfront's X-Forwarded-For IP seems to be set to the last IP, which is usually a Cloudfront IP in one of the subnets listed in https://d7uri8nf7uskq.cloudfront.net/tools/list-cloudfront-ips

Is there a way to get Roadrunner parsing the X-Original-Forwarded-For before the proxy middleware is called somehow so I can log the actual client IPs in the access logs?

An example of the X-Original-Forwarded-For header value would be 1.2.3.4, 4.5.6.7 for instance, where 1.2.3.4 would be the client.

[rustatian]

Hey @carestad 👋🏻
Hm, since RR does not parse this header, it should be just bypassed to the PHP worker. Usually, headers are not included in the access logs since they may contain sensitive information.
You may try to log it from the PHP worker.