0

I'm trying to setup a self-managed docker appsearch instance, together with kibana and elasticsearch, queried by a uvicorn python app, proxied by a nginx webserver

My current issue is that the appsearch logs show the python default user-agent and IP in the appsearch logs (i.e. python-requests/2.22.0 and a LAN IP).

I would like to forward the nginx custom headers that contain the correct IP and user agent of the remote client, to the appsearch logs that are so nicely queryable in kibana.

I've noticed that there is the output.elasticsearch.headers that can be set to custom headers in environment or filebeat.yml.

Do you guys have any ideas about this ?

Thank you.

Niloct
  • 101
  • 4

1 Answers1

0

Well, I've finally did it.

It wasn't necessary neither sufficient to use yml or environment variables since the real client IP and user agent weren't static.

I've changed first the nginx.conf proxy to this:

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-User-Agent $http_user_agent;

Then in each uvicorn FastAPI method I added a first request parameter:

from fastapi import FastAPI
app = FastAPI()
# ...
@app.get("/search")
async def search_endpoint(request: Request):
# ... method implementation

The search_endpoint calls my search class which in turn uses appsearch's python client:

import elastic_app_search
# ...
client = elastic_app_search.Client(api_key = XXX, base_endpoint = YYY, use_https=False)

And then in client I update the headers:

x_headers = {
    'Connection': 'close',
    'X-Forwarded-For': request.headers['X-Forwarded-For'],
    'X-User-Agent': request.headers['X-User-Agent']
}
client.session.session.headers.update(x_headers)
# For current appsearch python client the repeated name was necessary

Then the application logs started logging the custom X-headers =)

Niloct
  • 101
  • 4