HTTP handler error log improvement

[evenyag]

What type of enhancement is this?

User experience

What does the enhancement do?

Sometimes we may want to find out the reason for a non-200 HTTP response and we may not be able to get the error log from client side.

To collect the error logs in the HTTP handler, we added a log in our IntoResponse implementation for the Error.

greptimedb/src/servers/src/error.rs

Lines 804 to 816 in 69f0249

	impl IntoResponse for Error {
	fn into_response(self) -> Response {
	let error_msg = self.output_msg();
	let status = status_code_to_http_status(&self.status_code());
	log_error_if_necessary(&self);
	let body = Json(json!({
	"error": error_msg,
	}));
	(status, body).into_response()
	}
	}

greptimedb/src/servers/src/http/result/error_result.rs

Lines 71 to 90 in 69f0249

	impl IntoResponse for ErrorResponse {
	fn into_response(self) -> Response {
	let code = self.code;
	let execution_time = self.execution_time_ms;
	let new_header = from_err_code_msg_to_header(
	code,
	&format!(
	"error: {}, execution_time_ms: {}",
	self.error, execution_time
	),
	);
	let mut resp = Json(self).into_response();
	resp.headers_mut().extend(new_header);
	let status = StatusCode::from_u32(code).unwrap_or(StatusCode::Unknown);
	let status_code = status_code_to_http_status(&status);
	(status_code, resp).into_response()
	}
	}

It'd be better to have some logs for HTTP errors so we can debug later. We may need to check all IntoResponse implementations.

Implementation challenges

Not all handler implementations convert errors into an Error. Then we won't be able to get the error message.

Case 1, the handler may returns an HTTPResponse directly:

greptimedb/src/servers/src/http/event.rs

Lines 700 to 708 in 69f0249

	#[axum_macros::debug_handler]
	pub async fn log_ingester(
	State(log_state): State<LogState>,
	Query(query_params): Query<LogIngesterQueryParams>,
	Extension(mut query_ctx): Extension<QueryContext>,
	TypedHeader(content_type): TypedHeader<ContentType>,
	headers: HeaderMap,
	payload: Bytes,
	) -> Result<HttpResponse> {

Case 2, we can't catch the error raised by the axum itself. Unfortunately, the trace layer also can't get the error message and 400 is not considered a failure.
https://docs.rs/tower-http/latest/tower_http/trace/index.html#on_response
https://github.com/tokio-rs/axum/blob/main/examples/error-handling/src/main.rs

We can get the remote address, which may be useful in logging.
tokio-rs/axum#43
https://docs.rs/axum/latest/axum/struct.Router.html#method.into_make_service_with_connect_info