Table of Contents
TL;DR - Practical Tips for ASP.NET Core HTTP Logging Middleware
- Use
EnableBuffering()
to safely log request bodies and reset stream position after reading. - Capture response bodies by swapping
HttpResponse.Body
with aMemoryStream
and restoring after logging. - Always filter or redact sensitive data before logging HTTP bodies.
- Set size limits and skip large or binary payloads to avoid performance issues.
- Filter logs by endpoint, HTTP method, or custom attributes for clarity.
- Built-in
HttpLogging
is simple; custom middleware offers full control and advanced filtering. - Use structured logging and external services for production monitoring.
- Wrap middleware in extension methods for clean, configurable registration.
- Leverage
PipeReader
andSystem.IO.Pipelines
for more efficient memory usage. - Use
Span<T>
andArrayPool<T>
for zero-allocation processing of large payloads. - Consider implementing request sampling in production to reduce log volume.
- Use a complete middleware implementation that combines all best practices.
1. How does EnableBuffering
help you log request bodies in ASP.NET Core?
EnableBuffering allows you to read request bodies multiple times without breaking the pipeline.
By default, HttpRequest.Body
is a forward-only stream. Once you read it, subsequent middleware gets an empty stream. This breaks model binding and authentication.
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
// Enable buffering so we can read multiple times
context.Request.EnableBuffering();
// Read the body
using var reader = new StreamReader(context.Request.Body, leaveOpen: true);
var body = await reader.ReadToEndAsync();
// Reset position for next middleware
context.Request.Body.Position = 0;
_logger.LogInformation("Request Body: {Body}", body);
await next(context);
}
EnableBuffering internally replaces the request stream with a FileBufferingReadStream
that can seek. For small bodies, it uses memory. For large ones, it spills to disk.
Without this call, you’d get exceptions or empty request bodies downstream. Essential for any middleware that needs to inspect request content while preserving normal request processing.
2. What’s the trick to capturing response bodies using MemoryStream in middleware?
Replace HttpResponse.Body with a MemoryStream, then copy the content back after processing.
Response bodies are write-only streams that go directly to the client. To capture them, you need to intercept the stream temporarily.
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
var originalBodyStream = context.Response.Body;
using var responseBody = new MemoryStream();
context.Response.Body = responseBody;
await next(context);
// Capture the response
responseBody.Seek(0, SeekOrigin.Begin);
var responseText = await new StreamReader(responseBody).ReadToEndAsync();
_logger.LogInformation("Response: {Response}", responseText);
// Copy back to original stream
responseBody.Seek(0, SeekOrigin.Begin);
await responseBody.CopyToAsync(originalBodyStream);
context.Response.Body = originalBodyStream;
}
Critical: Always restore the original stream and copy your captured content back. Forgetting this step results in empty responses to clients.
3. Why should you reset the request stream position before passing to the next middleware?
Streams maintain position after reading, failing to reset breaks model binding and authentication.
When you read from HttpRequest.Body
, the stream position advances to the end. The next middleware expects to read from the beginning.
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
context.Request.EnableBuffering();
// Read request body
var body = await context.Request.ReadFromJsonAsync<MyModel>();
// Stream position is now at EOF
// Next middleware will read empty content!
// Reset position
context.Request.Body.Position = 0;
await next(context);
}
Common failure scenarios:
- JWT middleware can’t read bearer tokens from request body
- Model binding returns null objects
- Authentication middleware fails silently
Best practice: Always reset to position 0 after reading, even if you think no downstream middleware needs the body. Future code changes might break unexpectedly without this safety net.
Reset immediately after reading, before calling next(context)
.
4. How do you avoid logging sensitive data when capturing HTTP bodies?
Use conditional redaction based on content type, headers, or request paths to filter sensitive information.
Never log passwords, tokens, or PII. Implement filtering before writing to logs.
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
var path = context.Request.Path.Value;
var contentType = context.Request.ContentType;
// Skip sensitive endpoints
if (path?.Contains("/auth/login") == true ||
path?.Contains("/payment") == true)
{
_logger.LogInformation("Skipped logging for sensitive endpoint: {Path}", path);
await next(context);
return;
}
context.Request.EnableBuffering();
var body = await new StreamReader(context.Request.Body).ReadToEndAsync();
context.Request.Body.Position = 0;
// Redact JSON properties
var sanitized = RedactSensitiveFields(body);
_logger.LogInformation("Request: {Body}", sanitized);
await next(context);
}
private string RedactSensitiveFields(string json)
{
return Regex.Replace(json, @"""(password|token|ssn|creditCard)""\s*:\s*""[^""]*""",
@"""$1"":""***""", RegexOptions.IgnoreCase);
}
Production tip: Use allow-lists instead of deny-lists for maximum security.
5. When should you truncate logs or skip large request/response bodies?
Set size limits to prevent memory issues and log storage bloat, typical threshold is 1KB for debugging, 10KB maximum.
Large payloads can crash applications or fill disk space. Implement size checking before reading streams.
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
const int maxLogSize = 1024; // 1KB
var contentLength = context.Request.ContentLength;
if (contentLength > maxLogSize)
{
_logger.LogInformation("Request body too large ({Size} bytes), skipping log", contentLength);
await next(context);
return;
}
context.Request.EnableBuffering();
var body = await new StreamReader(context.Request.Body).ReadToEndAsync();
if (body.Length > maxLogSize)
{
var truncated = body.Substring(0, maxLogSize);
_logger.LogInformation("Request body (truncated): {Body}...", truncated);
}
else
{
_logger.LogInformation("Request body: {Body}", body);
}
context.Request.Body.Position = 0;
await next(context);
}
File uploads, images, and PDFs should always be skipped. Check Content-Type
headers like multipart/form-data
or application/octet-stream
.
6. How can you selectively log only specific API endpoints?
Filter by request path, HTTP method, or custom attributes to avoid noise from health checks and static files.
Logging every request creates noise. Target specific endpoints that matter for debugging.
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
// Skip common noise
var path = context.Request.Path.Value?.ToLower();
if (path?.StartsWith("/health") == true ||
path?.StartsWith("/metrics") == true ||
path?.Contains("/swagger") == true)
{
await next(context);
return;
}
// Only log API endpoints
if (!path?.StartsWith("/api/") == true)
{
await next(context);
return;
}
// Log specific HTTP methods
if (context.Request.Method is "POST" or "PUT" or "DELETE")
{
await LogRequestBody(context);
}
await next(context);
}
Advanced filtering using endpoint metadata:
var endpoint = context.GetEndpoint();
var shouldLog = endpoint?.Metadata.GetMetadata<LogRequestAttribute>() != null;
if (shouldLog)
{
await LogRequestBody(context);
}
Create a [LogRequest]
attribute for fine-grained control over which controllers or actions get logged.
7. What are the performance costs of logging HTTP bodies, and how to mitigate them?
HTTP body logging adds 15-50ms latency and significant memory pressure. Use async operations and size limits.
Performance impacts:
- Memory: Buffering doubles memory usage per request
- I/O: Additional file/stream operations slow response times
- CPU: String manipulation and JSON parsing overhead
Mitigation Strategy | Impact | Implementation |
---|---|---|
Size limits | High | Skip bodies > 1KB |
Async operations | Medium | Use ReadToEndAsync() |
Conditional logging | High | Filter by endpoint |
Background logging | Medium | Queue logs, process later |
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
// Fast path for large requests
if (context.Request.ContentLength > 1024)
{
await next(context);
return;
}
var stopwatch = Stopwatch.StartNew();
// Async operations prevent blocking
context.Request.EnableBuffering();
var body = await context.Request.Body.ReadToEndAsync();
context.Request.Body.Position = 0;
// Background logging
_ = Task.Run(() => _logger.LogInformation("Request: {Body}", body));
await next(context);
_logger.LogDebug("Logging overhead: {Duration}ms", stopwatch.ElapsedMilliseconds);
}
Production recommendation: Use sampling, log 1% of requests rather than all requests.
8. How to implement custom middleware vs built-in HttpLogging in ASP.NET Core?
Built-in HttpLogging is simpler but less flexible. Custom middleware gives complete control over filtering and formatting.
Feature | Built-in HttpLogging | Custom Middleware |
---|---|---|
Setup complexity | Low | Medium |
Request body logging | Yes | Yes |
Response body logging | Yes | Yes |
Custom filtering | Limited | Full control |
Performance tuning | Basic | Advanced |
Sensitive data handling | Basic redaction | Custom logic |
Built-in HttpLogging:
builder.Services.AddHttpLogging(options =>
{
options.LoggingFields = HttpLoggingFields.RequestBody | HttpLoggingFields.ResponseBody;
options.RequestBodyLogLimit = 1024;
});
app.UseHttpLogging();
Custom middleware:
public class CustomHttpLoggingMiddleware
{
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
// Your custom logic here
await LogRequest(context);
await next(context);
await LogResponse(context);
}
}
app.UseMiddleware<CustomHttpLoggingMiddleware>();
Choose custom middleware when you need endpoint-specific filtering, custom redaction, or performance optimizations.
9. Where should you send your logs: console, file, structured, or external service?
Development uses console, staging uses files, production uses structured logging to external services like Seq or ELK.
Logging destinations by environment:
Environment | Primary | Secondary | Reasoning |
---|---|---|---|
Development | Console | File | Immediate feedback |
Staging | File | Structured | Debugging integration issues |
Production | External service | File backup | Centralized monitoring |
builder.Logging.ClearProviders();
if (builder.Environment.IsDevelopment())
{
builder.Logging.AddConsole();
}
else if (builder.Environment.IsStaging())
{
builder.Logging.AddFile("logs/staging-{Date}.txt");
}
else
{
builder.Logging.AddSeq("https://seq.company.com");
builder.Logging.AddFile("logs/production-{Date}.txt"); // Backup
}
Structured logging for production:
_logger.LogInformation("HTTP Request processed {@RequestData}", new {
Method = context.Request.Method,
Path = context.Request.Path,
StatusCode = context.Response.StatusCode,
Duration = stopwatch.ElapsedMilliseconds
});
External services enable dashboards, alerts, and correlation across multiple application instances.
10. How to wrap this middleware implementation behind a clean extension method?
Create an extension method on IApplicationBuilder for clean registration and configuration options.
public static class HttpLoggingMiddlewareExtensions
{
public static IApplicationBuilder UseCustomHttpLogging(
this IApplicationBuilder app,
Action<HttpLoggingOptions>? configure = null)
{
var options = new HttpLoggingOptions();
configure?.Invoke(options);
return app.UseMiddleware<CustomHttpLoggingMiddleware>(options);
}
}
public class HttpLoggingOptions
{
public int MaxBodySize { get; set; } = 1024;
public string[] SkipPaths { get; set; } = { "/health", "/metrics" };
public bool LogResponseBodies { get; set; } = true;
public string[] SensitiveFields { get; set; } = { "password", "token" };
}
Clean usage in Program.cs:
app.UseCustomHttpLogging(options =>
{
options.MaxBodySize = 2048;
options.SkipPaths = new[] { "/health", "/swagger" };
options.LogResponseBodies = false;
});
Middleware constructor:
public CustomHttpLoggingMiddleware(RequestDelegate next, HttpLoggingOptions options)
{
_next = next;
_options = options;
}
This pattern provides a clean API while keeping configuration flexible and testable.
11. How to use PipeReader for more memory-efficient request body reading?
Use System.IO.Pipelines to read request bodies with minimal allocations and improved performance.
The PipeReader
API from System.IO.Pipelines
provides a more efficient way to read streams than traditional methods, reducing allocations and improving throughput.
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
context.Request.EnableBuffering();
// Get the PipeReader from the request body
var reader = PipeReader.Create(context.Request.Body);
StringBuilder requestBody = new StringBuilder();
while (true)
{
ReadResult result = await reader.ReadAsync();
ReadOnlySequence<byte> buffer = result.Buffer;
// Process the buffer to extract the request body
ProcessBuffer(buffer, requestBody);
// Mark how much we consumed
reader.AdvanceTo(buffer.Start, buffer.End);
if (result.IsCompleted)
break;
}
_logger.LogInformation("Request body: {Body}", requestBody.ToString());
// Reset position for next middleware
context.Request.Body.Position = 0;
await next(context);
}
private void ProcessBuffer(ReadOnlySequence<byte> buffer, StringBuilder stringBuilder)
{
foreach (ReadOnlyMemory<byte> memory in buffer)
{
// Convert bytes to string and append to the string builder
stringBuilder.Append(Encoding.UTF8.GetString(memory.Span));
}
}
Advantages of using PipeReader:
- Reduced allocations:
PipeReader
andPipeWriter
minimize memory allocations by using pooled buffers and avoiding intermediate arrays. - Backpressure support: Works seamlessly with async streams and supports backpressure, preventing memory overuse.
- Efficient reading: Reads data in chunks and allows processing as data arrives, improving responsiveness.
- Integration with middleware: Easily integrates with ASP.NET Core middleware for request/response processing.
12. How to use Span for zero-allocation HTTP body processing?
Use Span
Span
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
context.Request.EnableBuffering();
// Get the content length or use a default size
var contentLength = context.Request.ContentLength ?? 4096;
// Only process if under our limit
if (contentLength <= MaxBodySize)
{
// Rent a buffer from the array pool
byte[] rentedBuffer = ArrayPool<byte>.Shared.Rent((int)contentLength);
try
{
int bytesRead = await context.Request.Body.ReadAsync(rentedBuffer, 0, (int)contentLength);
// Create a span over the actual data
ReadOnlySpan<byte> bodySpan = new ReadOnlySpan<byte>(rentedBuffer, 0, bytesRead);
// Process the span (without allocating any additional memory)
if (IsJsonContent(context.Request.ContentType))
{
var sanitized = RedactSensitiveFieldsSpanBased(bodySpan);
_logger.LogInformation("Request body: {Body}", sanitized);
}
else
{
_logger.LogInformation("Request body: {Body}", Encoding.UTF8.GetString(bodySpan));
}
}
finally
{
// Return the buffer to the pool
ArrayPool<byte>.Shared.Return(rentedBuffer);
// Reset position for next middleware
context.Request.Body.Position = 0;
}
}
await next(context);
}
private static bool IsJsonContent(string? contentType) =>
contentType?.Contains("application/json") == true;
private string RedactSensitiveFieldsSpanBased(ReadOnlySpan<byte> bodySpan)
{
// Convert to string for processing (in production, you'd use a Span-aware JSON parser)
string json = Encoding.UTF8.GetString(bodySpan);
// Redact sensitive fields
return Regex.Replace(json, @"""(password|token|ssn|creditCard)""\s*:\s*""[^""]*""",
@"""$1"":""***""", RegexOptions.IgnoreCase);
}
Advantages of Span
- Zero heap allocations: Works directly with memory without creating intermediate strings or arrays
- Reduced GC pressure: Less garbage collection means better throughput
- Improved cache locality: Contiguous memory access patterns optimize CPU cache usage
- Rental pools: Using
ArrayPool<T>
further reduces allocations for temporary buffers
This approach works best in .NET Core 3.1+ and .NET 5+ applications where Span
13. Complete HTTP Logging Middleware Implementation
Here’s a production-ready HTTP logging middleware that combines all the best practices.
This example brings together the key concepts from this article into a complete, configurable middleware implementation.
using System;
using System.Buffers;
using System.Collections.Generic;
using System.IO;
using System.IO.Pipelines;
using System.Text;
using System.Text.Json;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace YourNamespace.Middleware
{
public class HttpLoggingOptions
{
public int MaxBodySize { get; set; } = 1024;
public string[] SkipPaths { get; set; } = { "/health", "/metrics" };
public bool LogResponseBodies { get; set; } = true;
public string[] SensitiveFields { get; set; } = { "password", "token", "ssn" };
public double SamplingRate { get; set; } = 1.0; // 1.0 = log 100%, 0.1 = log 10%
}
public class HttpLoggingMiddleware
{
private readonly RequestDelegate _next;
private readonly ILogger<HttpLoggingMiddleware> _logger;
private readonly HttpLoggingOptions _options;
private readonly Random _random = new Random();
public HttpLoggingMiddleware(
RequestDelegate next,
ILogger<HttpLoggingMiddleware> logger,
IOptions<HttpLoggingOptions> options)
{
_next = next;
_logger = logger;
_options = options.Value;
}
public async Task InvokeAsync(HttpContext context)
{
// Apply sampling
if (_options.SamplingRate < 1.0 && _random.NextDouble() > _options.SamplingRate)
{
await _next(context);
return;
}
// Skip excluded paths
var path = context.Request.Path.Value?.ToLowerInvariant();
foreach (var skipPath in _options.SkipPaths)
{
if (path?.StartsWith(skipPath.ToLowerInvariant()) == true)
{
await _next(context);
return;
}
}
// Skip binary content
var contentType = context.Request.ContentType?.ToLowerInvariant();
if (contentType?.Contains("multipart/form-data") == true ||
contentType?.Contains("application/octet-stream") == true ||
contentType?.Contains("image/") == true)
{
_logger.LogInformation("Skipping binary content: {ContentType}", contentType);
await _next(context);
return;
}
// Skip large content
if (context.Request.ContentLength > _options.MaxBodySize)
{
_logger.LogInformation("Content too large ({Size} bytes), skipping body logging",
context.Request.ContentLength);
await _next(context);
return;
}
// Prepare for logging
var stopwatch = System.Diagnostics.Stopwatch.StartNew();
context.Request.EnableBuffering();
// Log request body
string? requestBody = null;
if (context.Request.ContentLength > 0)
{
requestBody = await ReadRequestBodyAsync(context.Request);
context.Request.Body.Position = 0; // Reset for next middleware
}
// Process response
var originalResponseBody = context.Response.Body;
try
{
using var responseMemoryStream = new MemoryStream();
context.Response.Body = responseMemoryStream;
// Continue the pipeline
await _next(context);
// Log response if needed
string? responseBody = null;
if (_options.LogResponseBodies)
{
responseBody = await ReadResponseBodyAsync(responseMemoryStream, originalResponseBody);
}
else
{
// Just copy the response without reading for logging
responseMemoryStream.Seek(0, SeekOrigin.Begin);
await responseMemoryStream.CopyToAsync(originalResponseBody);
}
// Log complete info
LogRequestResponse(
context,
stopwatch.ElapsedMilliseconds,
RedactSensitiveData(requestBody),
RedactSensitiveData(responseBody));
}
finally
{
// Ensure the response body is restored
context.Response.Body = originalResponseBody;
}
}
private async Task<string?> ReadRequestBodyAsync(HttpRequest request)
{
using var reader = new StreamReader(
request.Body,
encoding: Encoding.UTF8,
detectEncodingFromByteOrderMarks: false,
leaveOpen: true);
var body = await reader.ReadToEndAsync();
return body;
}
private async Task<string?> ReadResponseBodyAsync(
MemoryStream responseMemoryStream,
Stream originalResponseBody)
{
responseMemoryStream.Seek(0, SeekOrigin.Begin);
var responseBody = await new StreamReader(
responseMemoryStream,
Encoding.UTF8).ReadToEndAsync();
responseMemoryStream.Seek(0, SeekOrigin.Begin);
await responseMemoryStream.CopyToAsync(originalResponseBody);
return responseBody;
}
private string? RedactSensitiveData(string? data)
{
if (string.IsNullOrEmpty(data))
return data;
// Simple redaction for sensitive fields
var pattern = string.Join("|", _options.SensitiveFields);
return Regex.Replace(
data,
$@"""({pattern})""\s*:\s*""[^""]*""",
@"""$1"":""***""",
RegexOptions.IgnoreCase);
}
private void LogRequestResponse(
HttpContext context,
long elapsedMs,
string? requestBody,
string? responseBody)
{
// Structured logging
_logger.LogInformation("HTTP {Method} {Path} completed in {Duration}ms with status {StatusCode} {@RequestResponse}",
context.Request.Method,
context.Request.Path,
elapsedMs,
context.Response.StatusCode,
new
{
RequestHeaders = GetHeadersDictionary(context.Request.Headers),
RequestBody = requestBody ?? "(empty)",
ResponseHeaders = GetHeadersDictionary(context.Response.Headers),
ResponseBody = responseBody ?? "(empty)"
});
}
private Dictionary<string, string?> GetHeadersDictionary(IHeaderDictionary headers)
{
var result = new Dictionary<string, string?>();
foreach (var header in headers)
{
// Skip headers with sensitive data like authentication
if (!header.Key.Equals("Authorization", StringComparison.OrdinalIgnoreCase) &&
!header.Key.Equals("Cookie", StringComparison.OrdinalIgnoreCase))
{
result[header.Key] = header.Value.ToString();
}
else
{
result[header.Key] = "***";
}
}
return result;
}
}
// Extension methods for clean registration
public static class HttpLoggingMiddlewareExtensions
{
public static IServiceCollection AddHttpLogging(
this IServiceCollection services,
Action<HttpLoggingOptions>? configure = null)
{
if (configure != null)
{
services.Configure(configure);
}
else
{
services.Configure<HttpLoggingOptions>(_ => { });
}
return services;
}
public static IApplicationBuilder UseHttpLogging(this IApplicationBuilder app)
{
return app.UseMiddleware<HttpLoggingMiddleware>();
}
}
}
Registration in Program.cs:
// Register and configure the HTTP logging
builder.Services.AddHttpLogging(options =>
{
options.MaxBodySize = 4096; // 4KB
options.LogResponseBodies = true;
options.SamplingRate = builder.Environment.IsProduction() ? 0.1 : 1.0;
options.SensitiveFields = new[] { "password", "token", "apiKey", "ssn" };
options.SkipPaths = new[] { "/health", "/metrics", "/swagger" };
});
// ... other service registrations
var app = builder.Build();
// Register early in the pipeline, but after exception handling
app.UseExceptionHandler("/error");
app.UseHttpLogging();
// ... other middleware registrations
This implementation combines all the best practices covered in this article:
- Request and response body logging with proper stream handling
- Memory-efficient processing
- Content type and size filtering
- Path-based exclusions
- Sensitive data redaction
- Structured logging output
- Performance optimizations like sampling
- Clean configuration API
By using this pattern, you get powerful HTTP logging capabilities without compromising performance or security.
To summarize, these tips help you implement robust HTTP logging middleware in ASP.NET Core, ensuring you can capture and analyze request and response bodies effectively while maintaining performance and security. By following these practices, you can create a logging solution that is both powerful and efficient, tailored to your application’s needs.