The “Double Charge” Nightmare

Picture this: A customer clicks “Place Order” on your e-commerce site. The network hiccups. The browser spins. Frustrated, they click again. Suddenly, you’ve charged them twice, shipped two orders, and earned yourself a chargeback dispute.

This isn’t a edge case. Mobile networks drop packets constantly. Load balancers timeout. Microservices retry failed requests. Without idempotency, your API turns user frustration into actual financial damage.

Idempotency means that making the same request multiple times produces the same result as making it once: f(x) = f(f(x)).

HTTP already handles this for most verbs: GET requests the same resource repeatedly without side effects. PUT and DELETE are designed to be repeatable. But POST, the verb we use for creating resources and processing payments, has no such guarantee. Hit it twice, get two orders.

This isn’t just about defensive programming. It’s about network reality. Retries happen whether you plan for them or not.

How Idempotency Keys Work

The solution is simple: clients send a unique identifier with each operation. The server tracks which operations it’s already processed and returns the cached result for duplicates.

The standard approach uses an Idempotency Key header (commonly Idempotency-Key or X-Request-ID). The client generates a UUID for each logical operation, not each HTTP request. If the network fails, the retry uses the same UUID.

Here’s the server-side flow:

  1. Check: Does this key exist in our state store?
  2. If found: Return the stored response immediately. Skip all business logic.
  3. If new:
    • Acquire a lock to prevent race conditions
    • Execute the business logic
    • Store the response (status code and body) with the key
    • Return the response

The key insight: you’re caching the outcome of the operation, not preventing duplicate requests from arriving. Two requests with the same key might hit your server simultaneously, but only one executes the actual work.

Architecture Decisions

Where to Store State

You need fast, distributed storage for idempotency records. Two options:

Redis (Distributed Cache)

  • Sub-millisecond lookups
  • Built-in TTL support (expire keys after 24-48 hours)
  • Handles distributed locks natively
  • Widely used for exactly this pattern

SQL Server

  • Transactional guarantees
  • Easier to correlate with business data
  • Slower than Redis
  • Requires manual cleanup of old keys

Recommendation: Use Redis with persistence enabled. Set a TTL of 24 hours for most use cases. If you’re building banking systems with strict audit requirements, consider SQL Server or a hybrid approach.

Middleware vs Action Filters

You could implement idempotency checks in ASP.NET Core middleware, but that’s the wrong abstraction. Middleware runs too early in the pipeline. It executes before:

  • Model binding
  • Validation
  • Route selection

You can’t access the actual request parameters or know which endpoint will handle it. More importantly, you don’t want every endpoint to be idempotent. GET requests don’t need it. Some POST operations are naturally idempotent.

Use an IAsyncActionFilter instead. This lets you apply idempotency selectively with an [Idempotent] attribute on specific controllers or actions. The filter runs after model binding, so you can access the full request context.

Implementation in ASP.NET Core

The Idempotent Attribute

First, create a marker attribute:

[AttributeUsage(AttributeTargets.Method | AttributeTargets.Class)]
public class IdempotentAttribute : Attribute
{
    public int CacheDurationSeconds { get; set; } = 86400; // 24 hours default
}

The Action Filter

Here’s the core implementation:

public class IdempotencyFilter : IAsyncActionFilter
{
    private readonly IDistributedCache _cache;
    private readonly ILogger<IdempotencyFilter> _logger;

    public IdempotencyFilter(IDistributedCache cache, ILogger<IdempotencyFilter> logger)
    {
        _cache = cache;
        _logger = logger;
    }

    public async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
    {
        var idempotencyKey = context.HttpContext.Request.Headers["Idempotency-Key"].FirstOrDefault();
        
        if (string.IsNullOrEmpty(idempotencyKey))
        {
            context.Result = new BadRequestObjectResult(new { error = "Idempotency-Key header is required" });
            return;
        }

        var cacheKey = $"idempotency:{idempotencyKey}";
        var requestHash = ComputeRequestHash(context.HttpContext.Request);
        var lockKey = $"{cacheKey}:lock";

        // Check if we've already processed this request
        var cachedResponse = await _cache.GetStringAsync(cacheKey);
        
        if (!string.IsNullOrEmpty(cachedResponse))
        {
            var cached = JsonSerializer.Deserialize<CachedResponse>(cachedResponse);
            
            // Validate request hasn't changed
            if (cached.RequestHash != requestHash)
            {
                context.Result = new BadRequestObjectResult(new { error = "Request body mismatch for idempotency key" });
                return;
            }

            _logger.LogInformation("Returning cached response for key {Key}", idempotencyKey);
            context.Result = new ObjectResult(cached.ResponseBody) { StatusCode = cached.StatusCode };
            return;
        }

        // Try to acquire lock
        var lockAcquired = await TryAcquireLockAsync(lockKey);
        if (!lockAcquired)
        {
            context.Result = new StatusCodeResult(409); // Conflict - processing in progress
            return;
        }

        try
        {
            // Execute the actual action
            var executedContext = await next();

            // Only cache successful responses
            if (executedContext.Result is ObjectResult objectResult && 
                objectResult.StatusCode >= 200 && objectResult.StatusCode < 300)
            {
                var cacheEntry = new CachedResponse
                {
                    StatusCode = objectResult.StatusCode ?? 200,
                    ResponseBody = objectResult.Value,
                    RequestHash = requestHash
                };

                var attribute = context.ActionDescriptor.EndpointMetadata
                    .OfType<IdempotentAttribute>()
                    .FirstOrDefault();

                var options = new DistributedCacheEntryOptions
                {
                    AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(
                        attribute?.CacheDurationSeconds ?? 86400)
                };

                await _cache.SetStringAsync(
                    cacheKey, 
                    JsonSerializer.Serialize(cacheEntry), 
                    options);
            }
        }
        finally
        {
            await ReleaseLockAsync(lockKey);
        }
    }

    private string ComputeRequestHash(HttpRequest request)
    {
        request.Body.Position = 0;
        using var reader = new StreamReader(request.Body, leaveOpen: true);
        var body = reader.ReadToEnd();
        request.Body.Position = 0;
        
        using var sha256 = SHA256.Create();
        var hash = sha256.ComputeHash(Encoding.UTF8.GetBytes(body));
        return Convert.ToBase64String(hash);
    }

    private async Task<bool> TryAcquireLockAsync(string lockKey)
    {
        var options = new DistributedCacheEntryOptions
        {
            AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(30)
        };

        try
        {
            await _cache.SetStringAsync(lockKey, "locked", options);
            return true;
        }
        catch
        {
            return false;
        }
    }

    private async Task ReleaseLockAsync(string lockKey)
    {
        await _cache.RemoveAsync(lockKey);
    }
}

public class CachedResponse
{
    public int StatusCode { get; set; }
    public object ResponseBody { get; set; }
    public string RequestHash { get; set; }
}

Registration

Wire it up in Program.cs:

builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = builder.Configuration.GetConnectionString("Redis");
});

builder.Services.AddScoped<IdempotencyFilter>();

Usage

Apply the attribute to your endpoints:

[ApiController]
[Route("api/[controller]")]
public class OrdersController : ControllerBase
{
    [HttpPost]
    [Idempotent(CacheDurationSeconds = 172800)] // 48 hours
    public async Task<IActionResult> CreateOrder([FromBody] CreateOrderRequest request)
    {
        var order = await _orderService.ProcessOrderAsync(request);
        return Ok(new { orderId = order.Id, total = order.Total });
    }
}

Handling Edge Cases

Key Collision and Request Validation

What if a client reuses an idempotency key with different request data? Maybe they’re malicious, or maybe they generated the UUID client-side and had a bug.

The solution: hash the request body and store it alongside the cached response. On subsequent requests, compare the incoming hash to the stored one. If they don’t match, return 400 Bad Request.

This is why the filter computes requestHash and validates it before returning cached responses.

Should You Cache Failures?

Here’s the judgment call: do you cache 500 Internal Server Error responses?

No. If your server failed due to a transient issue (database timeout, downstream service unavailable), the client should retry. Caching the failure prevents recovery.

Only cache:

  • Success responses (2xx) - the operation completed
  • Client errors (4xx) - the request itself was invalid and won’t succeed on retry

Don’t cache server errors (5xx). The filter above only caches 2xx status codes.

The Dual Write Problem

There’s a subtle race condition here: what if you successfully save the order to SQL Server but fail to write to Redis? The next retry will execute the business logic again, creating a duplicate order.

This is the classic dual write problem. You’re writing to two separate stores (SQL + Redis) without a transaction.

Pragmatic solution: For most applications, Redis is reliable enough that this risk is acceptable. If it crashes, you lose idempotency protection temporarily, which is annoying but not catastrophic.

Banking-grade solution: Use the transactional outbox pattern. Write both the business data and the idempotency record to SQL Server in the same transaction. Use a background job to sync successful operations to Redis for fast lookups.

Lock Expiration

The lock in TryAcquireLockAsync expires after 30 seconds. What if your business logic takes longer?

Options:

  1. Extend the lock: Use a background timer to refresh the lock while processing
  2. Fail fast: If your operations take that long, you have bigger problems. Optimize or use async patterns.
  3. Return 409: Let the client know processing is ongoing

For most APIs, operations should complete in seconds, not minutes. The 30-second timeout catches hung requests.

Testing Idempotency

Unit tests won’t catch race conditions. You need integration tests that simulate concurrent requests.

Use WebApplicationFactory to spin up your API and Task.WhenAll to fire simultaneous requests:

[Fact]
public async Task CreateOrder_WithSameIdempotencyKey_ExecutesOnlyOnce()
{
    var factory = new WebApplicationFactory<Program>();
    var client = factory.CreateClient();
    var idempotencyKey = Guid.NewGuid().ToString();

    var tasks = Enumerable.Range(0, 10).Select(async i =>
    {
        var request = new HttpRequestMessage(HttpMethod.Post, "/api/orders")
        {
            Content = JsonContent.Create(new { productId = 123, quantity = 1 }),
            Headers = { { "Idempotency-Key", idempotencyKey } }
        };
        return await client.SendAsync(request);
    });

    var responses = await Task.WhenAll(tasks);

    // All responses should be 200 OK
    Assert.All(responses, r => Assert.Equal(HttpStatusCode.OK, r.StatusCode));

    // Verify the order service was called exactly once
    var orderCount = await GetOrderCountFromDatabase();
    Assert.Equal(1, orderCount);
}

This test proves that even when 10 requests arrive simultaneously, only one executes the business logic.

Summary

Idempotent APIs aren’t optional in distributed systems. Network failures and retry logic are facts of life. Without idempotency, you create duplicate orders, double charges, and customer support nightmares.

The pattern is straightforward:

  • Clients send a unique Idempotency-Key header
  • Servers cache successful responses in Redis
  • Duplicate requests return the cached result

Implementation requires careful attention to:

  • Request validation - hash the body to detect changed parameters
  • Concurrency control - use locks to prevent parallel execution
  • Error handling - only cache successes and definitive client errors
  • Testing - simulate concurrent requests in integration tests

Use action filters, not middleware. Use Redis, not SQL Server (unless you need strict audit trails). Cache for 24-48 hours. Test with real concurrency.

Your users will never know you implemented this. That’s the point. Resilience should be invisible.

References

  1. IETF Draft: The Idempotency-Key HTTP Header Field - The emerging standard for idempotency keys
  2. Stripe API: Idempotent Requests - Real-world implementation from a payment processor
  3. Microsoft Docs: Distributed Caching in ASP.NET Core - Official guidance on IDistributedCache
  4. AWS Architecture Blog: Idempotency Patterns - Patterns for building resilient systems
  5. Martin Fowler: Transactional Outbox - Solution for the dual write problem

About the Author

Abhinaw Kumar is a software engineer who builds real-world systems: from resilient ASP.NET Core backends to clean, maintainable Angular frontends. With over 12+ years in production development, he shares what actually works when you're shipping software that has to last.

Read more on the About page or connect on LinkedIn.

Related Posts