Last year (2024), I had the chance to work on a payment system. While working on it, I noticed that synchronous approaches are impractical for long-running processes—common in production-scale applications.

The Problem with Synchronous Processing

What's a Synchronous Approach?

A synchronous process executes from start to finish in a single blocking call. The client waits and receives either a success or failure response immediately.

func CreateTransfer(ctx context.Context, request CreateTransferRequest) (CreateTransferResponse, error) {
  // 1. Validate user permissions
  // 2. Verify destination account validity
  // 3. Check sender's balance
  // 4. Authorize (hold) funds
  // 5. Call payment gateway (slow!)
  // 6. Update balances based on gateway response
  // 7. Record transaction in DB
  // 8. Return final result
}

Why Synchronous Approach Fail?

• Resource Bottlenecks: each call holds a connection thread (means more costly)
• Timeout Risks: network instability may interrupt long-running calls
• Poor Scalability: Handling 100 RPS for a long-running process can be very expensive

The Asynchronous Solution

instead of processing it in a single-go, lets split the process...

Step 1: Immediate Acceptance

func CreateTransfer(ctx context.Context, request CreateTransferRequest) (CreateTransferResponse, error) {
  // 1. Persist request to DB (status=PENDING)
  // 2. Publish to message queue (e.g., Kafka)
  // 3. Return 202 Accepted with status=PROCESSING
  return CreateTransferResponse{
    Status:    "PROCESSING",
    Message:   "Transaction queued",
    RequestID: requestID,
  }, nil
}

Step 2: Background Processing

func ProcessTransferWorker(ctx context.Context) {
  for {
    msg := queue.Consume()
    
    // 1. Validate all business rules
    // 2. Execute payment gateway call
    // 3. Update DB status (SUCCESS/FAILED)
    
    if err := process(msg); err != nil {
      metrics.RecordFailure()
      continue
    }
  }
}

ok, thats simple right, but there is a question how the client know when the transaction is either completed or failed ?

Step 3: Retrieve transaction status

// GET /transactions/{id}
func GetStatus(ctx context.Context, id string) (StatusResponse, error) {
  tx, err := db.GetTransaction(id)
  if err != nil {
    return nil, ErrNotFound
  }
  
  return StatusResponse{
    ID:      tx.ID,
    Status:  tx.Status,
    Updated: tx.UpdatedAt,
  }, nil
}

Sounds simple enough, right? But there's another approach we can take—without creating an extra endpoint (and you know, more endpoints mean more things to build, handle, and maintain 😅). Let’s keep that for another post.