跳到主要内容

2 篇博文 含有标签「transaction」

查看所有标签

Approaches to Rolling Back or Canceling Transactions in Microservice

· 阅读需 7 分钟

Handling rollback or cancellation of transactions in a Microservice Architecture when multiple writes are involved is a complex process because each microservice typically has its own database, and there is no central point of coordination like in monolithic applications. Traditional database transactions (ACID) are difficult to apply in microservices, especially when services span across multiple databases, message brokers, and external APIs. Therefore, rollback in microservices needs a more sophisticated approach.

Key Approaches to Rolling Back or Canceling Transactions in Microservice Architecture​

There are several strategies to handle rollback or cancellation in microservices when multiple services and writes are involved. These include:


1. Saga Pattern​

The Saga Pattern is the most popular strategy for handling distributed transactions in microservices. It breaks a large transaction into a sequence of smaller, independent transactions, each handled by a microservice. If any part of the sequence fails, compensating actions (i.e., rollbacks) are triggered to undo the work done by previous steps.

There are two common approaches to implementing the Saga Pattern:

  • Choreography-based Saga
  • Orchestration-based Saga

a. Choreography-based Saga​

In a Choreography-based Saga, each service involved in the transaction emits an event upon completion of its task. Other services listen to these events and take action accordingly. If an error occurs, the service responsible for the failed transaction emits a failure event, and other services listening for that failure event perform compensating actions.

Example: Imagine an e-commerce system where you place an order that spans multiple microservices:

  1. Order Service: Creates the order.
  2. Inventory Service: Reserves the product.
  3. Payment Service: Charges the customer.
  4. Shipping Service: Ships the product.

If the Payment Service fails after reserving the inventory, it can emit a failure event, and the Inventory Service would listen to this and release the reserved product (compensating action).

Advantages:

  • Fully decentralized and scalable.
  • Easy to add new services without affecting others.

Challenges:

  • Harder to track and manage because there is no central coordinator.
  • More complex error handling logic in each service.

b. Orchestration-based Saga​

In an Orchestration-based Saga, a central coordinator (Orchestrator) is responsible for managing the workflow of the transaction. The Orchestrator sends commands to each service to execute a step in the transaction and handles failures by issuing compensating commands if any service fails.

Example:

  • The Orchestrator sends a command to the Order Service to create an order.
  • After receiving success, the Orchestrator asks the Inventory Service to reserve the product.
  • If the Payment Service fails, the Orchestrator issues a compensating command to the Inventory Service to release the product.

Advantages:

  • Easier to implement because the Orchestrator handles the entire workflow.
  • Centralized error handling and compensations.

Challenges:

  • The Orchestrator can become a single point of failure or bottleneck.
  • Needs to manage state and retries, making it potentially complex.

2. Compensation Mechanism​

The Compensation Mechanism is the core of both the Saga Pattern and other eventual consistency models. A compensation transaction is essentially the "undo" operation that reverses the effects of a previously completed step in case of failure.

How It Works: Each microservice is responsible for defining and implementing compensating actions for its operations. For example:

  • If the Inventory Service reserves stock but the Payment Service fails, the Inventory Service must implement logic to release the stock.
  • If the Payment Service successfully charges a customer but the Shipping Service fails, the Payment Service must issue a refund.

Key Considerations:

  • Compensation is not always straightforward: Rolling back a transaction isn't always as simple as undoing a change. Some operations might be irreversible (e.g., shipping a product), requiring compensating actions that mitigate the issue (e.g., issuing a refund).
  • State-driven compensation: Services must maintain state information to understand which transactions require compensation.

3. Two-Phase Commit (2PC)​

Two-Phase Commit (2PC) is a protocol used for coordinating distributed transactions across multiple systems. It's traditionally used in tightly coupled distributed systems but is less popular in microservice architectures due to performance and complexity concerns.

How 2PC Works:​

  1. Prepare Phase: The transaction coordinator asks each microservice if they are ready to commit the transaction. The services reply with "yes" or "no."
  2. Commit/Rollback Phase: If all services reply with "yes," the coordinator instructs them to commit. If any service replies with "no," the coordinator sends a rollback command to all participating services.

Pros:

  • Strong consistency guarantees.
  • Ensures atomicity across distributed systems.

Cons:

  • Performance overhead: 2PC introduces a blocking wait time for services to prepare and commit, which can hurt performance in a microservices architecture.
  • Risk of blocking: If one service crashes, it can leave other services waiting indefinitely, potentially causing system-wide issues.

Given these challenges, 2PC is rarely used in modern, large-scale microservice architectures.


4. Eventual Consistency​

In distributed microservice architectures, eventual consistency is often the goal rather than strong consistency (ACID). Eventual consistency ensures that after some time, all participants in a distributed transaction will reach a consistent state, even if there are temporary inconsistencies.

How to Handle Rollbacks with Eventual Consistency:​

  • Delayed reconciliation: Some systems may use periodic reconciliation processes to detect inconsistencies and trigger compensating actions to ensure data integrity.
  • State-based updates: Services track their state, and if a failure is detected, each service is responsible for ensuring consistency through compensation.

Example: If a Payment Service charges a customer but the Shipping Service fails to ship the item, the system might temporarily be in an inconsistent state (payment made, no shipment). Over time, a reconciliation job or event listener detects the issue and initiates a refund to bring the system back to consistency.

Pros:

  • High scalability and resilience.
  • Tolerates temporary inconsistencies while ensuring long-term data integrity.

Cons:

  • Complex to reason about, as temporary inconsistencies can confuse users and developers.
  • Some actions (e.g., shipping a product) may be difficult to compensate.

5. Idempotency and Retries​

In microservice architectures, idempotency and retries play a critical role in handling rollbacks and retries, ensuring that repeated operations do not result in unwanted side effects.

Idempotency:​

An operation is idempotent if performing it multiple times has the same result as performing it once. For example, if a Payment Service is asked to process a payment twice, it should charge the customer only once.

Why It Matters:

  • When rolling back a transaction, services may need to retry operations (e.g., issuing a refund). These retries must be idempotent to prevent issues like charging a customer multiple times or reserving inventory repeatedly.
  • Retry policies are often implemented in communication mechanisms (e.g., message brokers) to ensure reliability.

Best Practices for Idempotency:​

  • Use unique transaction IDs for requests so that services can detect duplicate requests.
  • Design APIs and actions to ensure they are safe for retries.

6. Conclusion​

Rolling back or canceling transactions in a Microservice Architecture requires careful design to handle distributed data, asynchronous operations, and eventual consistency. The Saga Pattern (both choreography and orchestration) is the most widely used approach, allowing each service to independently handle parts of the transaction and apply compensating actions in case of failure. Other techniques like compensation mechanisms, eventual consistency, and idempotency ensure that microservices can maintain integrity, even across complex, distributed environments.

When choosing a rollback strategy, it's essential to balance the need for consistency with the performance and scalability requirements of the system.

Considerations on Transactions Across Distributed Systems

· 阅读需 7 分钟

Online Systems: Defining Transactions, Database Transaction Management, and Lock Management​

When working with complex online systems (e.g., e-commerce platforms, distributed systems, microservices architecture), transactions span beyond a single database. This requires a broader understanding of transaction management, involving distributed transactions, data consistency across systems, and sophisticated lock management techniques. Let's break this down with a focus on the key topics: defining transactions in online systems, transaction management (beyond the database), and lock management in such environments.


1. Defining Transactions in Online Systems​

In an online system, a transaction is a sequence of operations performed as a single logical unit, ensuring consistency, correctness, and isolation across different subsystems. These operations could involve:

  • Database operations (e.g., updating inventory, placing an order)
  • Third-party API calls (e.g., payment gateways, external services)
  • Message queue handling (e.g., Kafka, RabbitMQ for event-driven architecture)
  • Microservices interaction (e.g., coordination between an inventory service and an order service)

In simple terms, a transaction in an online system is any series of related actions across multiple components that must either complete successfully together or fail together to maintain a consistent system state.

Key Transaction Properties: ACID vs BASE​

In large-scale systems, transactions must be designed for both strong consistency (ACID) or eventual consistency (BASE), depending on the use case.

  1. ACID (Atomicity, Consistency, Isolation, Durability):

    • Guarantees strong consistency.
    • Used when all subsystems must maintain a single consistent state.
    • Common in single database transactions or within tightly-coupled services.
  2. BASE (Basically Available, Soft-state, Eventually consistent):

    • Sacrifices immediate consistency for availability and scalability.
    • Often used in distributed systems where components may not always be in sync (e.g., eventual consistency in NoSQL databases).

In online systems, distributed transactions often require a mix of ACID and BASE properties, depending on the criticality of the operation.


2. Transaction Management in Complex Online Systems​

Transaction management in online systems can no longer rely solely on single-database transaction managers. These systems often require distributed transaction management. Some of the most critical techniques include:

a. Distributed Transactions (Two-Phase Commit)​

When a transaction spans across multiple services or databases, a Two-Phase Commit (2PC) protocol is commonly used to ensure atomicity.

  • Phase 1 - Prepare: All participating systems are asked to prepare (lock resources and confirm they can commit the transaction).
  • Phase 2 - Commit/Rollback: If all participants agree, the transaction is committed. If any participant fails, the entire transaction is rolled back.

Example:

  • An order is placed in an online store:
    • The Order Service inserts an order into the database.
    • The Payment Service processes the payment.
    • The Inventory Service updates the stock.

All three operations must be successful for the order to be confirmed. The 2PC ensures that if any step fails, all previous operations are rolled back.

However, 2PC can be slow and does not scale well in high-throughput environments due to blocking resources during the "prepare" phase.

b. Saga Pattern​

The Saga Pattern is an alternative to 2PC, often used in microservices architectures where long-lived transactions are undesirable. Instead of blocking resources, the Saga breaks a distributed transaction into a series of smaller transactions.

  • Each service involved in the transaction performs its work independently.
  • If one step fails, the previous steps are compensated by executing defined rollback actions.

Example:

  1. Order Service places an order.
  2. Payment Service charges the customer.
  3. Inventory Service updates the stock.
  4. If the Inventory Service fails, a compensating transaction refunds the payment and cancels the order.

Sagas offer better scalability compared to 2PC and are particularly suited for event-driven architectures where each microservice manages its own transaction.

c. Eventual Consistency and Compensation​

In some distributed systems, strict ACID properties are relaxed for eventual consistency, especially in BASE systems (e.g., NoSQL databases, cloud-native architectures). Instead of locking resources, systems rely on compensation logic:

  • If a downstream service fails or an external API does not respond, the system might retry the operation or initiate compensating actions to restore consistency.

Eventual Consistency Example:

  • In a global payment system, payments made across multiple time zones and currencies may not reflect immediately but will eventually be reconciled to ensure data consistency.

d. Message-Driven Transactions​

In event-driven systems, message queues (e.g., Kafka, RabbitMQ) help decouple services, allowing them to handle transactions asynchronously. Here, services:

  • Emit events for actions like "Order Placed" or "Payment Processed".
  • Listen for events from other services to trigger corresponding actions (e.g., update inventory when an order is placed).

By designing for idempotency (ensuring repeated operations have the same effect as a single operation), these systems can handle eventual consistency and retries gracefully.


3. Lock Management​

In online systems, lock management becomes complex due to multiple participants, different types of data stores, and network latencies. Locking is crucial to prevent data corruption or race conditions during concurrent transactions.

a. Optimistic vs Pessimistic Locking​

  • Pessimistic Locking: A resource is locked for the entire duration of the transaction, preventing any other transaction from accessing it.

    • Used when contention for the resource is high or when updates must be serialized (e.g., bank account transfers).
    • Example: A row in the Inventory table is locked while updating stock.
    SELECT * FROM Products WHERE product_id = 1 FOR UPDATE;
  • Optimistic Locking: Assumes conflicts are rare and does not lock the resource. Instead, it checks for conflicts only at the end of the transaction.

    • Version columns or timestamps are used to detect if another transaction modified the same data. If so, the transaction fails, and a retry is triggered.
    • Example: Update an order only if no other transaction has modified it.
    UPDATE Orders
    SET status = 'SHIPPED'
    WHERE order_id = 101 AND version = 1;

b. Distributed Locks​

In distributed systems, transactions may need to span multiple nodes or databases. To handle this, distributed locks ensure that shared resources are not modified concurrently by different parts of the system.

  1. Redis-Based Locks: A simple solution is to use Redis to acquire distributed locks.

    Example:
    When updating inventory, each microservice tries to acquire a lock for the product:

    SET product_lock:1 my_unique_id NX PX 10000
    • NX ensures the lock is acquired only if it doesn't exist.
    • PX 10000 ensures the lock expires after 10 seconds if not released.
  2. Zookeeper or Etcd: More robust systems like Zookeeper or Etcd can handle distributed coordination, offering leader election, mutexes, and barriers.

    Example: Distributed systems can use Zookeeper to ensure that only one instance of a service performs a critical operation (e.g., process payments in a sharded environment).

c. Deadlock Detection and Prevention​

In systems with complex transactions and lock dependencies, deadlocks can occur, where two transactions are waiting for resources locked by each other.

  • Deadlock Detection: The system detects circular dependencies and aborts one of the transactions to resolve the deadlock.

    Example:

    SET innodb_lock_wait_timeout = 50;
  • Deadlock Prevention: In systems like databases or distributed systems, adopting strategies like acquiring locks in a consistent order can prevent deadlocks.


Conclusion​

In an online system, transaction management and lock management are more complex than in a simple intranet application. The system must account for distributed transactions across multiple services and databases, often relying on protocols like 2PC, Sagas, or message-driven event handling to ensure consistency and reliability. Locks, whether in-database or distributed, are critical for managing concurrency and ensuring data integrity but must be handled with care to avoid performance bottlenecks or deadlocks.

In essence, designing transactions in a modern online system requires a mix of strategies from distributed transaction management, eventual consistency models, and advanced locking mechanisms to maintain data reliability, availability, and scalability.