Skip to main content

Distributed Locks

In one process, a mutex or a lock statement guarantees that only one thread enters a critical section. Across several servers there is no shared memory to hold that mutex, and any server can pause or lose its network connection while it holds the lock. A distributed lock gives processes on different machines the same guarantee: one owner at a time, so they do not modify a shared resource concurrently.

Kahuna adds what a single-process lock does not need:

  • A lease: every lock expires unless its owner extends it, so a crashed or disconnected owner cannot block it forever.
  • A fencing token: a number that increases each time the lock changes hands, so the resource you protect can reject a late write from an owner whose lease already expired.
  • Replication for persistent locks: a persistent lock is stored by the replicas of its partition and committed by a quorum through Raft, so it stays consistent through node and network failures. Ephemeral locks are faster, but they live only in the partition leader's memory and are lost on a restart or leader change.

Kahuna batches persistent acquire, extend, and release operations per partition before proposing them to Raft. Bursts of requests that touch locks on the same partition therefore share fewer WAL appends and quorum round trips, with the same fencing-token and lease semantics.

A lock is either free or held. Only the current owner can extend or unlock it; a lock request from another owner is refused until the lease ends:

Use Cases​

  • Leader Election: Elect a single leader in a cluster of services. Only one node should act as the leader at any time (e.g., for scheduling, replication). A distributed lock ensures only one process "wins" and holds the leadership.
  • Preventing Double Execution of Scheduled Jobs: Ensure a cron job or background worker is executed only once across multiple nodes. In a horizontally scaled system, multiple nodes might try to run the same job. A distributed lock prevents multiple executions of the same scheduled task.
  • Safe Deployment CI/CD Coordination: Prevent multiple CI/CD pipelines from deploying the same environment simultaneously. Two deployment jobs could conflict and cause downtime. Use a lock to serialize deployments.
  • Database Migration Coordination: Ensure that only one service performs schema migration at startup. If multiple services run migrate up concurrently, it may corrupt the schema. Use a lock to ensure only the first instance runs migrations.
  • Session Control / Login Exclusivity: Allow only one active session per user. Used in banking apps, gaming, admin consoles, etc.
  • Distributed Queue Consumer Coordination: Keep two consumers from processing the same message at the same time. A lock alone does not guarantee the message is processed only once, because a consumer can crash after doing the work and before recording it. Record completion too; see Idempotent Jobs.
  • Throttling or Rate-Limiting Across Services: A global rate limit is a shared counter rather than a lock. Use the distributed rate limiters, which also include lease-based concurrency limits.
  • External System Coordination: Ensure only one node writes to an external system (e.g., shared database, billing API, payment gateway) to avoid double charges or inconsistent writes.

API​

The lock API has four operations: lock, unlock, extend, and get.

Lock​

Task<KahunaLock> GetOrCreateLock(
    string resource,
    TimeSpan expiry,
    LockDurability durability = LockDurability.Persistent
);
  • resource: The identifier for the resource you want to lock.
  • expiry: The duration for the lock lease.
  • durability: Defines whether the lock durability is Ephemeral or Persistent.

Returns:

  • IsAcquired: true if the lock was successfully acquired.
  • FencingToken: A monotonically increasing token for that lock resource. Downstream systems can reject operations with an older token.

Unlock​

Task<bool> Unlock(
    string resource,
    byte[] owner,
    LockDurability durability = LockDurability.Persistent
);
  • resource: The identifier for the resource to unlock.
  • owner: The unique identifier for the lock previously used to acquire the lock.
  • durability: Defines whether the lock durability is Ephemeral or Persistent.

Returns:

  • Unlocked: true if the resource was successfully unlocked.

Extend​

Task<(bool, long)> TryExtendLock(
    string resource,
    byte[] owner,
    TimeSpan duration,
    LockDurability durability = LockDurability.Persistent
);
  • resource: The identifier for the resource you want to extend.
  • owner: A unique identifier for the lock, usually associated with the client or process requesting the lock. It must be the current owner of the lock.
  • duration: The new lock lease duration.
  • durability: Defines whether the lock durability is Ephemeral or Persistent.

Returns:

  • Extended (first item): true if the lock was successfully extended. If the lease already expired or another owner holds the lock, the call throws a KahunaException instead.
  • FencingToken (second item): The current fencing token for that lock resource. Extending a lock keeps the same owner and token.

Get​

Task<KahunaLockInfo?> GetLockInfo(
    string resource,
    LockDurability durability = LockDurability.Persistent
);
  • resource: The identifier for the resource you want to get information.
  • durability: Defines whether the lock durability is Ephemeral or Persistent.

Returns:

  • Owner: The current owner of the lock.
  • FencingToken: The current fencing token for that lock resource.

Important Considerations​

  • Choose the durability per lock. A persistent lock survives node restarts and leader changes, and its fencing token always increases. An ephemeral lock is faster but lives only in the partition leader's memory. A node restart, a leader change, or eviction of an idle lock entry loses it: another worker can acquire it before the lease would have ended, and its fencing token count restarts at 0. Use ephemeral locks only where a short overlap between two holders is acceptable. See Durability Levels and Leases During a Leader Change.
  • A lease can end while its owner is still working. Extend the lease as you work and stop when an extend fails (Renewing a Lease). Send the fencing token with every write to the resource you protect, so a late writer is rejected.