Skip to main content

Transaction Priority Admission

Transaction priority admission lets Kahuna decide which transaction starts next when a node is saturated.

By default the feature is disabled. MaxConcurrentTransactions and MaxConcurrentSessions both default to 0, which means every transaction starts immediately. In that pass-through mode, Kahuna still records priority for metrics, but priority does not queue, reorder, or throttle anything.

When an operator enables a concurrency ceiling, transactions that cannot start immediately wait in a per-node priority queue. Kahuna starts the highest-priority eligible waiter first.

Priority affects only admission:

  • It governs when a transaction starts.
  • It does not preempt a transaction that is already running.
  • It does not change MVCC, locks, two-phase commit, durable decisions, or commit semantics.
  • It is per node, so each node orders only the transactions it receives.

Priority Levels

PriorityValueUse for
Background0Bulk or deferrable work such as analytics sweeps.
Low1Work below ordinary traffic but still latency-relevant.
Normal2Default for callers that do not specify a priority.
High3Latency-critical application transactions.
Critical4Work that must not be deferred behind anything else.

Do not mark ordinary traffic as Critical. If everything is critical, priority stops carrying useful information and the reserve cannot protect genuinely urgent work.

Unknown priority values are normalized to Normal, not clamped upward. That prevents untrusted or newer clients from accidentally claiming critical priority.

What Is Gated

The admission gate applies to transactions:

  • explicit BEGIN ... COMMIT ... END scripts
  • multi-statement scripts that run as auto-commit transactions
  • interactive transaction sessions

Single standalone script commands are not gated. A script that contains only one SET, GET, DELETE, EXTEND, bucket read, prefix scan, or ephemeral equivalent runs directly against the store. Priority can be supplied for those calls, but it is ignored because no transaction is opened.

Setting Priority

Interactive Sessions

await using KahunaTransactionSession session =
await client.StartTransactionSession(new KahunaTransactionOptions
{
Priority = TransactionPriority.High,
Timeout = 30_000,
AdmissionWaitMs = 2_000
});

Script Execution

KahunaKeyValueTransactionResult result =
await client.ExecuteKeyValueTransactionScript(
script,
hash: null,
parameters: parameters,
priority: TransactionPriority.Background
);

Compiled scripts expose the same control:

KahunaTransactionScript compiled = client.LoadTransactionScript(script);

KahunaKeyValueTransactionResult result =
await compiled.Run(TransactionPriority.High, parameters);

Inline Script Option

A script can set its own priority. Inline priority overrides the priority carried by the transport.

begin (priority="high", locking="optimistic")
let row = get `orders/42`
set `orders/42` row
commit
end

Accepted inline priority values are background, low, normal, high, and critical.

Scripts can also set admissionWait, in milliseconds. This is how long the caller is willing to wait for a slot before the transaction starts. It is separate from timeout, which bounds how long the transaction may run after it has been admitted.

begin (priority="high", admissionWait=2000, timeout=10000)
let row = get `orders/42`
set `orders/42` row
commit
end

Capacity Model

Kahuna has two independent admission gates:

GateCeilingSlot held for
ScriptMaxConcurrentTransactionsThe duration of the script transaction execution.
SessionMaxConcurrentSessionsAs long as the interactive session remains open.

They are separate because scripts and sessions hold capacity for very different lengths of time. A script is bounded by its execution. An interactive session is client-paced and can sit idle while it remains open.

Size MaxConcurrentSessions more generously than MaxConcurrentTransactions. It limits open sessions, not just actively running operations.

Reserved Slots

TransactionPriorityReservedSlots protects capacity for High and Critical transactions.

For each gate:

total in flight <= MaxConcurrent...
ordinary in flight <= MaxConcurrent... - TransactionPriorityReservedSlots

Background, Low, and Normal are ordinary priorities. They cannot occupy reserved slots. High and Critical can use both ordinary capacity and the reserve.

This prevents bulk traffic from filling every slot on a saturated node.

Aging

Strict priority can starve low-priority work. Kahuna avoids that with aging.

For every TransactionPriorityAgingThreshold milliseconds spent waiting, a queued transaction gains one effective priority level for dispatch ordering. Aging is capped at High; Critical is never aged because it is already at the top.

Aging changes position in line, not eligibility for reserved capacity. An aged Background transaction can overtake newer ordinary work, but it still cannot consume a reserved slot.

Set TransactionPriorityAgingThreshold to 0 to disable aging.

Queue Backpressure

TransactionPriorityMaxQueued bounds how many callers may wait per gate. When the queue is full, Kahuna refuses admission immediately and returns AdmissionRefused.

No transaction has started in that case, so retrying is safe. Clients should back off and retry rather than treating it as a conflict.

Admission wait is budgeted separately from transaction lifetime:

  • AdmissionWaitMs on KahunaTransactionOptions controls how long an interactive session waits to start.
  • admissionWait in a script begin (...) block controls how long that script waits to start.
  • timeout still controls the admitted transaction lifetime or script execution time.

If the caller's admission budget expires before a slot opens, Kahuna returns AdmissionRefused.

AdmissionRefused and MustRetry are both retryable, but they mean different things. Use backoff for AdmissionRefused because the node is shedding load. Retry MustRetry promptly because it usually means a transient routing, leadership, or replication condition.

Configuration

SettingCLI flagDefaultMeaning
MaxConcurrentTransactions--max-concurrent-transactions0Script transactions running at once. 0 disables the script gate.
MaxConcurrentSessions--max-concurrent-sessions0Interactive sessions open at once. 0 disables the session gate.
TransactionPriorityReservedSlots--transaction-priority-reserved-slots0Slots only High and Critical transactions may occupy.
TransactionPriorityAgingThreshold--transaction-priority-aging-threshold1000Milliseconds of waiting per effective priority promotion. 0 disables aging.
TransactionPriorityMaxQueued--transaction-priority-max-queued4096Waiting callers per gate before new arrivals receive AdmissionRefused. 0 makes the wait queue unbounded.
DefaultAdmissionWaitMs--default-admission-wait5000Admission wait used when the caller does not specify one.
MaxAdmissionWaitMs--max-admission-wait30000Maximum admission wait allowed by the server. Caller-supplied waits are clamped to this value.

Start with pass-through mode and observe the metrics before setting a ceiling. Set ceilings near healthy observed concurrency, then add a small reserve only if high-priority work queues behind bulk work.

Metrics

Transaction admission metrics are published on the Kahuna meter. Each instrument is tagged by gate (script or session) and priority.

InstrumentMeaning
kahuna.tx_admission.in_flightTransactions currently holding an admission slot.
kahuna.tx_admission.queuedTransactions currently waiting for a slot. Non-zero means the gate is active.
kahuna.tx_admission.max_queue_depthHigh-water mark of simultaneous waiters.
kahuna.tx_admission.admittedTransactions admitted since startup.
kahuna.tx_admission.aged_promotionsWaiters promoted by aging at least once.
kahuna.tx_admission.abandoned_while_waitingWaiters whose admission budget expired before they started.
kahuna.tx_admission.rejected_queue_fullRequests refused because the wait queue was full.

Watch queued first. If it stays zero, the gate is transparent. If queued is sustained for High or Critical, the ceiling may be too low or a reserve may be needed. If rejected_queue_full rises, the node is shedding transaction starts and callers should be backing off.