Package org.elasticsearch.xpack.esql.datasources.spi

Class DirectMemoryDebug

java.lang.Object
org.elasticsearch.xpack.esql.datasources.spi.DirectMemoryDebug
public final class DirectMemoryDebug extends Object
Use-after-free detection for off-heap (direct) read buffers.

Direct read buffers are exposed to decoders as detached ByteBuffer views over ArrowBuf native memory. Reads through those views bypass Arrow's reference-count tracking, so neither the Arrow debug allocator nor a Releasable can observe a read that aliases a buffer after it has been freed. That is the failure mode behind the nondeterministic zstd corruption ("Src size is incorrect" / "Destination buffer is too small"): a page slice handed out before free is read after the backing region was returned to the allocator and recycled — and it only manifests when the region happens to have been overwritten already.

Two independently-gated launch-time knobs (es.arrow.* system properties) remove that timing dependency by overwriting freed memory with a recognizable sentinel, so a surviving alias reads deterministically-corrupt bytes on every run:

  • es.arrow.poison_freed_buffers (default true) — the cheap, always-on tripwire. Poisons only the buffer header (64 bytes) just before free. A use-after-free that re-reads a zstd/Parquet page header (the common path) fails immediately and loudly instead of corrupting results. Cost is a handful of bytes written per buffer free — negligible against any real scan, so it is safe to leave enabled in production and on a performance run. When disabled the hot path is a single field read. Note: on a coalesced buffer (one DirectReadBuffer spanning several merged ranges) only the first constituent's header sits in the poisoned region; a use-after-free deeper in the buffer is caught only by the full-buffer mode below.
  • es.arrow.debug_buffers (default: on when assertions are enabled) — the thorough, opt-in mode for a correctness-validation pass. Poisons the whole buffer and enables DirectReadBuffer's allocation/free stack-trace capture plus its double-free / use-after-close throws (see trackingEnabled()). It is an explicit check rather than a Java assert, so it can be turned on with -Des.arrow.debug_buffers=true on a node that is not running with -ea. Slower (an extra write-pass over freed bytes, plus a stack walk per allocation), so not for headline timing runs.

The Arrow debug allocator (enabled via arrow.memory.debug.allocator in test builds) is a separate, heavier, test-only layer; it catches double-free and leaks through ArrowBuf, which these knobs deliberately do not duplicate.

  • Method Summary

    Modifier and Type
    Method
    Description
    static void
    poison(ByteBuffer buffer)
    Overwrite the direct memory backing buffer with the poison sentinel, just before the buffer is released — the whole buffer under es.arrow.debug_buffers, otherwise only the header under es.arrow.poison_freed_buffers.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • poison

      public static void poison(ByteBuffer buffer)
      Overwrite the direct memory backing buffer with the poison sentinel, just before the buffer is released — the whole buffer under es.arrow.debug_buffers, otherwise only the header under es.arrow.poison_freed_buffers. No-op when both knobs are off, on heap-backed buffers (test stubs), and does not mutate the caller's position/limit.