Module org.elasticsearch.compute
Package org.elasticsearch.compute.operator

Interface CapturingExternalSourceStatus

public interface CapturingExternalSourceStatus
Implemented by Operator.Status subclasses that capture per-file source-metadata contributions during data-node execution. The contributions are aggregated by DriverCompletionInfo into the return-flow back to the coordinator so the coordinator can enrich its SchemaCacheEntry for future queries.

The capture contract follows the well-known _stats.* flat-map key convention used by esql's SourceStatisticsSerializer, so the coordinator's existing aggregation primitive (SourceStatisticsSerializer.mergeStatistics) can merge across data nodes / chunks without format-specific branching.

  • Method Summary

    Modifier and Type
    Method
    Description
    Map<String,List<Map<String,Object>>>
    capturedSourceMetadata()
    Per-file capture surface.
    default boolean
    partial()
    Whether this operator returned partial results because a lenient policy dropped data during the read (e.g.

  • Method Details

    • capturedSourceMetadata

      Map<String,List<Map<String,Object>>> capturedSourceMetadata()
      Per-file capture surface. Keyed by file path; the value is the list of flat _stats.* contributions from each iterator that touched the file — one per chunk for parallel parsing, one per split for macro-splits, one for whole-file reads.

      DriverCompletionInfo's factories collect these from every completed operator and concatenate per file path into DriverCompletionInfo.capturedSourceMetadata. The per-path list is merged later by esql-side consumers (the compute module cannot reach the merge primitive). Returning Map.of() when nothing was captured is fine.

    • partial

      default boolean partial()
      Whether this operator returned partial results because a lenient policy dropped data during the read (e.g. a max_record_size truncation under a non-strict error_mode). DriverCompletionInfo's factories OR this across all completed operators so the coordinator can flip the response's is_partial flag. Defaults to false; only operators that can lose data under a lenient policy override it.