Package org.elasticsearch.xpack.esql.datasources.cache

Record Class SchemaCacheKey

java.lang.Object
java.lang.Record
org.elasticsearch.xpack.esql.datasources.cache.SchemaCacheKey
public record SchemaCacheKey(String canonicalPath, long lastModifiedEpochMillis, String formatType, String formatConfig, String endpoint, String region, FileSetFingerprint fileSetFingerprint) extends Record
Cache key for schema inference results. Includes mtime-in-key for invalidation. Endpoint and region are included because the same canonical path on different endpoints resolves to different objects.

fileSetFingerprint carries the 128-bit fingerprint of the resolved file set for a dataset-level aggregate key (see forDatasetAggregate(java.lang.String, org.elasticsearch.xpack.esql.datasources.FileSetFingerprint, java.lang.String, java.util.Map<java.lang.String, java.lang.Object>)); it is null for every per-file key. A named component rather than smuggling the fingerprint into the mtime/path slots — record equality/hashCode pick it up automatically.

  • Field Details

    • STRICT_DECLARED_SCHEMA_MARKER

      public static final String STRICT_DECLARED_SCHEMA_MARKER
      Reserved formatType suffix namespace: extension detection (detectFormatType) derives formatType from a file name's last dot, so for any sane object name a '#'-suffixed formatType is minted only by an explicit factory. (A pathological object name literally containing '#dataset-agg' would collide on the suffix, but a per-file key carries a null fileSetFingerprint so it can never equal a dataset key - the only cost is that one file losing its warm enrichment, a miss, never a wrong answer.) Two members exist: STRICT_DECLARED_SCHEMA_MARKER (per-file entries on the strict-declared warm rail, which the reconcile's contribution matching MUST still reach) and DATASET_AGGREGATE_MARKER (dataset-level aggregate entries, which contribution matching must NEVER reach - enforced in ExternalSourceCacheService#matchesContribution). Co-located here so their distinctness is visible at the declaration site.
      See Also:

    • DATASET_AGGREGATE_MARKER

      public static final String DATASET_AGGREGATE_MARKER
      See Also:

  • Constructor Details

    • SchemaCacheKey

      public SchemaCacheKey(String canonicalPath, long lastModifiedEpochMillis, String formatType, String formatConfig, String endpoint, String region, @Nullable FileSetFingerprint fileSetFingerprint)
      Creates an instance of a SchemaCacheKey record class.
      Parameters:
      canonicalPath - the value for the canonicalPath record component
      lastModifiedEpochMillis - the value for the lastModifiedEpochMillis record component
      formatType - the value for the formatType record component
      formatConfig - the value for the formatConfig record component
      endpoint - the value for the endpoint record component
      region - the value for the region record component
      fileSetFingerprint - the value for the fileSetFingerprint record component

  • Method Details

    • build

      public static SchemaCacheKey build(String canonicalPath, long mtime, String formatType, Map<String,Object> config)

    • forDatasetAggregate

      public static SchemaCacheKey forDatasetAggregate(String pattern, FileSetFingerprint fingerprint, String sourceType, Map<String,Object> config)
      Key for a dataset-level aggregate entry: the memoized multi-file stats fold for one resolved file SET under one format config. Identity is the listing's 128-bit file-set fingerprint (a commutative fold of every file's path + mtime + size, plus the file count - see FileList#fileSetFingerprint), which makes the key correct-or-miss by construction: any file added, removed, or modified derives a different key, and the stale entry simply ages out via LRU/TTL - no invalidation protocol. The fingerprint rides the dedicated fileSetFingerprint record component; canonicalPath is the glob pattern (diagnostics-friendly) and the marker-suffixed formatType keeps these entries out of the per-file contribution-matching paths.

      Known residual, inherited from the per-file rail: under a lenient error policy (skip_row/null_field) a harvested row count can be declaration-dependent (see the warmsRowCountSafely discussion on the strict single-file rail). The dataset aggregate memoizes exactly what the per-file rail serves, so it neither narrows nor widens that residual - both must be closed together by the declared-schema fingerprint follow-up.

    • isDatasetAggregate

      public boolean isDatasetAggregate()
      True when this key addresses a dataset-level aggregate entry (minted by forDatasetAggregate(java.lang.String, org.elasticsearch.xpack.esql.datasources.FileSetFingerprint, java.lang.String, java.util.Map<java.lang.String, java.lang.Object>)) rather than a per-file schema entry. Centralizes the DATASET_AGGREGATE_MARKER check so the taxonomy lives with the key instead of being re-derived at each call site.

    • buildFormatConfig

      public static String buildFormatConfig(Map<String,Object> config)
      Canonical, node-stable identity of the row-interpretation-affecting config: the format-affecting params (credentials and non-format keys excluded), sorted and rendered key=value,.... Deterministic across JVMs and independent of column projection, so a coordinator and a data node derive the same string for the same logical query config — the basis for the cross-node stats cache fingerprint.

    • toString

      public final String toString()
      Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.
      Specified by:
      toString in class Record
      Returns:
      a string representation of this object

    • hashCode

      public final int hashCode()
      Returns a hash code value for this object. The value is derived from the hash code of each of the record components.
      Specified by:
      hashCode in class Record
      Returns:
      a hash code value for this object

    • equals

      public final boolean equals(Object o)
      Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. Reference components are compared with Objects::equals(Object,Object); primitive components are compared with '=='.
      Specified by:
      equals in class Record
      Parameters:
      o - the object with which to compare
      Returns:
      true if this object is the same as the o argument; false otherwise.

    • canonicalPath

      public String canonicalPath()
      Returns the value of the canonicalPath record component.
      Returns:
      the value of the canonicalPath record component

    • lastModifiedEpochMillis

      public long lastModifiedEpochMillis()
      Returns the value of the lastModifiedEpochMillis record component.
      Returns:
      the value of the lastModifiedEpochMillis record component

    • formatType

      public String formatType()
      Returns the value of the formatType record component.
      Returns:
      the value of the formatType record component

    • formatConfig

      public String formatConfig()
      Returns the value of the formatConfig record component.
      Returns:
      the value of the formatConfig record component

    • endpoint

      public String endpoint()
      Returns the value of the endpoint record component.
      Returns:
      the value of the endpoint record component

    • region

      public String region()
      Returns the value of the region record component.
      Returns:
      the value of the region record component

    • fileSetFingerprint

      @Nullable public FileSetFingerprint fileSetFingerprint()
      Returns the value of the fileSetFingerprint record component.
      Returns:
      the value of the fileSetFingerprint record component