Package org.elasticsearch.xpack.esql.datasources.spi

Class DeclaredTypeCoercions

java.lang.Object
org.elasticsearch.xpack.esql.datasources.spi.DeclaredTypeCoercions
public final class DeclaredTypeCoercions extends Object
The single source of truth for declared-type coercion on external datasets: which (physical file type → declared type) pairs an external reader coerces at read time, how a decoded physical block becomes a declared-type block (castBlock(org.elasticsearch.compute.data.Block, org.elasticsearch.xpack.esql.core.type.DataType, org.elasticsearch.xpack.esql.core.type.DataType, org.elasticsearch.common.time.DateFormatter, org.elasticsearch.compute.data.BlockFactory, java.lang.String, org.elasticsearch.xpack.esql.datasources.spi.SkipWarnings)), the block shape a declared type reads into (elementTypeFor(org.elasticsearch.xpack.esql.core.type.DataType)/builderFor(org.elasticsearch.xpack.esql.core.type.DataType, org.elasticsearch.compute.data.BlockFactory, int), which every text and columnar reader consults instead of re-deriving it), and the one scalar conversion the string → date pair uses everywhere (parseDatetimeMillis(java.lang.String, org.elasticsearch.common.time.DateFormatter)).

The one concept: reading a file IS ingesting it

A dataset mapping may declare a column type that differs from the type physically in the file (Hive/Trino-style: the declaration is the table schema, readers coerce toward it). Reading a file value against a declared column is the same operation as indexing a document field against a mapping, so the coercion authority is the field mappers' lenient index-time coercion ("123" → long, long → double, string → datetime via the column's declared format, …), not ES|QL's query-cast rules. Once a value has been coerced into the declared shape it is an ordinary ES|QL value and query-layer conversions (::, TO_*) apply downstream as usual.

What is coercible — the mapper coercion set

supports(org.elasticsearch.xpack.esql.core.type.DataType, org.elasticsearch.xpack.esql.core.type.DataType) follows what the field mappers accept at ingest, deviating on the safe (reject) side where a mapper is more permissive than a reader can be faithful to — the date mapper's default format also parses numeric and fractional tokens (epoch_millis halves a 1.5 token down to a truncated instant), while supports(DOUBLE, DATETIME) is deliberately false because a fractional value has no unambiguous epoch reading:
  • whole-number targets (integer/long): any numeric or string source, reusing the ES|QL :: cast engine (numericCoercer(org.elasticsearch.xpack.esql.core.type.DataType, org.elasticsearch.xpack.esql.core.type.DataType)) so a declared read is value-identical to an explicit ::long/::integer — numeric strings parse (fractional and scientific accepted), the result rounds (not truncates), out-of-range throws InvalidArgumentException. unsigned_long keeps its ::-faithful coerceToUnsignedLong(java.lang.Object) twin (truncates toward zero, matching ::unsigned_long); double parses with Double.parseDouble and returns the IEEE value as-is (NaN/Infinity pass through, matching the native columnar double read and CSV — an external read preserves the file's value; the mapper's finite-only rule is an index-time concern, not a read one);
  • string targets (keyword/text): any decodable scalar source — ingest stringifies the token (temporal sources render in the ISO form the default date format parses back; ip sources render as address text, never the encoded bytes). The source set is closed over exactly the types the readers can decode a block of (string, whole-number, double, boolean, temporal, ip) so a pair supports admits can never reach a value reader that has no arm for it;
  • boolean: string sources only, parsed strictly and case-insensitively (strictParseBoolean(java.lang.String): only true/false in any case; every other token fails loudly). This deliberately diverges from ::boolean, which maps a non-true token silently to false — a silent wrong answer this read must not introduce;
  • datetime: string sources parse via parseDatetimeMillis(java.lang.String, org.elasticsearch.common.time.DateFormatter) with the column's declared format (else the ISO default), whole-number sources reinterpret as epoch milliseconds (the epoch_millis half of the default date format);
  • date_nanos: string sources parse ISO, datetime sources widen millis→nanos (what an epoch-millis token ingests to in a date_nanos field; out-of-nanos-range instants fail per value). Plain whole numbers stay out — a raw long is ambiguous between a millis and a nanos payload;
  • ip: string sources only, parsed with the same underlying primitive the ip mapper delegates to (InetAddresses parse + the 16-byte doc-values encoding).
NULL/UNSUPPORTED physical columns support nothing (the readers cannot decode a value to coerce). An unsupported pair is rejected at resolution with an actionable error; there is no third state — a declared type that cannot be produced from the physical column must never silently read as null.

Where the conversion runs — same predicate, different timing

TODO: the columnar-vs-text classification this predicate pairs with (ExternalSourceResolver.FILE_TYPED_FORMATS) has a standing TODO to move onto the FormatReader SPI as a capability; if that happens, per-format coercion support belongs on the same capability surface.