Package org.elasticsearch.xpack.esql.datasources.glob

Class GlobExpander

java.lang.Object

org.elasticsearch.xpack.esql.datasources.glob.GlobExpander

public final class GlobExpander
extends Object

Expands glob patterns and comma-separated path lists into resolved FileList instances. Delegates to StorageProvider.listObjects(org.elasticsearch.xpack.esql.datasources.spi.StoragePath, boolean) for directory listing and uses GlobMatcher for filtering results against the glob pattern. Supports partition-aware glob rewriting when filter hints are provided.

Method Summary

Modifier and Type	Method	Description
static List<StorageEntry>	applyFileMetadataFilters(List<StorageEntry> entries, List<PartitionFilterHintExtractor.PartitionFilterHint> hints)
static FileList	compact(FileList raw, String basePath)	Compresses a raw file list into a compact representation (dictionary or Hive-partitioned).
static FileList	expandAndCompact(String path, StorageProvider provider, List<PartitionFilterHintExtractor.PartitionFilterHint> hints, boolean hivePartitioning, StoragePath storagePath)	Expands a glob/comma pattern and compresses the result into a compact representation (DictionaryFileList or HiveFileList).
static FileList	expandAndCompact(String path, StorageProvider provider, List<PartitionFilterHintExtractor.PartitionFilterHint> hints, boolean hivePartitioning, StoragePath storagePath, int maxDiscoveredFiles, int maxGlobExpansion)	Expands a glob/comma pattern and compresses the result, with safety caps on discovery.
static FileList	expandCommaSeparated(String pathList, StorageProvider provider)
static FileList	expandCommaSeparated(String pathList, StorageProvider provider, List<PartitionFilterHintExtractor.PartitionFilterHint> hints, boolean hivePartitioning)
static FileList	expandCommaSeparated(String pathList, StorageProvider provider, List<PartitionFilterHintExtractor.PartitionFilterHint> hints, boolean hivePartitioning, int maxDiscoveredFiles, int maxGlobExpansion)
static FileList	expandGlob(String pattern, StorageProvider provider)
static FileList	expandGlob(String pattern, StorageProvider provider, List<PartitionFilterHintExtractor.PartitionFilterHint> hints, boolean hivePartitioning)
static FileList	expandGlob(String pattern, StorageProvider provider, List<PartitionFilterHintExtractor.PartitionFilterHint> hints, boolean hivePartitioning, int maxDiscoveredFiles, int maxGlobExpansion)
static FileList	expandGlob(String pattern, StorageProvider provider, List<PartitionFilterHintExtractor.PartitionFilterHint> hints, boolean hivePartitioning, PartitionConfig partitionConfig, Map<String,Object> config)
static FileList	fileListOf(List<StorageEntry> entries, String pattern)	Creates a file list from raw entries.
static FileList	fileListOf(List<StorageEntry> entries, String pattern, PartitionMetadata partitionMetadata)	Creates a file list from raw entries with partition metadata.
static boolean	isMultiFile(String path)	Returns true if the given path string represents multiple files — either because it contains glob metacharacters in the path component, or because it is a comma-separated list.
static PartitionDetector	resolveDetector(PartitionConfig config)
static String	rewriteGlobWithHints(String pattern, List<PartitionFilterHintExtractor.PartitionFilterHint> hints)
static String	rewriteGlobWithHints(String pattern, List<PartitionFilterHintExtractor.PartitionFilterHint> hints, PartitionConfig partitionConfig)
static String	rewriteGlobWithTemplate(String pattern, Map<String,PartitionFilterHintExtractor.PartitionFilterHint> rewritableHints, String template)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

fileListOf

public static FileList fileListOf(List<StorageEntry> entries,
 String pattern)

Creates a file list from raw entries. Primarily for tests.

fileListOf

public static FileList fileListOf(List<StorageEntry> entries,
 String pattern,
 @Nullable
 PartitionMetadata partitionMetadata)

Creates a file list from raw entries with partition metadata. Primarily for tests.

compact

public static FileList compact(FileList raw,
 String basePath)

Compresses a raw file list into a compact representation (dictionary or Hive-partitioned).

expandAndCompact

public static FileList expandAndCompact(String path,
 StorageProvider provider,
 @Nullable
 List<PartitionFilterHintExtractor.PartitionFilterHint> hints,
 boolean hivePartitioning,
 StoragePath storagePath)
                                 throws IOException

Expands a glob/comma pattern and compresses the result into a compact representation (DictionaryFileList or HiveFileList). This is the primary entry point for the resolver.

Throws:

IOException

expandAndCompact

public static FileList expandAndCompact(String path,
 StorageProvider provider,
 @Nullable
 List<PartitionFilterHintExtractor.PartitionFilterHint> hints,
 boolean hivePartitioning,
 StoragePath storagePath,
 int maxDiscoveredFiles,
 int maxGlobExpansion)
                                 throws IOException

Expands a glob/comma pattern and compresses the result, with safety caps on discovery.

Throws:

IOException

resolveDetector

public static PartitionDetector resolveDetector(PartitionConfig config)

isMultiFile

public static boolean isMultiFile(String path)

Returns true if the given path string represents multiple files — either because it contains glob metacharacters in the path component, or because it is a comma-separated list. IPv6 host literals in URL authorities (e.g. http://[::1]/data/*.parquet) use bracket notation per RFC 3986 §3.2.2. Those brackets are parsed as part of the authority, not the path, so they are not treated as glob character-class syntax.

expandGlob

public static FileList expandGlob(String pattern,
 StorageProvider provider)
                           throws IOException

Throws:

IOException

expandGlob

public static FileList expandGlob(String pattern,
 StorageProvider provider,
 @Nullable
 List<PartitionFilterHintExtractor.PartitionFilterHint> hints,
 boolean hivePartitioning,
 @Nullable
 PartitionConfig partitionConfig,
 @Nullable
 Map<String,Object> config)
                           throws IOException

Throws:

IOException

expandGlob

public static FileList expandGlob(String pattern,
 StorageProvider provider,
 @Nullable
 List<PartitionFilterHintExtractor.PartitionFilterHint> hints,
 boolean hivePartitioning)
                           throws IOException

Throws:

IOException

expandGlob

public static FileList expandGlob(String pattern,
 StorageProvider provider,
 @Nullable
 List<PartitionFilterHintExtractor.PartitionFilterHint> hints,
 boolean hivePartitioning,
 int maxDiscoveredFiles,
 int maxGlobExpansion)
                           throws IOException

Throws:

IOException

expandCommaSeparated

public static FileList expandCommaSeparated(String pathList,
 StorageProvider provider)
                                     throws IOException

Throws:

IOException

expandCommaSeparated

public static FileList expandCommaSeparated(String pathList,
 StorageProvider provider,
 @Nullable
 List<PartitionFilterHintExtractor.PartitionFilterHint> hints,
 boolean hivePartitioning)
                                     throws IOException

Throws:

IOException

expandCommaSeparated

public static FileList expandCommaSeparated(String pathList,
 StorageProvider provider,
 @Nullable
 List<PartitionFilterHintExtractor.PartitionFilterHint> hints,
 boolean hivePartitioning,
 int maxDiscoveredFiles,
 int maxGlobExpansion)
                                     throws IOException

Throws:

IOException

rewriteGlobWithHints

public static String rewriteGlobWithHints(String pattern,
 List<PartitionFilterHintExtractor.PartitionFilterHint> hints)

rewriteGlobWithHints

public static String rewriteGlobWithHints(String pattern,
 List<PartitionFilterHintExtractor.PartitionFilterHint> hints,
 @Nullable
 PartitionConfig partitionConfig)

rewriteGlobWithTemplate

public static String rewriteGlobWithTemplate(String pattern,
 Map<String,PartitionFilterHintExtractor.PartitionFilterHint> rewritableHints,
 String template)

applyFileMetadataFilters

public static List<StorageEntry> applyFileMetadataFilters(List<StorageEntry> entries,
 List<PartitionFilterHintExtractor.PartitionFilterHint> hints)