java.lang.Object
org.elasticsearch.xpack.core.ml.job.process.autodetect.state.DataCounts
All Implemented Interfaces:
Writeable, org.elasticsearch.xcontent.ToXContent, org.elasticsearch.xcontent.ToXContentObject

public final class DataCounts extends Object implements org.elasticsearch.xcontent.ToXContentObject, Writeable
Job processed record counts.

The getInput... methods return the actual number of fields/records sent the API including invalid records. The getProcessed... methods are the number sent to the Engine.

The inputRecordCount field is calculated so it should not be set in deserialisation but it should be serialised so the field is visible.

  • Field Details

    • PROCESSED_RECORD_COUNT

      public static final org.elasticsearch.xcontent.ParseField PROCESSED_RECORD_COUNT
    • PROCESSED_FIELD_COUNT

      public static final org.elasticsearch.xcontent.ParseField PROCESSED_FIELD_COUNT
    • INPUT_BYTES

      public static final org.elasticsearch.xcontent.ParseField INPUT_BYTES
    • INPUT_RECORD_COUNT

      public static final org.elasticsearch.xcontent.ParseField INPUT_RECORD_COUNT
    • INPUT_FIELD_COUNT

      public static final org.elasticsearch.xcontent.ParseField INPUT_FIELD_COUNT
    • INVALID_DATE_COUNT

      public static final org.elasticsearch.xcontent.ParseField INVALID_DATE_COUNT
    • MISSING_FIELD_COUNT

      public static final org.elasticsearch.xcontent.ParseField MISSING_FIELD_COUNT
    • OUT_OF_ORDER_TIME_COUNT

      public static final org.elasticsearch.xcontent.ParseField OUT_OF_ORDER_TIME_COUNT
    • EMPTY_BUCKET_COUNT

      public static final org.elasticsearch.xcontent.ParseField EMPTY_BUCKET_COUNT
    • SPARSE_BUCKET_COUNT

      public static final org.elasticsearch.xcontent.ParseField SPARSE_BUCKET_COUNT
    • BUCKET_COUNT

      public static final org.elasticsearch.xcontent.ParseField BUCKET_COUNT
    • EARLIEST_RECORD_TIME

      public static final org.elasticsearch.xcontent.ParseField EARLIEST_RECORD_TIME
    • LATEST_RECORD_TIME

      public static final org.elasticsearch.xcontent.ParseField LATEST_RECORD_TIME
    • LAST_DATA_TIME

      public static final org.elasticsearch.xcontent.ParseField LAST_DATA_TIME
    • LATEST_EMPTY_BUCKET_TIME

      public static final org.elasticsearch.xcontent.ParseField LATEST_EMPTY_BUCKET_TIME
    • LATEST_SPARSE_BUCKET_TIME

      public static final org.elasticsearch.xcontent.ParseField LATEST_SPARSE_BUCKET_TIME
    • LOG_TIME

      public static final org.elasticsearch.xcontent.ParseField LOG_TIME
    • TYPE

      public static final org.elasticsearch.xcontent.ParseField TYPE
    • PARSER

      public static final org.elasticsearch.xcontent.ConstructingObjectParser<DataCounts,Void> PARSER
  • Constructor Details

    • DataCounts

      public DataCounts(String jobId, long processedRecordCount, long processedFieldCount, long inputBytes, long inputFieldCount, long invalidDateCount, long missingFieldCount, long outOfOrderTimeStampCount, long emptyBucketCount, long sparseBucketCount, long bucketCount, Date earliestRecordTimeStamp, Date latestRecordTimeStamp, Date lastDataTimeStamp, Date latestEmptyBucketTimeStamp, Date latestSparseBucketTimeStamp, Instant logTime)
    • DataCounts

      public DataCounts(String jobId)
    • DataCounts

      public DataCounts(DataCounts lhs)
    • DataCounts

      public DataCounts(StreamInput in) throws IOException
      Throws:
      IOException
  • Method Details

    • documentId

      public static String documentId(String jobId)
    • v54DocumentId

      public static String v54DocumentId(String jobId)
    • getJobId

      public String getJobId()
    • getProcessedRecordCount

      public long getProcessedRecordCount()
      Number of records processed by this job. This value is the number of records sent passed on to the engine i.e. getInputRecordCount() minus records with bad dates or out of order
      Returns:
      Number of records processed by this job long
    • incrementProcessedRecordCount

      public void incrementProcessedRecordCount(long additional)
    • getProcessedFieldCount

      public long getProcessedFieldCount()
      Number of data points (processed record count * the number of analysed fields) processed by this job. This count does not include the time field.
      Returns:
      Number of data points processed by this job long
    • calcProcessedFieldCount

      public void calcProcessedFieldCount(long analysisFieldsPerRecord)
    • getInputRecordCount

      public long getInputRecordCount()
      Total number of input records read. This = processed record count + date parse error records count + out of order record count.

      Records with missing fields are counted as they are still written.

      Returns:
      Total number of input records read long
    • getInputBytes

      public long getInputBytes()
      The total number of bytes sent to this job. This value includes the bytes from any records that have been discarded for any reason e.g. because the date cannot be read
      Returns:
      Volume in bytes
    • incrementInputBytes

      public void incrementInputBytes(long additional)
    • getInputFieldCount

      public long getInputFieldCount()
      The total number of fields sent to the job including fields that aren't analysed.
      Returns:
      The total number of fields sent to the job
    • incrementInputFieldCount

      public void incrementInputFieldCount(long additional)
    • getInvalidDateCount

      public long getInvalidDateCount()
      The number of records with an invalid date field that could not be parsed or converted to epoch time.
      Returns:
      The number of records with an invalid date field
    • incrementInvalidDateCount

      public void incrementInvalidDateCount(long additional)
    • getMissingFieldCount

      public long getMissingFieldCount()
      The number of missing fields that had been configured for analysis.
      Returns:
      The number of missing fields
    • incrementMissingFieldCount

      public void incrementMissingFieldCount(long additional)
    • getOutOfOrderTimeStampCount

      public long getOutOfOrderTimeStampCount()
      The number of records with a timestamp that is before the time of the latest record. Records should be in ascending chronological order
      Returns:
      The number of records with a timestamp that is before the time of the latest record
    • incrementOutOfOrderTimeStampCount

      public void incrementOutOfOrderTimeStampCount(long additional)
    • getEmptyBucketCount

      public long getEmptyBucketCount()
      The number of buckets with no records in it. Used to measure general data fitness and/or configuration problems (bucket span).
      Returns:
      Number of empty buckets processed by this job long
    • incrementEmptyBucketCount

      public void incrementEmptyBucketCount(long additional)
    • getSparseBucketCount

      public long getSparseBucketCount()
      The number of buckets with few records compared to the overall counts. Used to measure general data fitness and/or configuration problems (bucket span).
      Returns:
      Number of sparse buckets processed by this job long
    • incrementSparseBucketCount

      public void incrementSparseBucketCount(long additional)
    • getBucketCount

      public long getBucketCount()
      The number of buckets overall.
      Returns:
      Number of buckets processed by this job long
    • incrementBucketCount

      public void incrementBucketCount(long additional)
    • getEarliestRecordTimeStamp

      public Date getEarliestRecordTimeStamp()
      The time of the first record seen.
      Returns:
      The first record time
    • setEarliestRecordTimeStamp

      public void setEarliestRecordTimeStamp(Date timeStamp)
      If earliestRecordTimeStamp has not been set (i.e. is null) then set it to timeStamp
      Parameters:
      timeStamp - Candidate time
      Throws:
      IllegalStateException - if earliestRecordTimeStamp is already set
    • getLatestRecordTimeStamp

      public Date getLatestRecordTimeStamp()
      The time of the latest record seen.
      Returns:
      Latest record time
    • setLatestRecordTimeStamp

      public void setLatestRecordTimeStamp(Date latestRecordTimeStamp)
    • getLastDataTimeStamp

      public Date getLastDataTimeStamp()
      The wall clock time the latest record was seen.
      Returns:
      Wall clock time of the lastest record
    • setLastDataTimeStamp

      public void setLastDataTimeStamp(Date lastDataTimeStamp)
    • getLatestEmptyBucketTimeStamp

      public Date getLatestEmptyBucketTimeStamp()
      The time of the latest empty bucket seen.
      Returns:
      Latest empty bucket time
    • setLatestEmptyBucketTimeStamp

      public void setLatestEmptyBucketTimeStamp(Date latestEmptyBucketTimeStamp)
    • updateLatestEmptyBucketTimeStamp

      public void updateLatestEmptyBucketTimeStamp(Date timestamp)
    • getLatestSparseBucketTimeStamp

      public Date getLatestSparseBucketTimeStamp()
      The time of the latest sparse bucket seen.
      Returns:
      Latest sparse bucket time
    • setLatestSparseBucketTimeStamp

      public void setLatestSparseBucketTimeStamp(Date latestSparseBucketTimeStamp)
    • updateLatestSparseBucketTimeStamp

      public void updateLatestSparseBucketTimeStamp(Date timestamp)
    • setLogTime

      public void setLogTime(Instant logTime)
    • getLogTime

      public Instant getLogTime()
    • writeTo

      public void writeTo(StreamOutput out) throws IOException
      Specified by:
      writeTo in interface Writeable
      Throws:
      IOException
    • toXContent

      public org.elasticsearch.xcontent.XContentBuilder toXContent(org.elasticsearch.xcontent.XContentBuilder builder, org.elasticsearch.xcontent.ToXContent.Params params) throws IOException
      Specified by:
      toXContent in interface org.elasticsearch.xcontent.ToXContent
      Throws:
      IOException
    • doXContentBody

      public org.elasticsearch.xcontent.XContentBuilder doXContentBody(org.elasticsearch.xcontent.XContentBuilder builder, org.elasticsearch.xcontent.ToXContent.Params params) throws IOException
      Throws:
      IOException
    • equals

      public boolean equals(Object other)
      Equality test
      Overrides:
      equals in class Object
    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object