Articles in this section

Single-dim Compaction task fails with: Unrecognized token 'Cannot'

ISSUE:

In Imply 3.x, running a compaction ask with single-dim in partionsSpec fails with the following error:

2020-06-29T05:14:40,409 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner - Exception while running task[AbstractTask{id='partial_dimension_distribution_TestDs_eghdjnoh_2020-06-29T05:13:07.913Z', groupId='compact_TestDs_onnbilcc_2020-06-29T05:09:49.094Z', taskResource=TaskResource{availabilityGroup='partial_dimension_distribution_TestDs_eghdjnoh_2020-06-29T05:13:07.913Z', requiredCapacity=1}, dataSource='TestDs', context={appenderatorTrackingTaskId=compact_TestDs_onnbilcc_2020-06-29T05:09:49.094Z, forceTimeChunkLock=true, priority=25}}]
java.lang.RuntimeException: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'Cannot': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
at [Source: (String)"Cannot find any task with id: [compact_TestDs_onnbilcc_2020-06-29T05:09:49.094Z_0]"; line: 1, column: 7]
at org.apache.druid.client.indexing.HttpIndexingServiceClient.getTaskStatus(HttpIndexingServiceClient.java:268) ~[druid-server-0.18.0-iap8.jar:0.18.0-iap8]
...

Caused by: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'Cannot':
was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
at [Source: (String)"Cannot find any task with id: [compact_TestDs_onnbilcc_2020-06-29T05:09:49.094Z_0]"; line: 1, column: 7]

ENVIRONMENT:

Imply Version 3.x and above.  (This is fixed in Imply 4.0 and higher.)

CAUSE:

In compaction and native ingestion tasks, in 3.x, single_dim is supported only for parallel tasks, i.e.,maxNumConcurrentSubTasks should be set to a value higher than 1.

In this version, default value for maxNumConcurrentSubTasks is 1 and it was not defined in the compaction task spec. Hence the task failed.

RESOLUTION:

Add maxNumConcurrentSubTasks to the tuningConfig and set it to a value > 1. For example,

{
    "type": "compact",
    "dataSource": "TestDs",
    "ioConfig": {
        "type": "compact",
        "inputSpec": {
            "type": "interval",
            "interval": "2020-04-27/2020-04-28"
        }
    },
    "tuningConfig": {
        "type": "index_parallel",
        "maxNumConcurrentSubTasks": 5,
        "forceGuaranteedRollup": true,
        "partitionsSpec": {
            "type": "single_dim",
            "targetRowsPerSegment": 5000000,
            "partitionDimension": "test"
        }
    }
}

 

REFERENCES:

Apache Druid Documentation: Single-dimension range partitioning

Was this article helpful?
0 out of 0 found this helpful