SYMPTOM:
Ingestion from S3 fails with a message similar to below:
<Code>NoSuchKey</>
<Message>The specified key does not exist</Message>
Ingestion spec has ioConfig block similar to the following:
"ioConfig": {
"type": "index",
"firehose": {
"fetchTimeout": 300000,
"type": "static-s3",
"prefixes": ["s3://not-real-bucket/filepath/dt=2018-10-31"]
},
"appendToExisting": "false"
}
ROOT CAUSE:
The cause of this issue is a bug in the jetS3t library when using AWS S3 Signature V4.
WORKAROUND:
The workaround is to revert to using Signature V2 for AWS S3. This can be enabled in the Experimental Features section of the cluster settings as illustrated below.
Resolution:
Beginning with Imply 2.8.0, jetS3t is no longer used. Instead the native AWS S3 API is leveraged, thus resolving this issue.
Comments
0 comments
Please sign in to leave a comment.