When a kill task is submitted, it completes these three steps:
1. Gets a list of all segments marked as unused (used=0) in the metadata store.
2. Loops through each segment, deletes from druid_segments where id = x. This process takes a couple hundred milliseconds per segment, so deleting 10k segments can take ~30 min.
3. Once step 2 is complete, removes each segment from deep storage.
If the kill task gets interrupted, there is a chance that segments get deleted from the metadata store (step 2) but not deep storage. If this happens, segments must be removed from deep storage manually.
Comments
0 comments
Please sign in to leave a comment.