When a kill task is submitted, it completes these three steps:
1. Gets a list of all segments marked as unused (used=0) in the metadata store.
2. Loops through each segment, deletes from druid_segments where id = x. This process takes a couple hundred milliseconds per segment, so deleting 10k segments can take ~30 min.
3. Once step 2 is complete, removes each segment from deep storage.
If the kill task gets interrupted, there is a chance that segments get deleted from the metadata store (step 2) but not deep storage. If this happens, segments must be removed from deep storage manually.
Please sign in to leave a comment.