How to update heap memory, direct memory for a specific Kafka/kinesis supervisor

OBJECTIVE :

There are a few scenarios where real-time ingestion tasks fail with OutOfMemory on heap OR java.lang.OutOfMemoryError: Direct buffer memory and users do not want to update the druid core configurations for heap/direct memory, as it's not required for other supervisors. This article will explain the steps required to update javaOpts values for a specific supervisor.

SOLUTION :
Add a task context in a supervisor spec and overridedruid.indexer.runner.javaOpts 

   "context":
    {
    "druid.indexer.runner.javaOpts":"-server -d64 -Xms1g -Xmx10g -XX:MaxDirectMemorySize=12g"
    }

Example: In this example spec I am updating `xmx` to `10 gigs` and MaxDirectMemorySize to 12 gigs in a Kafka supervisor by adding a task context.

{
  "type": "kafka",
  "dataSchema": {
    "dataSource": "wikipedia-kafka-t2",
    "parser": {
      "type": "string",
      "parseSpec": {
        "format": "json",
        "timestampSpec": {
          "column": "timestamp",
          "format": "iso"
        },
        "dimensionsSpec": {
          "dimensions": [
            "isRobot",
            "channel",
            "flags",
            "isUnpatrolled",
            "page",
            "diffUrl",
            "comment",
            "isNew",
            "isMinor",
            "user",
            "namespace",
            { "name" : "commentLength", "type" : "long" },
            { "name" : "deltaBucket", "type" : "long" },
            "cityName",
            "countryIsoCode",
            "countryName",
            "isAnonymous",
            "metroCode",
            "regionIsoCode",
            "regionName",
            { "name": "added", "type": "long" },
            { "name": "deleted", "type": "long" },
            { "name": "delta", "type": "long" }
          ]
        }
      }
    },
    "metricsSpec" : [],
    "granularitySpec": {
      "type": "uniform",
      "segmentGranularity": "HOUR",
      "queryGranularity": "NONE",
      "rollup": true
    }
  },
  "ioConfig": {
    "topic": "wikipedia",
    "consumerProperties": {
      "bootstrap.servers": "<KAFKA-BROKER-HOST>:9092"
    }
  },
"context":
{
"druid.indexer.runner.javaOpts":"-server -d64 -Xms1g -Xmx10g -XX:MaxDirectMemorySize=12g"
}
}

VALIDATION :
After submitting the updated supervisor spec, changes can be verified in the Kafka-indexing task logs:

[i] Check if the configuration is picked up by the task by looking any entries like the one below:

  "context" : {
    "checkpoints" : "{\"0\":{\"0\":0}}",
    "druid.indexer.runner.javaOpts" : "-server -d64 -Xms1g -Xmx10g -XX:MaxDirectMemorySize=12g",
    "IS_INCREMENTAL_HANDOFF_SUPPORTED" : true,
    "forceTimeChunkLock" : true
  },

[ii] Check for the peon launch message - It should be launched with the updated values 

2020-03-27T13:49:23,531 INFO [main] org.apache.druid.cli.CliPeon - Starting up with processors[12], memory[1,053,294,592], maxMemory[9,544,663,040], directMemory[12,884,901,888].