Prometheus Emitter Setup Failing with org.apache.druid.java.util.common.ISE: Emitter was null, that's bad!

Error Stack:

Error in custom provider, org.apache.druid.java.util.common.ISE: Emitter was null, that's bad!
at org.apache.druid.server.emitter.EmitterModule.configure(EmitterModule.java:83) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.emitter.EmitterModule)
at org.apache.druid.server.emitter.EmitterModule.configure(EmitterModule.java:83) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.emitter.EmitterModule)
while locating org.apache.druid.java.util.emitter.core.Emitter
for the 2nd parameter of org.apache.druid.server.emitter.EmitterModule.getServiceEmitter(EmitterModule.java:105)
at org.apache.druid.server.emitter.EmitterModule.getServiceEmitter(EmitterModule.java:105) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.emitter.EmitterModule)
while locating org.apache.druid.java.util.emitter.service.ServiceEmitter
for the 5th parameter of org.apache.druid.server.audit.SQLAuditManager.<init>(SQLAuditManager.java:76)
at org.apache.druid.server.audit.SQLAuditManager.class(SQLAuditManager.java:55)
while locating org.apache.druid.server.audit.SQLAuditManager
at org.apache.druid.guice.PolyBind.createChoice(PolyBind.java:73) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.metadata.storage.derby.DerbyMetadataStorageDruidModule)
while locating org.apache.druid.audit.AuditManager
for the 3rd parameter of org.apache.druid.common.config.JacksonConfigManager.<init>(JacksonConfigManager.java:52)
at org.apache.druid.guice.JacksonConfigManagerModule.configure(JacksonConfigManagerModule.java:41) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.guice.JacksonConfigManagerModule)
while locating org.apache.druid.common.config.JacksonConfigManager
for the 1st parameter of org.apache.druid.guice.JacksonConfigProvider.configure(JacksonConfigProvider.java:65)
at org.apache.druid.guice.JacksonConfigProvider.bind(JacksonConfigProvider.java:38) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.cli.CliOverlord$1 -> org.apache.druid.cli.CliOverlord$1$2)
Caused by: org.apache.druid.java.util.common.ISE: Emitter was null, that's bad!
at org.apache.druid.server.emitter.EmitterModule$EmitterProvider.get(EmitterModule.java:171)
at org.apache.druid.server.emitter.EmitterModule$EmitterProvider.get(EmitterModule.java:117)

Above is one of the errors that you see when setting up Prometheus but you will find the root cause of this when searching any druid service log with 'ClassNotFoundException'. Below is one such scenario where there is a missing class.

Caused by: java.lang.ClassNotFoundException: io.prometheus.client.Collector

Solution:

These exceptions missing due to dependent jars in the Prometheus-emitter extension. Make sure that all the below dependent jars are part of the tar file.

You can run the pulldeps command to download the extension correctly. Then tar and place it on the Imply Manager node.

java -classpath "druid_home/dist/druid/lib/*" -Ddruid.extensions.directory="druid_home/dist/druid/extensions" org.apache.druid.cli.Main tools pull-deps -c org.apache.druid.extensions.contrib:prometheus-emitter:29.0.0