MSQ Ingestion failing with java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FileSystem

Issue:

Exception Stack:

WARN [task-runner-0-priority-0] org.apache.druid.msq.exec.ControllerImpl - Worker: Work failed; stage 0; task query-671ff9bb-4eb0-4c99-86d0-8a84bc1fec9d-worker0_0; host 10.217.16.3:8104: UnknownError: java.lang.RuntimeException: java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FileSystem
java.lang.RuntimeException: java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FileSystem
at org.apache.druid.java.util.common.Either.valueOrThrow(Either.java:95)
at org.apache.druid.frame.processor.FrameProcessorExecutor$1ExecutorRunnable.runProcessorNow(FrameProcessorExecutor.java:259)
at org.apache.druid.frame.processor.FrameProcessorExecutor$1ExecutorRunnable.run(FrameProcessorExecutor.java:138)
at org.apache.druid.msq.exec.WorkerImpl$1$2.run(WorkerImpl.java:840)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.apache.druid.query.PrioritizedListenableFutureTask.run(PrioritizedExecutorService.java:259)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FileSystem
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
at org.apache.druid.msq.querykit.BaseLeafFrameProcessor.runIncrementally(BaseLeafFrameProcessor.java:89)
at org.apache.druid.frame.processor.FrameProcessors$1FrameProcessorWithBaggage.runIncrementally(FrameProcessors.java:75)
at org.apache.druid.frame.processor.FrameProcessorExecutor$1ExecutorRunnable.runProcessorNow(FrameProcessorExecutor.java:230)
... 58 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FileSystem
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
... 59 more

This issue occurs when the jar gcs-connector-hadoop3-2.2.19-shaded.jar is present in the druid classpath.

Solution:

Log into Imply Manager > Setup > Advanced Config > Custom Files > Click on the gcs-connector file and change the Location type to Hadoop Dependency.

This should fix the MSQ ingestion issue.