"Too many open files" messages showing up in your logs? Tasks failing?
First step - check your ulimit
If the value is < 10,000 you should probably increase it.
The default open file settings for linux or often too low for an Apache Druid system, which can be interacting with thousands of files simultaneously.
Per session solution (will be reverted on reboot):
NOTE: Ensure you have root access
Execute the following
ulimit -n 65536
ulimit -u 65536
Ensure the following values are set:
* soft nproc 65535 * hard nproc 65535 * soft nofile 65535 * hard nofile 65535
Wait! are you on RHEL 7 or using systemd to boot the systems?
Above wouldn't work! To set the values, review the following RedHat Subscriber Article instead: