OBJECTIVE:
The purpose of this article is to outline the steps in enabling druid-influxdb-emitter
extension.
ENVIRONMENT:
Imply-2021.01-01 LTS
ADDITIONAL INFORMATION:
PROCEDURE:
Download and unpack InfluxDB
cd /opt
curl -X GET https://dl.influxdata.com/influxdb/releases/influxdb2-2.0.4-darwin-amd64.tar.gz -o influxdb2-2.0.4-darwin-amd64.tar.gz
tar -xvzf influxdb2-2.0.4-darwin-amd64.tar.gz
cd influxdb2-2.0.4-darwin-amd64
To start InfluxDB, run
/opt/influxdb2-2.0.4-darwin-amd64/influxd
which will output messages similar to below showing your influxDB is started on default port 8086
2021-03-31T19:13:28.563181Z info Welcome to InfluxDB {"log_id": "0TER7PxW000", "version": "2.0.4", "commit": "4e7a59bb9a", "build_date": "2021-02-08T17:47:02Z"}
2021-03-31T19:13:28.659868Z info Resources opened {"log_id": "0TER7PxW000", "service": "bolt", "path": "/Users/sivamannem/.influxdbv2/influxd.bolt"}
2021-03-31T19:13:28.715154Z info Bringing up metadata migrations {"log_id": "0TER7PxW000", "service": "migrations", "migration_count": 14}
2021-03-31T19:13:32.809238Z info Using data dir {"log_id": "0TER7PxW000", "service": "storage-engine", "service": "store", "path": "/Users/sivamannem/.influxdbv2/engine/data"}
2021-03-31T19:13:32.809583Z info Compaction settings {"log_id": "0TER7PxW000", "service": "storage-engine", "service": "store", "max_concurrent_compactions": 6, "throughput_bytes_per_second": 50331648, "throughput_bytes_per_second_burst": 50331648}
2021-03-31T19:13:32.809628Z info Open store (start) {"log_id": "0TER7PxW000", "service": "storage-engine", "service": "store", "op_name": "tsdb_open", "op_event": "start"}
2021-03-31T19:13:32.809951Z info Open store (end) {"log_id": "0TER7PxW000", "service": "storage-engine", "service": "store", "op_name": "tsdb_open", "op_event": "end", "op_elapsed": "0.326ms"}
2021-03-31T19:13:32.810014Z info Starting retention policy enforcement service {"log_id": "0TER7PxW000", "service": "retention", "check_interval": "30m"}
2021-03-31T19:13:32.810036Z info Starting precreation service {"log_id": "0TER7PxW000", "service": "shard-precreation", "check_interval": "10m", "advance_period": "30m"}
2021-03-31T19:13:32.810138Z info Starting query controller {"log_id": "0TER7PxW000", "service": "storage-reads", "concurrency_quota": 10, "initial_memory_bytes_quota_per_query": 9223372036854775807, "memory_bytes_quota_per_query": 9223372036854775807, "max_memory_bytes": 0, "queue_size": 10}
2021-03-31T19:13:32.811271Z info Configuring InfluxQL statement executor (zeros indicate unlimited). {"log_id": "0TER7PxW000", "max_select_point": 0, "max_select_series": 0, "max_select_buckets": 0}
2021-03-31T19:13:33.239943Z info Starting {"log_id": "0TER7PxW000", "service": "telemetry", "interval": "8h"}
2021-03-31T19:13:33.239934Z info Listening {"log_id": "0TER7PxW000", "transport": "http", "addr": ":8086", "port": 8086}
2021-03-31T19:17:13.445635Z info index opened with 8 partitions {"log_id": "0TER7PxW000", "service": "storage-engine", "index": "tsi"}
2021-03-31T19:17:13.446241Z info Reindexing TSM data {"log_id": "0TER7PxW000", "service": "storage-engine", "engine": "tsm1", "db_shard_id": 1}
2021-03-31T19:17:13.446260Z info Reindexing WAL data {"log_id": "0TER7PxW000", "service": "storage-engine", "engine": "tsm1", "db_shard_id": 1}
Now you can access influx at http://localhost:8086
Create a bucket (eg, druid-influx), user (druid) and password (druid123) in influxDB
Let's get the druid-influxdb-emitter extension for druid and enable it.
If your imply home directory is /opt/imply-2021.01-1, then run the following commands
cd /opt/imply-2021.01-1
java -cp "dist/druid/lib/*" -Ddruid.extensions.directory="dist/druid/extensions" -Ddruid.extensions.hadoopDependenciesDir="dist/druid/hadoop-dependencies" org.apache.druid.cli.Main tools pull-deps --no-default-hadoop -c "org.apache.druid.extensions.contrib:druid-influxdb-emitter:0.20.0"
The above command creates druid-influxdb-emitter in /opt/imply-2021.01-1/dist/druid/extensions.
Now open common.runtime.properties in /opt/imply-2021.01-1/[conf|conf-quickstart]/druid/_common and add or modify the following entries
druid.extensions.loadList=["druid-histogram", "druid-datasketches", "druid-kafka-indexing-service", "imply-utility-belt", "druid-influxdb-emitter"]
druid.emitter=influx
druid.emitter.influxdb.hostname=localhost
druid.emitter.influxdb.port=8086
druid.emitter.influxdb.databaseName=druid-influx
druid.emitter.influxdb.influxdbUserName=druid
druid.emitter.influxdb.influxdbPassword=druid123
Then restart your cluster - eg, for a quickstart:
/opt/imply-2021.01-1/bin/supervise -c /opt/imply-2021.01-1//conf/supervise/quickstart.conf &
After ingesting a data source and running couple of queries in Query tab, the influxDB dashboard should display a few metrics:
REFERENCES:
https://docs.influxdata.com/influxdb/v2.0/get-started/
https://docs.influxdata.com/influxdb/v2.0/monitor-alert/
Comments
0 comments
Please sign in to leave a comment.