InfluxDB是一款专门处理高写入和查询负载的时间序列数据库,用于存储大规模的时间序列数据并进行实时分析,包括来自DevOps监控、应用指标和IoT传感器上的数据。
主要特点:
专为时间序列数据量身订造高性能数据存储。TSM引擎提供数据高速读写和压缩等功能。
简单高效的HTTP API写入和查询接口。
针对时序数据,量身订造类似SQL的查询语言,轻松查询聚合数据。
允许对tag建索引,实现快速有效的查询。
数据保留策略(Retention policies)能够有效地使旧数据自动失效。
为什么使用InfluxDB?
- 高性能:InfluxDB是专门为时间序列数据编写的高性能数据存储。它允许高吞吐量的摄取,压缩和实时查询。InfluxDB完全用Go编写,并且可以编译成一个二进制文件,没有外部依赖关系。它提供了命令行界面,内置HTTP API,客户端库(例如Go,Java和JavaScript)以及用于常见数据格式(例如Telegraf,Graphite,Collected和OpenTSDB)的插件的写入和查询功能。
- 类SQL查询:InfluxDB与InfluxQL配合使用,InfluxQL是一种类似于SQL的查询语言,用于与数据进行交互。它经过精心设计,可以使来自其他SQL或类似SQL环境的用户快速熟悉,同时还提供特定于存储和分析时间序列数据的功能。InfluxQL支持正则表达式,算术表达式以及特定于时间序列的函数,以加快数据处理速度。
- 下采样和数据保留:InfluxDB每秒可以处理数百万个数据点。长时间使用大量数据可能会导致存储问题。InfluxDB自动压缩数据以最小化您的存储空间。另外,您可以轻松地对数据进行下采样;将高精度的原始数据保留有限的时间,并将精度较低的汇总数据存储更长的时间或直到时间结束。InfluxDB具有两个有助于自动执行下采样和数据过期过程的功能-连续查询(Continuous Queries)和保留策略(Retention Policies)。
在网上找了很多InfluxDB的安装教程,也有一些在Kubernetes上搭建的教程,但是InfluxDB Enterprise并没有人进行一个完整的教程,所以本文将讲述如何在Kubernetes上搭建InfluxDB Enterprise。
在Influxdata的官网上找到了如下描述:
在Kubernetes中部署TICK(telegraf、InfluxDB、Chronograf、Kapacitor)技术栈:
如上描述,找到了三种在Kubernetes中部署的方式,我选择第一种,使用Helm部署
在官网的网页中我还看到了一些在Kubernetes中运行InfluxDB的缺点,如下:
尽管Kubernetes迅速应用程序的稳定部署平台,但它仍然带来了极大的复杂性,并且几乎没有为数据库工作负载带来好处。因此,我们不建议在生产环境中在Kubernetes上运行InfluxDB或InfluxDB Enterprise。尽管许多用户设法成功地在Kubernetes中运行数据库,但是许多InfluxDB用户也遇到了问题,包括大量的停机时间,甚至由于Kubernetes重新调度Pod或安装卷的问题而导致数据丢失。
虽然InfluxDB的官网表述说Kubernetes对数据库的支持并不友好,但是我就要在上面部署,谁劝我都没有用。好了,下面进入正题。
添加helm提供的源
helm repo add influxdata https://helm.influxdata.com/
使用helm repo list
看一下,已经添加进来了
NAME URL
stable https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
influxdata https://helm.influxdata.com/
使用helm search repo | grep influxdata
看看这个源里都有啥
influxdata/chronograf 1.1.8 1.8.0 Open-source web application written in Go and R...
influxdata/influxdb 4.4.3 1.7.10 Scalable datastore for metrics, events, and rea...
influxdata/influxdb2 1.0.1 2.0.0-beta A Helm chart for InfluxDB v2
influxdata/kapacitor 1.2.6 1.5.4 InfluxDB's native data processing engine. It ca...
influxdata/telegraf 1.7.7 1.13 Telegraf is an agent written in Go for collecti...
influxdata/telegraf-ds 1.0.3 1.13 Telegraf is an agent written in Go for collecti...
使用helm pull influxdata/influxdb
先把influxdata/influxdb
下载下来,会发现有一个tgz包,解压它
tar -zxvf influxdb-4.4.3.tgz
使用 vim influxdb/values.yaml
更改配置
因为我申请的Lisences只有14天,支持2 x 16 cores,所以我更改的配置如下:
image.tag: 1.7.10-data
persistence.size:50G
enterprise.enabled: true
enterprise.meta.image.tag: 1.7.10-meta
enterprise.clusterSize: 2
enterprise.meta.clusterSize: 3
安装Chart
运行helm install influxdb ./influxdb
NAME: influxdb
LAST DEPLOYED: Thu Apr 2 14:30:45 2020
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
InfluxDB can be accessed via port 8086 on the following DNS name from within your cluster:
- http://influxdb.default:8086
You can easily connect to the remote instance with your local influx cli. To forward the API port to localhost:8086 run the following:
- kubectl port-forward --namespace default $(kubectl get pods --namespace default -l app=influxdb -o jsonpath='{ .items[0].metadata.name }') 8086:8086
You can also connect to the influx cli from inside the container. To open a shell session in the InfluxDB pod run the following:
- kubectl exec -i -t --namespace default $(kubectl get pods --namespace default -l app=influxdb -o jsonpath='{.items[0].metadata.name}') /bin/sh
To tail the logs for the InfluxDB pod run the following:
- kubectl logs -f --namespace default $(kubectl get pods --namespace default -l app=influxdb -o jsonpath='{ .items[0].metadata.name }')
查看pod的状态,发现一直是Pending的状态
kubectl get pods
NAME READY STATUS RESTARTS AGE
influxdb-0 0/1 Pending 0 3m26s
influxdb-meta-0 0/1 Pending 0 3m26s
看一下pod的描述,发现有两条警告:pod has unbound immediate PersistentVolumeClaims。应该是没有绑定到pvc的原因。
kubectl describe pod influxdb-0 pods
Name: influxdb-0
Namespace: default
Priority: 0
Node: <none>
Labels: app.kubernetes.io/instance=influxdb
app.kubernetes.io/name=influxdb
controller-revision-hash=influxdb-68fd856997
statefulset.kubernetes.io/pod-name=influxdb-0
Annotations: <none>
Status: Pending
IP:
IPs: <none>
Controlled By: StatefulSet/influxdb
Containers:
influxdb:
Image: influxdb:1.7.10-data
Ports: 8086/TCP, 8088/TCP, 8091/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP
Liveness: http-get http://:api/ping delay=30s timeout=5s period=10s #success=1 #failure=3
Readiness: http-get http://:api/ping delay=5s timeout=1s period=10s #success=1 #failure=3
Environment: <none>
Mounts:
/etc/influxdb from config (rw)
/var/lib/influxdb from influxdb-data (rw)
/var/run/secrets/kubernetes.io/serviceaccount from influxdb-token-lfvjn (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
influxdb-data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: influxdb-data-influxdb-0
ReadOnly: false
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: influxdb
Optional: false
influxdb-token-lfvjn:
Type: Secret (a volume populated by a Secret)
SecretName: influxdb-token-lfvjn
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling <unknown> default-scheduler running "VolumeBinding" filter plugin for pod "influxdb-0": pod has unbound immediate PersistentVolumeClaims
Warning FailedScheduling <unknown> default-scheduler running "VolumeBinding" filter plugin for pod "influxdb-0": pod has unbound immediate PersistentVolumeClaims
所以我们看一下pvc是什么状况,也是Penging状态
kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
influxdb-data-influxdb-0 Pending 9m40s
influxdb-meta-influxdb-meta-0 Pending 9m40s
kubectl describe pvc influxdb-data-influxdb-0
Name: influxdb-data-influxdb-0
Namespace: default
StorageClass:
Status: Pending
Volume:
Labels: app.kubernetes.io/instance=influxdb
app.kubernetes.io/name=influxdb
Annotations: <none>
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Filesystem
Mounted By: influxdb-0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal FailedBinding 2m25s (x162 over 42m) persistentvolume-controller no persistent volumes available for this claim and no storage class is set
所以应该是pv的问题,接下来看一下pv是什么情况,空空如也。
kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
所以我们需要创建对应的pv,说来就来
单机版需要配置的选项(别问我为什么没有翻译,”懒“):
Parameter | Description | Default |
---|---|---|
image.repository | Image repository url | influxdb |
image.tag | Image tag | 1.7.10-alpine |
image.pullPolicy | Image pull policy | IfNotPresent |
image.pullSecrets | It will store the repository's credentials to pull image | nil |
serviceAccount.create | It will create service account | true |
serviceAccount.name | Service account name | "" |
serviceAccount.annotations | Service account annotations | {} |
livenessProbe | Health check for pod | {} |
readinessProbe | Health check for pod | {} |
startupProbe | Health check for pod | {} |
service.type | Kubernetes service type | ClusterIP |
persistence.enabled | Boolean to enable and disable persistance | true |
persistence.existingClaim | An existing PersistentVolumeClaim, ignored if enterprise.enabled=true | nil |
persistence.storageClass | If set to "-", storageClassName: "", which disables dynamic provisioning. If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner. (gp2 on AWS, standard on GKE, AWS & OpenStack | |
persistence.annotations | Annotations for volumeClaimTemplates | nil |
persistence.accessMode | Access mode for the volume | ReadWriteOnce |
persistence.size | Storage size | 8Gi |
podAnnotations | Annotations for pod | {} |
ingress.enabled | Boolean flag to enable or disable ingress | false |
ingress.tls | Boolean to enable or disable tls for ingress. If enabled provide a secret in ingress.secretName containing TLS private key and certificate. |
false |
ingress.secretName | Kubernetes secret containing TLS private key and certificate. It is only required if ingress.tls is enabled. |
nil |
ingress.hostname | Hostname for the ingress | influxdb.foobar.com |
annotations | ingress annotations | nil |
schedulerName | Use an alternate scheduler, e.g. "stork". | nil |
nodeSelector | Node labels for pod assignment | {} |
affinity | Affinity for pod assignment | { |
tolerations | Tolerations for pod assignment | [] |
env | environment variables for influxdb container | {} |
config.reporting_disabled | Details | false |
config.rpc | RPC address for backup and storage | {} |
config.meta | Details | {} |
config.data | Details | {} |
config.coordinator | Details | {} |
config.retention | Details | {} |
config.shard_precreation | Details | {} |
config.monitor | Details | {} |
config.http | Details | {} |
config.logging | Details | {} |
config.subscriber | Details | {} |
config.graphite | Details | {} |
config.collectd | Details | {} |
config.opentsdb | Details | {} |
config.udp | Details | {} |
config.continous_queries | Details | {} |
config.tls | Details | {} |
initScripts.enabled | Boolean flag to enable and disable initscripts. If the container finds any files with the extensions .sh or .iql inside of the /docker-entrypoint-initdb.d folder, it will execute them. The order they are executed in is determined by the shell. This is usually alphabetical order. | false |
initScripts.scripts | Init scripts | {} |
backup.enabled | Enable backups, if true must configure one of the storage providers |
false |
backup.gcs | Google Cloud Storage config | nil |
backup.azure | Azure Blob Storage config | nil |
backup.schedule | Schedule to run jobs in cron format | 0 0 * * * |
backup.annotations | Annotations for backup cronjob | {} |
backup.podAnnotations | Annotations for backup cronjob pods | {} |
另外,如果搭建集群版需要配置的选项如下:
Key | Description | Recommended value |
---|---|---|
livenessProbe.initalDelaySeconds |
Used to allow enough time to join meta nodes to a cluster | 3600 |
image.tag |
Set to a data image. See https://hub.docker.com/_/influxdb for details |
data |
service.ClusterIP |
Use a headless service for StatefulSets | "None" |
env.name[_HOSTNAME] |
Used to provide a unique name.service for InfluxDB. See values.yaml for an example |
valueFrom.fieldRef.fieldPath: metadata.name |
enterprise.enabled |
Create StatefulSets for use with influx-data and influx-meta images |
true |
enterprise.licensekey |
License for InfluxDB Enterprise | |
enterprise.clusterSize |
Replicas for influx StatefulSet |
Dependent on license |
enterprise.meta.image.tag |
Set to an meta image. See https://hub.docker.com/_/influxdb for details |
meta |
enterprise.meta.clusterSize |
Replicas for influxdb-meta StatefulSet. |
3 |
enterprise.meta.resources |
Resources requests and limits for meta influxdb-meta pods |
See values.yaml
|