- A+
所属分类：K8S
说明：
可以采集控制台日志，也可以采集log日志文件
说明文档：
搭建：
a.下载yaml文件
|
1 |
wget https://acs-logging.oss-cn-hangzhou.aliyuncs.com/log-pilot.yml |
b.如果是自己搭建的elasticsearch集群，修改
|
1 2 3 4 5 6 7 8 9 10 11 |
env: - name: "FILEBEAT_OUTPUT" value: "elasticsearch" - name: "ELASTICSEARCH_HOST" value: "elasticsearch-api" #修改为自己的es主机地址 - name: "ELASTICSEARCH_PORT" value: "9200" #修改es主机端口 - name: "ELASTICSEARCH_USER" value: "elastic" #如果有es账号密码配置，如果没有删除 - name: "ELASTICSEARCH_PASSWORD" value: "changeme" |
c.应用：
|
1 |
kubectl apply -f log-pilot.yml |
d.查看是否运行：（注意log-pilot是通过DaemonSet来每个node启动采集）
|
1 2 3 4 5 6 7 8 9 10 11 |
[root@iZwz9iqbscb8v5z26k21o3Z ~]# kubectl get pods -n kube-system -o wide|grep log-pilot log-pilot-2pgck 1/1 Running 0 44m 172.16.1.148 cn-shenzhen.i-wz9iqbscb8v5z26k21o5 <none> log-pilot-2qftj 1/1 Running 0 44m 172.16.0.131 cn-shenzhen.i-wz90ky5oya6sv3ftbtjp <none> log-pilot-8ln9x 1/1 Running 0 44m 172.16.3.151 cn-shenzhen.i-wz9iqbscb8v5z26k21o8 <none> log-pilot-cdcsh 1/1 Running 0 44m 172.16.4.144 cn-shenzhen.i-wz9iqbscb8v5z26k21o3 <none> log-pilot-jb7wl 1/1 Running 0 44m 172.16.4.22 cn-shenzhen.i-wz9iqbscb8v5z26k21o6 <none> log-pilot-nks57 1/1 Running 0 44m 172.16.2.21 cn-shenzhen.i-wz9iqbscb8v5z26k21o2 <none> log-pilot-nnpjq 1/1 Running 0 44m 172.16.1.3 cn-shenzhen.i-wz9j27sd0gs85wsdmrsk <none> log-pilot-nq2kp 1/1 Running 0 44m 172.16.0.8 cn-shenzhen.i-wz9ihky27ka1nnykchwp <none> log-pilot-pj7ms 1/1 Running 0 44m 172.16.3.18 cn-shenzhen.i-wz9iqbscb8v5z26k21o7 <none> log-pilot-xwtnl 1/1 Running 0 44m 172.16.2.143 cn-shenzhen.i-wz9iqbscb8v5z26k21o4 <none> |
应用日志使用范例：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
apiVersion: v1 kind: Pod metadata: name: tomcat namespace: default labels: name: tomcat spec: containers: - image: tomcat name: tomcat-test volumeMounts: - mountPath: /usr/local/tomcat/logs name: accesslogs env: - name: aliyun_logs_catalina ##索引名字是 catalina ，catalina开始不能有_ value: "stdout" ##采集标准输出日志 - name: aliyun_logs_access ##索引名字是 access value: "/usr/local/tomcat/logs/catalina.*.log" ## 采集容器内日志文件 volumes: - name: accesslogs emptyDir: {} |
多行日志：
默认都是逐行采集日志，如果需要多行采集日志，需要定制化更改镜像来实现，解决的办法网址github
步骤：
a.clone代码
|
1 |
<strong>git clone https://github.com/AliyunContainerService/log-pilot.git</strong> |
b.修改filebeat模板
|
1 |
vim log-pilot/assets/filebeat/filebeat.tpl |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
{{range .configList}} - type: log enabled: true paths: - {{ .HostDir }}/{{ .File }} multiline.pattern: '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]' #新增正则条件 multiline.negate: true #新增 multiline.match: after #新增 multiline.max_lines: 10000 #新增 scan_frequency: 10s fields_under_root: true {{if .Stdout}} docker-json: true {{end}} {{if eq .Format "json"}} json.keys_under_root: true {{end}} fields: {{range $key, $value := .Tags}} {{ $key }}: {{ $value }} {{end}} {{range $key, $value := $.container}} {{ $key }}: {{ $value }} {{end}} tail_files: false close_inactive: 2h close_eof: false close_removed: true clean_removed: true close_renamed: false {{end}} |
c.从新打包镜像,第一次估计有点久
|
1 |
cd log-pilot/ && ./build-image.sh |
d.镜像打tag ，并push到私有仓库
|
1 2 |
docker tag log-pilot registry.cn-shenzhen.aliyuncs.com/hr_plub/log-pilot:v2 docker push registry.cn-shenzhen.aliyuncs.com/hr_plub/log-pilot:v2 |
e.修改之前下载的log-pilot.yml，将原来镜像改为 registry.cn-shenzhen.aliyuncs.com/hr_plub/log-pilot:v2，注意，如果私有仓库需要用户密码，需要加imagespullsecret权限
- 我的微信
- 这是我的微信扫一扫
-
- 我的微信公众号
- 我的微信公众号扫一扫
-
