输出jetty日志到efk，其中jetty日志使用的是logback

xiaoxiao2021-02-28 25

发布一个k8s部署视频：https://edu.csdn.net/course/detail/26967

课程内容：各种k8s部署方式。包括minikube部署，kubeadm部署，kubeasz部署，rancher部署，k3s部署。包括开发测试环境部署k8s，和生产环境部署k8s。

腾讯课堂连接地址https://ke.qq.com/course/478827?taid=4373109931462251&tuin=ba64518

第二个视频发布 https://edu.csdn.net/course/detail/27109

腾讯课堂连接地址https://ke.qq.com/course/484107?tuin=ba64518

介绍主要的k8s资源的使用配置和命令。包括configmap，pod，service，replicaset，namespace，deployment，daemonset，ingress，pv，pvc，sc，role，rolebinding，clusterrole，clusterrolebinding，secret，serviceaccount，statefulset，job，cronjob，podDisruptionbudget，podSecurityPolicy,networkPolicy,resourceQuota，limitrange，endpoint，event，conponentstatus，node，apiservice，controllerRevision等。

第三个视频发布：https://edu.csdn.net/course/detail/27574

详细介绍helm命令，学习helm chart语法，编写helm chart。深入分析各项目源码，学习编写helm插件 ————————————————

增加maven配置

<dependency> <groupId>ch.qos.logback.contrib</groupId> <artifactId>logback-json-classic</artifactId> <version>0.1.5</version> </dependency> <dependency> <groupId>ch.qos.logback.contrib</groupId> <artifactId>logback-jackson</artifactId> <version>0.1.5</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-core</artifactId> <version>2.7.5</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-annotations</artifactId> <version>2.7.5</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>2.7.5</version> </dependency> <dependency> <groupId>ch.qos.logback</groupId> <artifactId>logback-classic</artifactId> <version>1.0.1</version> </dependency>

配置logback配置文件：

<?xml version="1.0" encoding="utf-8"?> <!DOCTYPE configuration> <configuration> <jmxConfigurator/> <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> <layout class="ch.qos.logback.classic.PatternLayout"> <pattern>%d [%X{requestURIWithQueryString}] %-5p %c{2} - %m%n</pattern> </layout> </appender> <appender name="PROJECT" class="wint.lang.logback.WintRollingFileAppender"> <File>${orange.log.dir}/orange.log</File> <append>true</append> <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"> <FileNamePattern>orange.log.%i.bak</FileNamePattern> <MinIndex>1</MinIndex> <MaxIndex>12</MaxIndex> </rollingPolicy> <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy"> <MaxFileSize>100MB</MaxFileSize> </triggeringPolicy>  <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="ch.qos.logback.contrib.json.classic.JsonLayout"> <jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter"/> <appendLineSeparator>true</appendLineSeparator> </layout> </encoder> </appender> <appender name="PROJECT-DEBUG" class="wint.lang.logback.WintFileAppender"> <File>${orange.log.dir}/orange-debug.log</File> <append>false</append> <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"> <FileNamePattern>orange-debug.log.%i.bak</FileNamePattern> <MinIndex>1</MinIndex> <MaxIndex>12</MaxIndex> </rollingPolicy> <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy"> <MaxFileSize>100MB</MaxFileSize> </triggeringPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%msg%n</Pattern> </layout> </appender> <appender name="BIZ" class="wint.lang.logback.WintRollingFileAppender"> <File>${orange.log.dir}/orange-biz.log</File> <append>true</append> <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"> <FileNamePattern>orange-biz.log.%i.bak</FileNamePattern> <MinIndex>1</MinIndex> <MaxIndex>12</MaxIndex> </rollingPolicy> <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy"> <MaxFileSize>100MB</MaxFileSize> </triggeringPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%msg%n</Pattern> </layout> </appender> <appender name="FILTER" class="wint.lang.logback.WintRollingFileAppender"> <File>${orange.log.dir}/orange-filter.log</File> <append>true</append> <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"> <FileNamePattern>orange-filter.log.%i.bak</FileNamePattern> <MinIndex>1</MinIndex> <MaxIndex>12</MaxIndex> </rollingPolicy> <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy"> <MaxFileSize>100MB</MaxFileSize> </triggeringPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%msg%n</Pattern> </layout> </appender> <appender name="APACHE" class="wint.lang.logback.WintFileAppender"> <File>${orange.log.dir}/orange-apache.log</File> <append>true</append> <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"> <FileNamePattern>orange-apache.log.%i.bak</FileNamePattern> <MinIndex>1</MinIndex> <MaxIndex>12</MaxIndex> </rollingPolicy> <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy"> <MaxFileSize>100MB</MaxFileSize> </triggeringPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%msg%n</Pattern> </layout> </appender> <appender name="TEMPLATE" class="wint.lang.logback.WintRollingFileAppender"> <File>${orange.log.dir}/orange-template.log</File> <append>true</append> <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"> <FileNamePattern>orange-template.log.%i.bak</FileNamePattern> <MinIndex>1</MinIndex> <MaxIndex>12</MaxIndex> </rollingPolicy> <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy"> <MaxFileSize>100MB</MaxFileSize> </triggeringPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%msg%n</Pattern> </layout> </appender> <appender name="SEARCH" class="wint.lang.logback.WintRollingFileAppender"> <File>${orange.log.dir}/orange-search.log</File> <append>true</append> <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"> <FileNamePattern>orange-search.log.%i.bak</FileNamePattern> <MinIndex>1</MinIndex> <MaxIndex>100</MaxIndex> </rollingPolicy> <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy"> <MaxFileSize>100MB</MaxFileSize> </triggeringPolicy> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} -%msg%n</Pattern> </layout> </appender> <logger name="com.lie100.orange"> <level value="info"/> <appender-ref ref="BIZ"/> </logger> <logger name="wint.lang.misc.profiler.Profiler" additivity="false"> <level value="info"/> <appender-ref ref="FILTER"/> </logger> <logger name="com.lie100.orange.biz.bo.impl.ResumeSearchBOImpl" additivity="false"> <level value="warn"/> <appender-ref ref="SEARCH"/> </logger> <logger name="wint.mvc.template.engine"> <level value="error"/> <appender-ref ref="TEMPLATE"/> </logger> <logger name="org.apache"> <level value="info"/> <appender-ref ref="APACHE"/> </logger> <logger name="org.apache.commons.beanutils"> <level value="error"/> </logger> <logger name="org.apache.commons.digester"> <level value="error"/> </logger> <root> <level value="${orange.log.root.level}"/> <appender-ref ref="PROJECT"/> <appender-ref ref="PROJECT-DEBUG"/> </root> </configuration>

关键是这段：

json格式日志输出

fluentd配置文件：

kind: ConfigMap apiVersion: v1 metadata: name: fluentd-es-config-jetty-orange labels: addonmanager.kubernetes.io/mode: Reconcile data: system.conf: |- <system> root_dir /tmp/fluentd-buffers/ </system> input.conf: |- <source> @id jetty.log @type tail path /var/lib/jetty/logs/orange.log pos_file /root/jetty.log.pos time_format %Y-%m-%dT%H:%M:%S.%NZ tag jetty_orange format json read_from_head true </source> output.conf: |- <match **> @id elasticsearch @type elasticsearch @log_level info include_tag_key true host elasticsearch-logging.kube-system.svc.cluster.local port 9200 logstash_format true logstash_prefix docker.jetty logstash_dateformat %Y-%m-%d type_name docker_container_orange_log <buffer> @type file path /var/log/fluentd-buffers/kubernetes.system.buffer flush_mode interval retry_type exponential_backoff flush_thread_count 2 flush_interval 5s retry_forever retry_max_interval 30 chunk_limit_size 2M queue_limit_length 8 overflow_action block </buffer> </match>

以sidecar方式启动fluentd，发送日志到elasticsearch

apiVersion: extensions/v1beta1 kind: Deployment metadata: name: orange-web labels: app: orange spec: replicas: 1 template: metadata: labels: app: orange spec: terminationGracePeriodSeconds: 60 nodeSelector: deploy: app containers: - name: orange-web image: 192.168.1.225:5000/youben/orange-web:1.0-SNAPSHOT imagePullPolicy: Always readinessProbe: httpGet: path: /error.htm port: 8080 scheme: HTTP initialDelaySeconds: 30 timeoutSeconds: 5 livenessProbe: httpGet: path: /error.htm port: 8080 scheme: HTTP initialDelaySeconds: 30 timeoutSeconds: 5 ports: - containerPort: 8080 name: http - containerPort: 8888 name: dobbo volumeMounts: - name: shared-data mountPath: /var/lib/jetty/logs - name: fluentd-es image: registry.cn-hangzhou.aliyuncs.com/hxpdocker/fluentd-elasticsearch:v2.0.4 imagePullPolicy: IfNotPresent env: - name: FLUENTD_ARGS value: --no-supervisor -q volumeMounts: - name: config-volume mountPath: /etc/fluent/config.d - name: shared-data mountPath: /var/lib/jetty/logs volumes: - name: config-volume configMap: name: fluentd-es-config-jetty-orange - name: shared-data emptyDir: {} imagePullSecrets: - name: regcred --- apiVersion: v1 kind: Service metadata: name: orange-web labels: app: orange spec: ports: - port: 8080 targetPort: 8080 selector: app: orange --- apiVersion: extensions/v1beta1 kind: Ingress metadata: name: orange-ingress labels: app: orange annotations: kubernetes.io/ingress.class: "app" spec: rules: - host: resume.lie100.net http: paths: - backend: serviceName: orange-web servicePort: 8080

转载请注明原文地址: https://www.6miu.com/read-2630931.html

技术

最新回复(0)