# 一、把这几个配置文件丢到主节点服务器


vi service-account.yaml

```yml
apiVersion: v1
automountServiceAccountToken: false
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.17.0
  name: kube-state-metrics
  namespace: kube-system-xuan
```
vi cluster-role.yaml

```yml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.17.0
  name: kube-state-metrics
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - secrets
  - nodes
  - pods
  - services
  - serviceaccounts
  - resourcequotas
  - replicationcontrollers
  - limitranges
  - persistentvolumeclaims
  - persistentvolumes
  - namespaces
  - endpoints
  verbs:
  - list
  - watch
- apiGroups:
  - apps
  resources:
  - statefulsets
  - daemonsets
  - deployments
  - replicasets
  verbs:
  - list
  - watch
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs
  verbs:
  - list
  - watch
- apiGroups:
  - autoscaling
  resources:
  - horizontalpodautoscalers
  verbs:
  - list
  - watch
- apiGroups:
  - authentication.k8s.io
  resources:
  - tokenreviews
  verbs:
  - create
- apiGroups:
  - authorization.k8s.io
  resources:
  - subjectaccessreviews
  verbs:
  - create
- apiGroups:
  - policy
  resources:
  - poddisruptionbudgets
  verbs:
  - list
  - watch
- apiGroups:
  - certificates.k8s.io
  resources:
  - certificatesigningrequests
  verbs:
  - list
  - watch
- apiGroups:
  - discovery.k8s.io
  resources:
  - endpointslices
  verbs:
  - list
  - watch
- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  - volumeattachments
  verbs:
  - list
  - watch
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - mutatingwebhookconfigurations
  - validatingwebhookconfigurations
  verbs:
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - networkpolicies
  - ingressclasses
  - ingresses
  verbs:
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - list
  - watch
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - clusterrolebindings
  - clusterroles
  - rolebindings
  - roles
  verbs:
  - list
  - watch

```
vi cluster-role-binding.yaml

```yml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.17.0
  name: kube-state-metrics
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-state-metrics
subjects:
- kind: ServiceAccount
  name: kube-state-metrics
  namespace: kube-system-xuan
```
vi  deployment.yaml

```js
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.17.0
  name: kube-state-metrics
  namespace: kube-system-xuan
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: kube-state-metrics
  template:
    metadata:
      labels:
        app.kubernetes.io/component: exporter
        app.kubernetes.io/name: kube-state-metrics
        app.kubernetes.io/version: 2.17.0
    spec:
      automountServiceAccountToken: true
      containers:
      - image: bitnami/kube-state-metrics:latest
        livenessProbe:
          httpGet:
            path: /livez
            port: http-metrics
          initialDelaySeconds: 5
          timeoutSeconds: 5
        name: kube-state-metrics
        ports:
        - containerPort: 8080
          name: http-metrics
        - containerPort: 8081
          name: telemetry
        readinessProbe:
          httpGet:
            path: /readyz
            port: telemetry
          initialDelaySeconds: 5
          timeoutSeconds: 5
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 65534
          seccompProfile:
            type: RuntimeDefault
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: kube-state-metrics
```

vi service.yaml

```js
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.17.0
  name: kube-state-metrics
  namespace: kube-system-xuan
spec:
  type: NodePort
  ports:
  - name: http-metrics
    port: 8080
    targetPort: 8080
    nodePort: 31666
    protocol: TCP
  selector:
    app.kubernetes.io/name: kube-state-metrics
```


```js
kubectl apply -f service-account.yaml
kubectl apply -f cluster-role.yaml
kubectl apply -f cluster-role-binding.yaml
kubectl apply -f deployment.yaml
kubectl apply -f service.yaml
```

# 二、配置普罗米修斯
prometheus.yml 
```yml
# 全局配置
global:
  # 数据抓取间隔（默认1分钟）
  scrape_interval: 15s
  # 告警规则评估频率（建议与抓取间隔一致）
  evaluation_interval: 15s
  # 抓取超时时间（默认10秒）

# 告警配置
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # 启用Alertmanager时需配置实际地址
          # - alertmanager:9093

# 规则文件配置
rule_files:
  # 告警规则文件路径（支持通配符）
  # - 'alert_rules/*.rules'
  # 记录规则文件路径
  # - 'recording_rules/*.rules'

# 监控目标配置
scrape_configs:
  - job_name: "prometheus-server"
    file_sd_configs:
      - files:
          - /server/prometheus/node_exporter.yml
        refresh_interval: 5s

#nginx服务
  - job_name: nginx
    static_configs:
     - targets: ['192.168.201.104:9113']
     
#K8S服务器
  - job_name: K8S
    static_configs:
     - targets: ['192.168.201.100:31666']

```
然后重启普罗米修斯

# Grafana模版配置
https://grafana.com/grafana/dashboards/17519-kube-state-metrics-v2/
```json
17519
13105
```

![image.png](/static/img/d0406c74fe652fb2bc51fe37785f5994.image.webp)

7.Prometheus监控K8S


```js
vi /etc/containerd/config.toml
```


```js
bled_plugins = ["restart"]
[plugins.linux]
shim_debug = true

[plugins.cri.registry.mirrors."docker.io"]
endpoint = [
    "https://docker.m.daocloud.io",
    "https://docker.1panel.live",
    "https://hub.rat.dev",
    "https://registry-1.docker.io"  # 官方源保底
]

[plugins.cri]
sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.2"
```

```js
systemctl restart containerd
```



containerd加速镜像配置

# 一、先在nginx.conf配置加上监控配置

```js
    # 新增监控模块配置
location /nginx_status {
    stub_status on;
    allow all;  # 允许所有IP访问
}
```

完整配置：
```yml
    server {
        listen       81;   #监听的端口
        server_name  localhost;   #监听的主机IP
        

    # 新增监控模块配置
location /nginx_status {
    stub_status on;
    allow all;  # 允许所有IP访问
}



        location / {
            root   /usr/local/nginx/html;
            index  weihu.html;
          #  proxy_pass  http://songxuan;  #直接起个命名
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

    }
```
然后重启nginx让配置生效。
如果遇到执行报错问题，是你这个监控项安装nginx编译时没加这个。

```js
nginx: [emerg] unknown directive "stub_status" in /usr/local/nginx/conf/songxuan.conf:12
```

解决这个问题：
执行命令检查Nginx编译参数，若无输出则表示未编译该模块：
```js
nginx -V 2>&1 | grep -o with-http_stub_status_module
```
重新进入解压的nginx的包内，重新带编译安装，然后在用检查命令检查，然后启动。
```js
./configure --prefix=/usr/local/nginx --with-http_stub_status_module
make
make install
```

# 二、安装nginx-exporter，因为需要go语言环境，只能选择docker安装

```js
version: '3.8'
services:
  nginx-exporter:
    image: nginx/nginx-prometheus-exporter:latest
    ports:
      - "9113:9113"
    command: 
      - -nginx.scrape-uri=http://192.168.201.104/nginx_status  # 替换为您的实际Nginx状态页地址
    restart: unless-stopped
```
#  三、配置普罗米修斯的配置文件

```yml
# 全局配置
global:
  # 数据抓取间隔（默认1分钟）
  scrape_interval: 15s
  # 告警规则评估频率（建议与抓取间隔一致）
  evaluation_interval: 15s
  # 抓取超时时间（默认10秒）

# 告警配置
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # 启用Alertmanager时需配置实际地址
          # - alertmanager:9093

# 规则文件配置
rule_files:
  # 告警规则文件路径（支持通配符）
  # - 'alert_rules/*.rules'
  # 记录规则文件路径
  # - 'recording_rules/*.rules'

# 监控目标配置
scrape_configs:
  - job_name: "prometheus-server"
    file_sd_configs:
      - files:
          - /server/prometheus/node_exporter.yml
        refresh_interval: 5s
        
#nginx监控项
  - job_name: nginx
    static_configs:
     - targets: ['192.168.201.104:9113']

```
然后重启普罗米修斯。

然后查看普罗米修斯nginx的已经监控到了

![image.png](/static/img/f058521c73e35f834f01c5a8eed6eb44.image.webp)

# 四、最后去grafana导入仪表盘展示

![image.png](/static/img/c733bb4cc8e8e6671f878775f6590243.image.webp)

![image.png](/static/img/7a65255344f5817873a1fa75007390e2.image.webp)


6.Prometheus监控Nginx

# 一、安装Grfana

centos下载安装，其余去看官网：https://grafana.com/grafana/download?platform=linux
```js
sudo yum install -y https://dl.grafana.com/grafana-enterprise/release/12.2.0/grafana-enterprise_12.2.0_17949786146_linux_amd64.rpm
```

启动grfana
```js
systemctl start grafana-server.service
```

访问grfana:
IP+3000端口

账号密码默认：admin

登录后跳过，先不改密码
![image.png](/static/img/23fc9be4bc554aa1f7c8d08911ec878b.image.webp)

# 二、添加一个普罗米修斯的数据源
![image.png](/static/img/92961102f34d814e71c8a6cb4a6963a4.image.webp)

![image.png](/static/img/d86e0ce302245f3c23192063965c0b1d.image.webp)

![image.png](/static/img/06780055d957af8955bf4ae08c34c0bd.image.webp)
最后保存

![image.png](/static/img/35cc176506d8b4dab088b73273d6044a.image.webp)
仪表盘导入
![](https://songxuan.vip/static/img/382a172415b712c8a39ce9a59ab2e776.image.webp)


# 二、使用仪表盘查询

![image.png](/static/img/a68c721021512b9025cb7877777c629e.image.webp)

![image.png](/static/img/fafb41d52b634dc49ae847b07692fa7c.image.webp)

然后查看为啥没数据

![image.png](/static/img/63e1fc968e75b5191aa3261eea3c41c8.image.webp)

![image.png](/static/img/6b963a77be1532762e27a4a108214cc8.image.webp)

因为这里跟我们yml文件配置的job名称不一样

![image.png](/static/img/bd094c0eabf75c01121a08b599b8b0db.image.webp)
![image.png](/static/img/20bfa733c170bfa48cba83f14bad94d2.image.webp)
改了后就出来了

![image.png](/static/img/d36ddf4da25f2e913c006ce364b60304.image.webp)

# 三、模版的太少了，直接去官网下载
https://grafana.com/grafana/dashboards/1860-node-exporter-full/


![image.png](/static/img/1a654497c0e9bf5a453e6c5af2918669.image.webp)

![image.png](/static/img/7cea9fbd32d65c6dd17bebe9bf357b6d.image.webp)

![image.png](/static/img/f607930b73b172aa100e8af9ce1378fd.image.webp)

![image.png](/static/img/f788777bad02717795ad4227a5525454.image.webp)


![image.png](/static/img/59bd1bd8993a2b266523cc9fd7198c6a.image.webp)

5.Grfana

# 一、 PromQL查询
## 查询所有主机节点的占用的内存
```js
node_memory_Mapped_bytes
```

![image.png](/static/img/dc30af6d2695b47816c4ea6bbe1cd8f0.image.webp)

图形化界面查看
![image.png](/static/img/013f8cca1e743e384cc08bda00222a88.image.webp)


## 查询所有主机节点的CPU

```js
node_cpu_guest_seconds_total
```

![image.png](/static/img/2f19274cf9e42a5c03e51761dbe3907f.image.webp)

![](https://songxuan.vip/static/img/1d3418e2d85ae50a9e3318a85195a20a.image.webp)

## 查询所有主机节点的CPU，并进行过滤

```js
node_cpu_guest_seconds_total{instance="192.168.201.111:9100"}
```
这样就把该192.168.201.111:9100节点的cpu情况查询出来了
![image.png](/static/img/b48217fa335990b8bac2d28ae525d2a5.image.webp)

## 查询所有主机节点的CPU，排除不要的

```js
node_cpu_guest_seconds_total{mode!="nice"}
```
这样就只有user的了
![image.png](/static/img/ab588bace3363ed08d9d87e3ac22b1cb.image.webp)

## 查询所有主机磁盘的信息

```js
node_disk_info
```

![image.png](/static/img/9ea9e7b48ab798eedd48ce3b3a662ac9.image.webp)

## 模糊匹配
```js
node_disk_info{instance=~"192.168.201.110.*"}
```

# 二、常用函数
rate(v range-vector)计算时间序列在范围向量中的每秒平均增长率。自动调整单调性中断（例如由于目标重启而重置的计数器）。此外，计算会延伸到时间范围的两端，允许错过抓取或抓取周期与范围时间周期的不完美对齐。

```js
rate(node_cpu_guest_seconds_total{instance="192.168.201.111:9100"}[1m])
```



COUNT() 函数‌

‌用途‌：计算匹配标签的时间序列 ‌数量‌（统计序列条数）

‌本质‌：计数操作，‌不关心具体的指标数值

统计所有磁盘的数量（每个磁盘对应一条时间序列）：
```js
count(node_disk_info)
```

SUM() 函数‌

‌用途‌：对匹配时间序列的 ‌数值进行求和‌（计算总值）

‌本质‌：数值累加操作，‌要求指标必须是数值类型

计算所有磁盘的写入字节总量（累加每个磁盘的写入量）：
```js
sum(node_disk_written_bytes_total)
```


# 三、复杂的查询
内存使用率计算
```js
(node_memory_MemTotal_bytes - node_memory_MemFree_bytes) 
  / 
node_memory_MemTotal_bytes * 100
```
‌作用‌：计算‌已用内存百分比‌

‌公式‌：(总内存 - 空闲内存) / 总内存 * 100
![image.png](/static/img/0ef5593886fd5659babd5e8930859ccc.image.webp)


内存可用率计算
```js
node_memory_MemAvailable_bytes 
  / 
node_memory_MemTotal_bytes * 100
```
作用‌：计算‌可用内存百分比‌

‌公式‌：(可用内存) / 总内存 * 100

![image.png](/static/img/610f6d086b3c3143dc369bec2080fef0.image.webp)