标签 EdgeX 下的文章

准备工作

硬件

  • 主 K3S Master 节点：3台, master101, master102, master103
  • 业务 K3S Master 节点：3台，master151, master152, master153
  • Worker 节点：N台，随意, worker01, worker02, ...

软件

基础配置

  • 操作系统：Ubuntu Server 22.04
    最小化安装系统，包含：openssh-server, iputils-ping, net-tools, vim
PRETTY_NAME="Ubuntu 22.04.5 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
  • 管理集群
主机名 IP 其他
database 192.168.198.100 数据库+NFS存储，如有需要做 HA
master101 192.168.198.101 管理 K3S master
master102 192.168.198.102 管理 K3S master
master103 192.168.198.103 管理 K3S master
  • 业务集群
主机名 IP 其他
master151 192.168.123.151 业务 K3S master + worker
master152 192.168.123.152 业务 K3S master + worker
master153 192.168.123.153 业务 K3S master + worker
worker161 192.168.123.161 业务 worker
worker162 192.168.123.162 业务 worker

数据库配置

Docker

  • 准备 docker 环境

    curl -fsSL https://get.docker.com | sh -

  • 验证 docker 

    docker images
REPOSITORY   TAG       IMAGE ID   CREATED   SIZE

  • 设置一个 Registry 镜像

    cat /etc/docker/daemon.json
{
  "insecure-registries" : [ "0.0.0.0/0" ],
  "registry-mirrors": [ "https://xxx.mirror.swr.myhuaweicloud.com" ]
}

  • 重启 docker

    systemctl restart docker

数据库

PostgreSQL

  • 创建 compose.yml

    services:
postgres:
  image: postgres:alpine
  container_name: postgres
  restart: unless-stopped
  environment:
    POSTGRES_USER: k3s
    POSTGRES_PASSWORD: talent
    POSTGRES_DB: k3s
  ports:
    - "5432:5432" 
  volumes:
    - ./data:/var/lib/postgresql/data
    - ./init-db.sh:/docker-entrypoint-initdb.d/init-db.sh

  • 创建脚本 init-db.sh

    #!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname=postgres <<-EOSQL
  CREATE DATABASE harbor;
EOSQL

  • 启动 PostgreSQL 数据库容器

    docker compose up -d

  • 验证数据库

    docker exec -it postgres psql -U k3s -c '\l'
                                               List of databases
 Name    | Owner | Encoding | Locale Provider |  Collate   |   Ctype    | Locale | ICU Rules | Access privileges 
-----------+-------+----------+-----------------+------------+------------+--------+-----------+-------------------
harbor    | k3s   | UTF8     | libc            | en_US.utf8 | en_US.utf8 |        |           | 
k3s       | k3s   | UTF8     | libc            | en_US.utf8 | en_US.utf8 |        |           | 
...
(5 rows)

MySQL

因为 Rancher 只支持 MySQL 不支持 PostgreSQL

  • 创建 compose.yml

    services:
mysql:
  image: mysql:latest
  container_name: mysql
  restart: unless-stopped
  environment:
    MYSQL_ROOT_PASSWORD: talent
    MYSQL_DATABASE: rancher
    MYSQL_USER: rancher
    MYSQL_PASSWORD: talent
  ports:
    - "3306:3306"
  volumes:
    - ./data:/var/lib/mysql             
    - ./init:/docker-entrypoint-initdb.d

  • 创建脚本 init/permissions.sql

    GRANT ALL PRIVILEGES ON *.* TO 'rancher'@'%' IDENTIFIED BY 'talent';
FLUSH PRIVILEGES;

  • 启动 MySQL 数据库容器

    docker compose up -d

  • 验证数据库

    docker exec -it mysql mysql -u rancher -p'talent' -e "SHOW DATABASES;"
+--------------------+
| Database           |
+--------------------+
| rancher            |
+--------------------+

部署 K3S 管理集群

本章节内容在管理 K3S 主机 上执行

主机名

  • 所有的 Master 配置相同的 /etc/hosts
    127.0.0.1 localhost
127.0.1.1 master101 # CHAMGEME
192.168.198.100      database
192.168.198.101      master101
192.168.198.102      master102
192.168.198.103      master103

K3S 核心安装

自动安装(推荐)

  • 可以连接互联网的情况下
    默认安装最新版本，这里是: v1.30.5+k3s1，注意：k3s:talent 是你自己的数据库用户名密码，使用一样的 token

    curl -sfL https://get.k3s.io | sh -s - server \
--datastore-endpoint="postgres://k3s:talent@database:5432/k3s?sslmode=disable" --token K10c71232f051944de58ab058871ac7a85b42cbdee5c6df94deb6bb493c79b15a92::server:3d6c2d90cec2e63db66bde0e6f24013d

  • 验证安装

    kubectl get nodes
NAME        STATUS   ROLES                  AGE   VERSION
master101   Ready    control-plane,master   43s   v1.30.5+k3s1

  • 其他 Master 安装方式一样。在任何一台 Master 上确认安装完成

    kubectl get nodes 
NAME        STATUS   ROLES                  AGE    VERSION
master101   Ready    control-plane,master   117s   v1.30.5+k3s1
master102   Ready    control-plane,master   118s   v1.30.5+k3s1
master103   Ready    control-plane,master   29s    v1.30.5+k3s1

手动二进制文件安装(不推荐)

  • 下载二进制文件

    wget https://github.com/k3s-io/k3s/releases/download/vX.X.X/k3s -O /usr/local/bin/k3s
chmod +x /usr/local/bin/k3s

  • 初始化

    k3s server --cluster-init

  • 设置自启动脚本
    所有 Master 机器上一样的配置

    cat <<EOF | sudo tee /etc/systemd/system/k3s.service
[Unit]
Description=Lightweight Kubernetes
Documentation=https://k3s.io
After=network.target
[Service]
ExecStart=/usr/local/bin/k3s server --datastore-endpoint="postgres://k3s:talent@database:5432/k3s?sslmode=disable" --token K10c71232f051944de58ab058871ac7a85b42cbdee5c6df94deb6bb493c79b15a92::server:3d6c2d90cec2e63db66bde0e6f24013d
Restart=always
RestartSec=5s
[Install]
WantedBy=multi-user.target
EOF

  • 更新自启动设置

    systemctl daemon-reload
systemctl enable k3s
systemctl start k3s

  • 查看集群状态
    任何一台 Master 都可以

    /usr/local/bin/k3s kubectl get nodes

    结果

    NAME        STATUS   ROLES                  AGE   VERSION
master101   Ready    control-plane,master   10m   v1.30.4+k3s1
master102   Ready    control-plane,master   10m   v1.30.4+k3s1
master103   Ready    control-plane,master   81s   v1.30.4+k3s1

Master / Worker 手动加入(如有必要)

  • 获取集群 token

    cat /var/lib/rancher/k3s/server/node-token

  • Master 直接加入集群（不建议）

    k3s server --server https://master101:6443 --token <token>

    或修改 /etc/systemd/system/k3s.service

  • Worker加入（可选）
    替换<node-token> 为你的token

    cat <<EOF | sudo tee /etc/systemd/system/k3s-agent.service
[Unit]
Description=Lightweight Kubernetes
Documentation=https://k3s.io
After=network.target
[Service]
ExecStart=/usr/local/bin/k3s agent --server https://master101:6443 --token <node-token>
Restart=always
RestartSec=5s
[Install]
WantedBy=multi-user.target
EOF

  • 更新自启动设置

    systemctl daemon-reload
systemctl enable k3s-agent
systemctl start k3s-agent

不管怎么操作，基础三个主机的 K3S 集群要跑起来，保证基础工作完成。

安装 Rancher 集群管理工具和 Harbor Registry 仓库

准备工作

安装 Kubernetes 包管理工具 Helm

  • 选择一个 Master 或全部 Master，安装 Helm

    curl -fsSL  https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash -

  • 设置环境变量
    加入到 /etc/profile 自启动

    echo "export KUBECONFIG=/etc/rancher/k3s/k3s.yaml" >> /etc/profile && source /etc/profile

安装证书管理器 cert-manager

  • helm 脚本安装

    kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.15.3/cert-manager.crds.yaml
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager --namespace cert-manager \
--create-namespace --version v1.15.3

  • 验证安装成功

    kubectl get pods --namespace cert-manager
NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-9647b459d-9jwpj               1/1     Running   0          2m47s
cert-manager-cainjector-5d8798687c-66r95   1/1     Running   0          2m47s
cert-manager-webhook-c77744d75-k22h5       1/1     Running   0          2m47s

  • 设置自签名 Issuer
    创建 issuer-cert.yaml, 这里创建两个： rancher.yiqisoft.com hub.yiqisoft.com, 后面访问集群业务就是通过域名来通信

    apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: selfsigned-issuer
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: rancher-cert
namespace: cattle-system
spec:
secretName: rancher-cert-secret
issuerRef:
  name: selfsigned-issuer
  kind: ClusterIssuer
commonName: rancher.yiqisoft.com
duration:  315360000s
renewBefore: 240h
dnsNames:
  - rancher.yiqisoft.com
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: hub-cert
namespace: harbor
spec:
secretName: hub-cert-secret
issuerRef:
  name: selfsigned-issuer
  kind: ClusterIssuer
commonName: hub.yiqisoft.com
duration:  315360000s
renewBefore: 240h
dnsNames:
  - hub.yiqisoft.com

  • 安装自定义 Certificate
    先创建两个 namespace，再安装脚本

    kubectl create namespace harbor
kubectl create namespace cattle-system
kubectl apply -f issuer-cert.yaml

    结果输出

    clusterissuer.cert-manager.io/selfsigned-issuer created
certificate.cert-manager.io/rancher-cert created
certificate.cert-manager.io/hub-cert created

安装 Rancher

  • 增加 Rancher Helm 仓库

    helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
helm repo update

  • 安装 Rancher
    这个过程比较漫长，需要从互联网下载

    helm install rancher rancher-latest/rancher \
--namespace cattle-system \
--set hostname=rancher.yiqisoft.com \
--set replicas=3 \
--set extraEnv[0].name=CATTLE_DATABASE_ENDPOINT \
--set extraEnv[0].value=database:3306 \
--set extraEnv[1].name=CATTLE_DATABASE_USER \
--set extraEnv[1].value=rancher \
--set extraEnv[2].name=CATTLE_DATABASE_PASSWORD \
--set extraEnv[2].value=talent \
--set extraEnv[3].name=CATTLE_DATABASE_NAME \
--set extraEnv[3].value=rancher

  • 验证安装成功

    kubectl get pods --namespace cattle-system

    结果

    NAME                               READY   STATUS    RESTARTS     AGE
rancher-567cc5c6ff-bzx42           1/1     Running   0            65m
rancher-567cc5c6ff-mgvns           1/1     Running   0            86m
rancher-567cc5c6ff-rbgjn           1/1     Running   0            86m
rancher-webhook-77b49b9ff9-mp2dm   1/1     Running   0            142m

安装 Harbor

Harbor 作为内部 docker hub 使用

使用 Ceph（推荐）

准备一个 Ceph 集群用于存储（这里省略）。

使用 NFS

简单实用，但是效率不高，安全性一般。

  • 在一个文件服务器上部署 NFS，这里用 database 主机
    apt update
apt install nfs-kernel-server -y
mkdir -p /opt/nfs_share
chown nobody:nogroup /opt/nfs_share
chmod 777 /opt/nfs_share
echo "/opt/nfs_share *(rw,sync,no_subtree_check)" >> /etc/exports
exportfs -ra
systemctl restart nfs-kernel-server

部署 pv 和 pvc

  • 编辑 pv-pvc.yaml

    apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv
spec:
capacity:
  storage: 30Gi
accessModes:
  - ReadWriteMany
nfs:
  path: /opt/nfs_share
  server: database
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nfs-pvc
spec:
accessModes:
  - ReadWriteMany
resources:
  requests:
    storage: 10Gi
volumeName: nfs-pv
storageClassName: ""

  • 应用到 namespace harbor

    kubectl apply -f pv-pvc.yaml -n harbor

    结果

    persistentvolume/nfs-pv created
persistentvolumeclaim/nfs-pvc created

  • 查看结果，nfs-pvc 必须是 Bound

    kubectl get pvc -n harbor

    结果

    NAME      STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
nfs-pvc   Bound    nfs-pv   30Gi       RWX                           <unset>                 92s

  • 查看 nfs-pv 必须是 Bound

    kubectl get pv -n harbor

    结果

    NAME     CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM            STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
nfs-pv   30Gi       RWX            Retain           Bound    harbor/nfs-pvc                  <unset>                          97s

部署 Harbor Helm

  • 安装 NFS 依赖
    所有的 Master 上需要安装 nfs-common，并验证（可选），可以访问 /tmp/nfs 目录

    apt install nfs-common -y
mkdir /tmp/nfs
mount -t nfs database:/opt/nfs_share /tmp/nfs
ll /tmp/nfs

  • 增加 Harbor Helm 仓库

    helm repo add harbor https://helm.goharbor.io
helm repo update

  • 导出默认 values，修改

    helm show values harbor/harbor > harbor-values.yml
# 内容修改如下
expose:
ingress:
  hosts:
    core: hub.yiqisoft.cn
externalURL: https://hub.yiqisoft.cn
persistentVolumeClaim:
  registry:/exter
    existingClaim: "nfs-pvc"
    storageClass: "-"
database:
type: external
external:
  host: "192.168.198.100"
  port: "5432"
  username: "k3s"
  password: "talent"
  coreDatabase: "harbor"
harborAdminPassword: "Harbor12345"

  • 安装 Harbor，需要通过互联网下载所有的镜像，时间根据网络带宽决定

    helm install harbor harbor/harbor -f harbor-values.yml -n harbor

  • 确认 Harbor 是否安装成功

    kubectl get pods -n harbor

    都显示 READY 即可

    NAME                               READY   STATUS    RESTARTS      AGE
harbor-core-666cf84bc6-vq8mp       1/1     Running   0             85s
harbor-jobservice-c68bc669-6twbp   1/1     Running   0             85s
harbor-portal-d6c99f896-cq9fx      1/1     Running   0             85s
harbor-redis-0                     1/1     Running   0             84s
harbor-registry-5984c768c8-4xnrg   2/2     Running   0             84s
harbor-trivy-0                     1/1     Running   0             84s

  • 验证 Harbor 工作状态

    curl -k https://hub.yiqisoft.com/v2/ -u admin:Harbor12345

    提示 {} 证明成功

    {}

以下在 database 主机上运行

  • 登录 Harbor

    docker login hub.yiqisoft.com -u admin -p Harbor12345

    登录成功

    Login Succeeded

  • 上传已经存在的 PostgreSQL image 测试

    docker tag postgres:alpine hub.yiqisoft.com/library/postgres:alpine
docker push hub.yiqisoft.com/library/postgres:alpine

    结果显示

    The push refers to repository [hub.yiqisoft.com/library/postgres]
284f9fec9bcb: Pushed 
07188499987f: Pushed 
...
63ca1fbb43ae: Pushed 
alpine: digest: sha256:f0407919d966a86e9***c81641801e43c29b size: 2402

批量上传所需 image 到 Harbor

在准备好 docker image 的机器上运行

  • 导出需要的 image

    docker images --format "{{.Repository}}:{{.Tag}}" |grep yiqisoft.com > images.txt

  • 新建 /etc/docker/daemon.json，跳过安全证书

    {
  "insecure-registries" : [ "0.0.0.0/0" ]
}

  • 重启 docker 

    systemctl restart docker

  • 新建一个脚本 push-images.sh

    #!/bin/bash
# Check if the correct number of arguments is provided
if [ $# -ne 1 ]; then
echo "Usage: $0 <path_to_images_file>"
exit 1
fi
# Assign the first argument to the file variable
file=$1
# Check if the file exists
if [ ! -f "$file" ]; then
echo "File not found: $file"
exit 1
fi
while read image; do
echo "Pushing image: $image"
docker push $image
if [ $? -eq 0 ]; then
  echo "Successfully pushed $image"
else
  echo "Failed to push $image"
fi
done < "$file"

  • 登录 Harbor 并上传: push image

    docker login hub.yiqisoft.com -u admin -p Harbor12345
bash push-images.sh images.txt

  • 查看 Harbor Registry 仓库结果

    curl -k https://hub.yiqisoft.com/v2/_catalog -u admin:Harbor12345 |json_pp

    结果显示

    {
 "repositories" : [
    "edgexfoundry/app-service-gateway",
    "edgexfoundry/consul",
    "edgexfoundry/core-command",
    "edgexfoundry/core-common-config-bootstrapper",
    "edgexfoundry/core-data",
    "edgexfoundry/core-metadata",
    "edgexfoundry/device-modbus",
    "edgexfoundry/device-onvif-camera",
    "edgexfoundry/device-openvino-face-recognition",
    "edgexfoundry/device-openvino-object-detection",
    "edgexfoundry/edgex-ui",
    "edgexfoundry/redis",
    "edgexfoundry/support-notifications",
    "edgexfoundry/support-scheduler",
    "library/hello-world",
    "library/postgres",
    "rancher/system-agent",
    "yiqisoft/media_server",
    "yiqisoft/model_server",
    "yiqisoft/nginx",
    "yiqisoft/nvr"
 ]
}

部署 K3S 业务集群

以下操作注意：全部在新建的业务集群中操作

所有的 Master 和 Worker 都加入业务集群

Rancher 中创建一个新的集群

  • Rancher UI 操作（省略）, Cluster Name: yiqisoft, Kubernetes Version: 1.30.4+k3s1

Master 加入集群(初始化业务集群)

  • 通过新集群的 Registration 加入，可以自定义一个 namespace。需要几个 Master 根据自己业务需要来定。

  • 导入 hub CA

    echo | openssl s_client -showcerts -connect hub.yiqisoft.com:443 2>/dev/null | \
openssl x509 -outform PEM > /etc/ssl/certs/hub.yiqisoft.com.crt

  • 分别在 master151-master153 主机上执行，主机名不能重复

    curl --insecure -fL https://rancher.yiqisoft.com/system-agent-install.sh | \
sudo  sh -s - --server https://rancher.yiqisoft.com --label 'cattle.io/os=linux' \
--token xft4fsrjgkn4rzqxrgrb4mztkcxtz4nhpwm58t84q54lvpsscdgd72 \
--ca-checksum 30d78ad0ece6355c40a4b3764ba02dfc96388a9c20f6b4e2bffb131cf7402c1f \
--etcd --controlplane --worker

  • 查看业务集群 node

    kubectl get nodes

    确认 Master 都在线

    NAME        STATUS   ROLES                              AGE     VERSION
master151   Ready    control-plane,etcd,master,worker   32m     v1.30.4+k3s1
master152   Ready    control-plane,etcd,master,worker   3h39m   v1.30.4+k3s1
master153   Ready    control-plane,etcd,master,worker   23m     v1.30.4+k3s1

Worker 加入业务集群

  • 首先通过 curl 直接输出 hub 证书到主机证书目录
    containerd 拉取 image 时需要信任此证书 CA

    echo | openssl s_client -showcerts -connect hub.yiqisoft.com:443 2>/dev/null | \
openssl x509 -outform PEM > /etc/ssl/certs/hub.yiqisoft.com.crt

  • 需要等 Master 加入成功后（至少一个成功注册），Worker 主机再次通过 Registration 加入，只需要选择 Worker 即可，需要定义一个 namespace label， 比如： namespace=worker01，以后可单独部署到独立应用 到 namespace 作为隔离使用。

  • 在所有业务 Worker 上执行，同样注意主机名不能重复

    curl --insecure -fL https://rancher.yiqisoft.com/system-agent-install.sh | \
sudo  sh -s - \
--server https://rancher.yiqisoft.com \
--label 'cattle.io/os=linux' \
--token xft4fsrjgkn4rzqxrgrb4mztkcxtz4nhpwm58t84q54lvpsscdgd72 \
--ca-checksum 30d78ad0ece6355c40a4b3764ba02dfc96388a9c20f6b4e2bffb131cf7402c1f \
--worker \
--label namespace=worker161

  • 验证 Worker 节点

    kubectl get nodes -A

    结果正常

    NAME        STATUS   ROLES                              AGE     VERSION
master151   Ready    control-plane,etcd,master,worker   74m     v1.30.4+k3s1
master152   Ready    control-plane,etcd,master,worker   4h21m   v1.30.4+k3s1
master153   Ready    control-plane,etcd,master,worker   65m     v1.30.4+k3s1
worker161   Ready    worker                             25m     v1.30.4+k3s1

(部署/更新) Harbor hub CA 证书到所有的 Master 和 Worker

(手动操作)导出 Harbor hub HTTPS 证书

  • 增加自签名证书到 ca-certificates
    所有的 Master 和 Worker 上执行（不太可行）
    echo | openssl s_client -showcerts -connect hub.yiqisoft.com:443 2>/dev/null | \
openssl x509 -outform PEM > /etc/ssl/certs/hub.yiqisoft.com.crt

(自动部署)Harbor Hub CA 证书

新建一个 Secret

  • 在业务集群中，把 Harbor 证书导入集群 kube-system

    kubectl create secret generic hub-ca --from-file=ca.crt=/hub.yiqisoft.com.crt -n kube-system

  • 查看

    kubectl get secret -A |grep hub-ca

    结果

    NAMESPACE    NAME    TYPE               DATA   AGE
kube-system  hub-ca  kubernetes.io/tls  2      101m

新建一个 DaemonSet

  • 新建一个部署文件: ca-distributor.yaml，用于分发 hub 的证书，非常重要，否则无法从私有 Registry 拉取镜像。

    apiVersion: apps/v1
kind: DaemonSet
metadata:
name: ca-distributor
namespace: kube-system
spec:
selector:
  matchLabels:
    name: ca-distributor
template:
  metadata:
    labels:
      name: ca-distributor
  spec:
    containers:
    - name: ca-distributor
      image: busybox:latest
      command:
      - /bin/sh
      - -c
      - "cp /etc/ca-certificates/tls.crt /etc/ssl/certs/hub.yiqisoft.cn.crt && echo 'Certificate Updated' && sleep 3600"
      volumeMounts:
      - name: ca-cert
        mountPath: /etc/ca-certificates
      - name: ssl-certs
        mountPath: /etc/ssl/certs
    volumes:
    - name: ca-cert
      secret:
        secretName: hub-ca
    - name: ssl-certs
      hostPath:
        path: /etc/ssl/certs
        type: Directory
    restartPolicy: Always

  • 部署 yaml

    kubectl apply -f ca-distributor.yaml

  • 验证结果,确保所有的节点都执行成功。

    kubectl get pods -A  |grep ca-distributor
kube-system     ca-distributor-ckhgh  1/1     Running     0  100s
kube-system     ca-distributor-hqzwt  1/1     Running     0  101s
kube-system     ca-distributor-lj72n. 1/1     Running     0  101s
kube-system     ca-distributor-mpp7m  1/1     Running     0  101s

  • 在 Worker 上拉取镜像 image

    k3s ctr i pull hub.yiqisoft.com/library/hello-world:latest

    验证结果

    hub.yiqisoft.cn/library/hello-world:latest:  resolved       |++++++++++++++++++++++++++++++++++++++| 
manifest-sha256:d37ada95d47ad12224c205a938129df7a3e52345828b4fa27b03a98825d1e2e7: done           |++++++++++++++++++++++++++++++++++++++| 
config-sha256:d2c94e258dcb3c5ac2798d32e1249e42ef01cba4841c2234249495f87264ac5a:   done           |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:c1ec31eb59444d78df06a974d155e597c894ab4cda84f08294145e845394988e:    done           |++++++++++++++++++++++++++++++++++++++| 
elapsed: 0.6 s total:  524.0  (871.0 B/s)                                       
unpacking linux/amd64 sha256:d37ada95d47ad12224c205a938129df7a3e52345828b4fa27b03a98825d1e2e7...
done: 54.873311ms

基于 K3S 部署业务应用(边缘计算框架)

以下所有部署都是通过 Helm Chart 工具进行，需要自行编写。

EdgeX Foundry Base

  • 部署基本框架结果
    kubectl get pods -n worker161
NAME                                          READY   STATUS    RESTARTS    AGE
edgex-core-command-tspw7                      1/1     Running   0           5m41s
edgex-core-common-config-bootstrapper-92q25   1/1     Running   0           5m25s
edgex-core-consul-8lqxz                       1/1     Running   0           6m4s
edgex-core-data-nv26d                         1/1     Running   0           6m23s
edgex-core-metadata-l6c49                     1/1     Running   0           5m3s
edgex-device-onvif-camera-5tt4w               1/1     Running   0           3m26s
edgex-mqtt-broker-j2jdr                       1/1     Running   0           81s
edgex-redis-rf4v7                             1/1     Running   0           4m50s
edgex-support-notifications-ws6l5             1/1     Running   0           4m28s
edgex-support-scheduler-drx2p                 1/1     Running   0           3m56s
edgex-ui-v55vm                                1/1     Running   0           3m45s

部署辅助应用：流媒体，推理服务器，NVR

  • 部署辅助应用
    业务 pod 之间有依赖关系，需等待其他 pod 起来才能工作，会不断的重试
    kubectl get pods -n worker161 |grep -v edgex

    结果

    NAME                   READY   STATUS      RESTARTS        AGE
media-server-wtjxk     1/1     Running     1 (5m29s ago)   13m
model-server-9fpb2     1/1     Running     0               89s
nvr-gqlst              1/1     Running     2 (5m29s ago)   11m

部署 OpenVINO AI 应用

  • 部署基于 OpenVINO 推理框架的 AI 应用 pod
    kubectl get pods -n worker161 |grep openvino

    结果

    NAME                                           READY   STATUS    RESTARTS    AGE
edgex-device-openvino-face-recognition-bgch4   1/1     Running   0           5m43s
edgex-device-openvino-object-detection-tbvwk   1/1     Running   0           7m21s

总结

2024-10-08T03:19:07.png

  • 通过本部署方案，可以管理基于 EdgeX 的边缘计算方案，EdgeX 具备边缘自治，即使脱离 K3S 集群，也照样能正常工作。
  • 同时，此方案可以管理成千上万个边缘设备，之间互不干扰，适合大型项目部署。
  • 云端配置相对较为复杂，但是边缘端就轻松很多，只需要手动注册到 K3S 集群，其他都可以通过 K8S API 进行部署和管理。
  • 对比 K8S 而言，K3S 非常适合在边缘端工作，占用资源少，管理较为方便。
  • 其实，最终结果就是：K3S 适合管理边缘容器分发。
  1. 1
  2. 2
  3. 3
  4. 4
  5. ...
  6. 18