如何在K3s中像使用Docker一样使用Containerd

Posted by Ksd on September 30, 2021

自从 Kubernetes 宣布在 v1.20 之后弃用 Docker 作为容器运行时,而改用 containerd 之后,大家对 containerd 的关注度越来越高。近些年 CNCF 社区一直在不断完善 containerd,其定位也发生了改变,由原来的系统嵌入组件,变成了今天的“工业级标准的容器运行时”。

而对于我们习惯使用的 Docker CLI 的用户来说,Containerd 虽然提供的 CLI( ctr 和 crictl ),但设计对人类非常不友好,它无法像 Docker 一样去全生命周期的管理容器。

还好,另外一个命令行工具项目 nerdctl 可供我们选择。nerdctl 是一个与 docker CLI 风格兼容的 containerd 的 CLI 工具,使用体验和 docker 基本一致。 nerdctl 已经作为子项目加入了 containerd 项目,它的 github 地址是 https://github.com/containerd/nerdctl。

Nerdctl 基本涵盖了 docker CLI 的所有功能,同时,它还实现了很多 docker 中不具备的功能,比如:延迟拉取镜像(lazy-pulling)、镜像加密(imgcrypt)等。

K3s 默认都使用的 containerd 作为容器运行时,下文将给大家介绍如何在 K3s 中使用 nerdctl 轻松管理容器。

安装 k3s

root@k3s:~# curl -sfL https://get.k3s.io | sh -
[INFO]  Finding release for channel stable
[INFO]  Using v1.21.5+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.21.5+k3s1/sha256sum-amd64.txt
[INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.21.5+k3s1/k3s
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Creating /usr/local/bin/ctr symlink to k3s
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO]  systemd: Starting k3s

安装并配置 Nerdctl

从 nerdctl 的 release(https://github.com/containerd/nerdctl/releases) 中下载二进制文件,然后将 nerdctl 移动到/usr/local/bin 下即可完成安装。

因为上一步安装的 K3s 中已经包含了 containerd,所以只需要下载nerdctl-<VERSION>-<OS>-<ARCH>.tar.gz 即可,否则需要安装nerdctl-full-<VERSION>-<OS>-<ARCH>.tar.gz

如果要使用 nerdctl 管理 K3s 环境中的容器,还需要手动指定 containerd socket

root@k3s:~# export CONTAINERD_ADDRESS="unix:///run/k3s/containerd/containerd.sock"

接下来,就可以使用 nerdctl 来查询 K3s 中的容器了:

root@k3s:~# nerdctl -n k8s.io ps
CONTAINER ID    IMAGE                                                                                                               COMMAND                   CREATED           STATUS    PORTS    NAMES
19a2751ecaf2    docker.io/rancher/pause:3.1                                                                                         "/pause"                  24 minutes ago    Up
1fb15a152a65    docker.io/rancher/coredns-coredns:1.8.3                                                                             "/coredns -conf /etc…"    24 minutes ago    Up
3005a774e1c1    docker.io/rancher/library-traefik:2.4.8                                                                             "/entrypoint.sh --gl…"    23 minutes ago    Up
364c9f6a7a5a    docker.io/rancher/pause:3.1                                                                                         "/pause"                  24 minutes ago    Up
46aac7428aec    docker.io/rancher/klipper-lb:v0.2.0                                                                                 "entry"                   23 minutes ago    Up
6442944d5514    docker.io/rancher/pause:3.1                                                                                         "/pause"                  23 minutes ago    Up
6f2a5e9a955c    docker.io/rancher/local-path-provisioner@sha256:9666b1635fec95d4e2251661e135c90678b8f45fd0f8324c55db99c80e2a958c    "local-path-provisio…"    24 minutes ago    Up
758b0400700f    docker.io/rancher/klipper-lb:v0.2.0                                                                                 "entry"                   23 minutes ago    Up
808fdd380c8b    docker.io/rancher/pause:3.1                                                                                         "/pause"                  24 minutes ago    Up
8421cc3f260d    docker.io/rancher/pause:3.1                                                                                         "/pause"                  23 minutes ago    Up
c16c1a7560e8    docker.io/rancher/metrics-server:v0.3.6                                                                             "/metrics-server"         24 minutes ago    Up
root@k3s:~#

通过 nerdctl 创建容器

如果要使用 nerdctl 创建容器,需要提前配置 CNI Plugins:

root@k3s:~# mkdir -p /opt/cni/bin
root@k3s:~# wget -c https://github.com/containernetworking/plugins/releases/download/v1.0.1/cni-plugins-linux-amd64-v1.0.1.tgz  -O - |  tar -xz -C /opt/cni/bin/

使用 nerdctl 运行容器:

root@k3s:~# nerdctl run -d --name nginx -p 8000:80 nginx:alpine
docker.io/library/nginx:alpine:                                                   resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:686aac2769fd6e7bab67663fd38750c135b72d993d0bb0a942ab02ef647fc9c3:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:af466e4f12e3abe41fcfb59ca0573a3a5c640573b389d5287207a49d1324abd8: done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:513f9a9d8748b25cdb0ec6f16b4523af7bba216a6bf0f43f70af75b4cf7cb780:   done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 4.8 s                                                                    total:  3.1 Ki (669.0 B/s)
4dbc8925dcb69082c9a4c9959853280a9154d2b42cbbf4ce3bdb846b955d34c1

查看创建的容器:

root@k3s:~# nerdctl ps
CONTAINER ID    IMAGE                             COMMAND                   CREATED               STATUS    PORTS                   NAMES
4dbc8925dcb6    docker.io/library/nginx:alpine    "/docker-entrypoint.…"    About a minute ago    Up        0.0.0.0:8000->80/tcp    nginx

Build 镜像

Nerdctl 构建镜像需要结合 buildkit (https://github.com/moby/buildkit),所以,我们需要先安装并启动 buildkit:

root@k3s:~# wget -c https://github.com/moby/buildkit/releases/download/v0.9.0/buildkit-v0.9.0.linux-amd64.tar.gz -O - |  tar -xz -C /usr/local/
root@k3s:~# buildkitd --containerd-worker-addr="/run/k3s/containerd/containerd.sock" --oci-worker=false --containerd-worker=true &

接下来,我们就可以使用 nerdctl 构建容器镜像:

root@k3s:~# nerdctl build -t "nginx:t1" .
[+] Building 7.6s (4/6)
[+] Building 7.9s (4/6)
[+] Building 8.2s (4/6)
[+] Building 8.4s (4/6)
[+] Building 9.1s (4/6)
[+] Building 9.5s (4/6)
[+] Building 14.6s (7/7) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                         0.1s
 => => transferring dockerfile: 93B                                                                                                                                          0.0s
 => [internal] load .dockerignore                                                                                                                                            0.0s
 => => transferring context: 2B                                                                                                                                              0.0s
 => [internal] load metadata for docker.io/library/nginx:alpine                                                                                                              6.4s
 => [internal] load build context                                                                                                                                            0.1s
 => => transferring context: 652B                                                                                                                                            0.0s
 => [1/2] FROM docker.io/library/nginx:alpine@sha256:686aac2769fd6e7bab67663fd38750c135b72d993d0bb0a942ab02ef647fc9c3                                                        4.2s
 => => resolve docker.io/library/nginx:alpine@sha256:686aac2769fd6e7bab67663fd38750c135b72d993d0bb0a942ab02ef647fc9c3                                                        0.1s
 => => sha256:61074acc7dd227cfbeaf719f9b5cdfb64711bc6b60b3865c7b886b7099c15d15 0B / 1.39kB                                                                                   7.3s
 => => sha256:969825a5ca61c8320c63ff9ce0e8b24b83442503d79c5940ba4e2f0bd9e34df8 0B / 663B                                                                                     7.3s
 => => sha256:3e72c40d0ff43c52c5cc37713b75053e8cb5baea8e137a784d480123814982a2 0B / 891B                                                                                     7.2s
 => => sha256:c1368e94e1ec563b31c3fb1fea02c9fbdc4c79a95e9ad0cac6df29c228ee2df3 0B / 602B                                                                                     7.3s
 => => sha256:a0d0a0d46f8b52473982a3c466318f479767577551a53ffc9074c9fa7035982e 2.81MB / 2.81MB                                                                               6.9s
 => => sha256:4dd4efe90939ab5711aaf5fcd9fd8feb34307bab48ba93030e8b845f8312ed8e 6.29MB / 7.15MB                                                                               6.7s
 => => extracting sha256:a0d0a0d46f8b52473982a3c466318f479767577551a53ffc9074c9fa7035982e                                                                                    0.7s
 => => extracting sha256:4dd4efe90939ab5711aaf5fcd9fd8feb34307bab48ba93030e8b845f8312ed8e                                                                                    1.4s
 => => extracting sha256:c1368e94e1ec563b31c3fb1fea02c9fbdc4c79a95e9ad0cac6df29c228ee2df3                                                                                    0.0s
 => => extracting sha256:3e72c40d0ff43c52c5cc37713b75053e8cb5baea8e137a784d480123814982a2                                                                                    0.0s
 => => extracting sha256:969825a5ca61c8320c63ff9ce0e8b24b83442503d79c5940ba4e2f0bd9e34df8                                                                                    0.0s
 => => extracting sha256:61074acc7dd227cfbeaf719f9b5cdfb64711bc6b60b3865c7b886b7099c15d15                                                                                    0.0s
 => [2/2] ADD index.html /usr/share/nginx/html/                                                                                                                              0.3s
 => exporting to oci image format                                                                                                                                            2.8s
 => => exporting layers                                                                                                                                                      0.4s
 => => exporting manifest sha256:b9bd561041b26c433f0d556c9b14496a543746cc6ffceaa5d86efcaae8fd60e3                                                                            0.0s
 => => exporting config sha256:016a6b9885092667ab5418dbad43f3d1ac35056452431fe8d1a4b0e895ad76f3                                                                              0.0s
 => => sending tarball                                                                                                                                                       2.4s
unpacking docker.io/library/nginx:t1 (sha256:b9bd561041b26c433f0d556c9b14496a543746cc6ffceaa5d86efcaae8fd60e3)...done

查看构建镜像:

root@k3s:~# nerdctl images
REPOSITORY    TAG       IMAGE ID        CREATED              SIZE
nginx         alpine    686aac2769fd    About an hour ago    26.0 MiB
nginx         t1        b9bd561041b2    2 minutes ago        26.0 MiB

总结

实际上,nerdctl 的使用方式和 docker cli 几乎一致,我们也很容易的从 docker 过渡到 nerdctl+containerd。 如果你是因为习惯了 docker cli 才在 K3s 集群中使用 docker 容器运行时的话,那么现在借住 nerdctl 管理 containerd 也许是更好的选择。

本文只是给大家 demo 了如何在 K3s 中借住 nerdctl 管理本地容器,如果你使用的是 RKE2 集群,你也可以参考本文的说明来在 RKE2 集群中使用 nerdctl 来管理 containerd。