1. 项目概述在云计算和微服务架构盛行的当下Kubernetes作为容器编排的事实标准已经成为运维和开发人员的必备技能。本文将详细介绍在Ubuntu 22.04系统上搭建一个3节点Kubernetes集群的全过程包含从虚拟机配置到集群网络部署的每个关键步骤。2. 环境准备2.1 虚拟机基础配置首先需要准备3台Ubuntu 22.04虚拟机分别作为master节点和两个worker节点。建议每台虚拟机至少配置2核CPU和4GB内存存储空间不少于20GB。网络配置采用桥接模式确保各节点间可以互相通信。为每个节点分配静态IP地址Master节点192.168.31.200Node1节点192.168.31.201Node2节点192.168.31.202设置主机名命令sudo hostnamectl set-hostname master # 在master节点执行 sudo hostnamectl set-hostname node1 # 在node1节点执行 sudo hostnamectl set-hostname node2 # 在node2节点执行2.2 系统基础设置在所有节点上更新软件包并安装必要工具sudo apt update sudo apt upgrade -y sudo apt install -y curl gnupg2 software-properties-common apt-transport-https ca-certificates net-tools配置hosts文件确保节点间可以通过主机名互相访问sudo tee -a /etc/hosts EOF 192.168.31.200 master 192.168.31.201 node1 192.168.31.202 node2 EOF3. Kubernetes前置配置3.1 禁用SwapKubernetes要求必须禁用swap分区sudo swapoff -a sudo sed -i / swap / s/^\(.*\)$/#\1/g /etc/fstab3.2 内核参数调整加载必要的内核模块并配置网络参数sudo tee /etc/modules-load.d/k8s.conf EOF overlay br_netfilter EOF sudo modprobe overlay sudo modprobe br_netfilter sudo tee /etc/sysctl.d/k8s.conf EOF net.bridge.bridge-nf-call-ip6tables 1 net.bridge.bridge-nf-call-iptables 1 net.ipv4.ip_forward 1 EOF sudo sysctl --system4. 容器运行时安装4.1 Containerd安装Kubernetes 1.24版本默认使用containerd作为容器运行时sudo apt install -y containerd sudo mkdir -p /etc/containerd containerd config default | sudo tee /etc/containerd/config.toml sudo sed -i s/SystemdCgroup false/SystemdCgroup true/g /etc/containerd/config.toml sudo systemctl restart containerd sudo systemctl enable containerd5. Kubernetes组件安装5.1 添加Kubernetes仓库sudo curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-archive-keyring.gpg echo deb [signed-by/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main | sudo tee /etc/apt/sources.list.d/kubernetes.list5.2 安装kubeadm、kubelet和kubectlsudo apt update sudo apt install -y kubelet1.28.0-00 kubeadm1.28.0-00 kubectl1.28.0-00 sudo apt-mark hold kubelet kubeadm kubectl6. 集群初始化6.1 Master节点初始化在master节点执行sudo kubeadm init --control-plane-endpoint192.168.31.200 --pod-network-cidr192.168.0.0/16初始化完成后按照提示配置kubectlmkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config6.2 Worker节点加入在worker节点执行master初始化时输出的join命令sudo kubeadm join 192.168.31.200:6443 --token token --discovery-token-ca-cert-hash hash7. 网络插件安装7.1 安装Calico网络插件kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/calico.yaml验证网络插件状态kubectl get pods -n kube-system8. 集群验证8.1 基本功能测试部署测试应用kubectl create deployment nginx --imagenginx kubectl expose deployment nginx --port80 --typeNodePort kubectl get svc nginx8.2 集群状态检查kubectl get nodes kubectl get pods -A kubectl cluster-info9. 常见问题排查9.1 节点NotReady状态可能原因网络插件未正确安装容器运行时异常节点资源不足解决方案journalctl -u kubelet -f # 查看kubelet日志 systemctl status containerd # 检查容器运行时状态9.2 Pod卡在Pending状态可能原因资源不足节点选择器不匹配PV/PVC问题诊断命令kubectl describe pod pod-name kubectl get events --sort-by.metadata.creationTimestamp10. 生产环境建议使用高可用控制平面配置集群自动扩缩容设置合理的资源限制和请求启用RBAC和网络策略定期备份etcd数据对于生产环境建议使用kubeadm的高可用配置kubeadm init --control-plane-endpoint LOAD_BALANCER_DNS:LOAD_BALANCER_PORT --upload-certs11. 性能优化技巧调整kubelet参数--max-pods100 --kube-api-qps50 --kube-api-burst100优化容器运行时配置[plugins.io.containerd.grpc.v1.cri.containerd] snapshotter overlayfs disable_snapshot_annotations true网络性能调优sysctl -w net.core.somaxconn32768 sysctl -w net.ipv4.tcp_tw_reuse112. 安全加固措施启用Pod安全准入apiVersion: apiserver.config.k8s.io/v1 kind: AdmissionConfiguration plugins: - name: PodSecurity configuration: apiVersion: pod-security.admission.config.k8s.io/v1 kind: PodSecurityConfiguration defaults: enforce: restricted定期轮换证书kubeadm certs renew all启用审计日志apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata13. 监控与日志部署Prometheus监控helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm install prometheus prometheus-community/kube-prometheus-stack配置日志收集helm repo add elastic https://helm.elastic.co helm install elasticsearch elastic/elasticsearch helm install filebeat elastic/filebeat14. 持续集成方案GitOps工作流配置helm repo add argo https://argoproj.github.io/argo-helm helm install argocd argo/argo-cdTekton流水线示例apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: build-and-deploy spec: tasks: - name: build taskRef: name: build-push - name: deploy taskRef: name: deploy-k8s15. 存储方案选型本地存储配置apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: local-storage provisioner: kubernetes.io/no-provisioner volumeBindingMode: WaitForFirstConsumer分布式存储集成helm repo add rook-release https://charts.rook.io/release helm install rook-ceph rook-release/rook-ceph16. 服务网格集成Istio安装curl -L https://istio.io/downloadIstio | sh - cd istio-1.17.2 ./bin/istioctl install --set profiledemo -yLinkerd配置curl -sL https://run.linkerd.io/install | sh linkerd install | kubectl apply -f -17. 多集群管理Cluster API部署clusterctl init --infrastructureawsKubefed配置helm repo add kubefed-charts https://raw.githubusercontent.com/kubernetes-sigs/kubefed/master/charts helm install kubefed kubefed-charts/kubefed --version0.9.1 --namespace kube-federation-system18. 自定义扩展开发Operator SDK初始化operator-sdk init --domainexample.com --repogithub.com/example/memcached-operator编写自定义控制器func (r *MemcachedReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { // 业务逻辑实现 }19. 备份与恢复etcd备份ETCDCTL_API3 etcdctl --endpointshttps://127.0.0.1:2379 \ --cacert/etc/kubernetes/pki/etcd/ca.crt \ --cert/etc/kubernetes/pki/etcd/server.crt \ --key/etc/kubernetes/pki/etcd/server.key \ snapshot save snapshot.dbVelero灾难恢复velero install \ --provider aws \ --plugins velero/velero-plugin-for-aws:v1.0.0 \ --bucket velero-backups \ --secret-file ./credentials-velero \ --use-volume-snapshotsfalse20. 升级与维护集群升级计划kubeadm upgrade plan kubeadm upgrade apply v1.28.1节点维护模式kubectl drain node-name --ignore-daemonsets kubectl uncordon node-name21. 资源优化实践垂直Pod自动扩缩apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: name: my-app-vpa spec: targetRef: apiVersion: apps/v1 kind: Deployment name: my-app updatePolicy: updateMode: Auto水平Pod自动扩缩kubectl autoscale deployment my-app --cpu-percent50 --min1 --max1022. 网络策略实施默认拒绝所有流量apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-all spec: podSelector: {} policyTypes: - Ingress - Egress按需开放访问apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-frontend-to-backend spec: podSelector: matchLabels: app: backend ingress: - from: - podSelector: matchLabels: app: frontend23. 服务暴露方式Ingress控制器helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm install ingress-nginx ingress-nginx/ingress-nginxLoadBalancer服务apiVersion: v1 kind: Service metadata: name: my-service spec: selector: app: MyApp ports: - protocol: TCP port: 80 targetPort: 9376 type: LoadBalancer24. 配置管理方案ConfigMap使用apiVersion: v1 kind: ConfigMap metadata: name: game-config data: game.properties: | enemy.typesaliens,monsters player.maximum-lives5Secret管理kubectl create secret generic db-secret \ --from-literalusernameadmin \ --from-literalpasswordS!B\*d$zDsb25. 最佳实践总结基础设施即代码使用Terraform或Pulumi管理集群基础设施GitOps工作流采用Argo CD或Flux实现声明式部署可观测性集成Prometheus、Grafana和Loki监控栈安全防护定期扫描镜像漏洞实施网络策略成本优化使用集群自动扩缩和资源配额对于长期运行的集群建议建立完善的备份策略和灾难恢复计划同时定期进行版本升级和安全补丁更新。