Shawn's Blog

蒸汽兔

初探k8s-阿里云安装k8s公网ip部署(一)

字数：889 字 阅读时长：约 4 分钟 阅读次

背景：

一般情况下，”kubeadm”部署集群时指定”–apiserver-advertise-address="参数，即可在其他机器上，通过公网ip join到本机器，然而，阿里云ecs里没配置公网ip，etcd会无法启动，导致初始化失败。

环境：阿里云按量付费ECS ,专用网络，kubeadm版本1.18.3，kubectl版本1.18.3, kubelet版本1.18.3，ubuntu18.03

解决

kubeadm初始化的过称中对etcd配置文件/etc/kubernetes/manifests/etcd.yaml进行修改，因此，需要建立两个ssh对话，即用ssh工具新建两个标签，一个用来初始化节点，另一个在初始化过程中修改配置文件。注意是初始化过程中，每次运行kubeadm init，kubeadm都会生成etcd的配置文件，如果提前修改了配置文件，在运行kubeadm init时会把修改的结果覆盖，那么也就没有作用了。

步骤

前提

安装好kubeadm，kubectl, kubelet三项k8s必备工具

# 安装系统工具
apt-get update && apt-get install -y apt-transport-https

# 安装 GPG 证书
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -

# 写入软件源；注意：我们用系统代号为 bionic，但目前阿里云不支持，所以沿用 16.04 的 xenial
cat << EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF

# 安装
apt-get update && apt-get install -y kubelet kubeadm kubectl

并配置主机名

# 修改主机名
hostnamectl set-hostname kubernetes-master

# 配置 hosts xx.xx.xx.xx为你的公网ip地址
cat >> /etc/hosts << EOF
xx.xx.xx.xx kubernetes-master
EOF

拉取并修改配置文件

# 导出配置文件
kubeadm config print init-defaults --kubeconfig ClusterConfiguration > kubeadm.yml

apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  # 修改为主节点ip xx.xx.xx.xx为你的公网ip地址
  advertiseAddress: xx.xx.xx.xx
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: kubernetes-master
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
# 国内不能访问 Google，修改为阿里云
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.18.3
networking:
  dnsDomain: cluster.local
  # 配置 POD 所在网段为我们虚拟机不重叠的网段（这里用的是 Flannel 默认网段）
  podSubnet: "10.244.0.0/16"
  serviceSubnet: 10.96.0.0/12
scheduler: {}

查看所需镜像

kubeadm config images list --config kubeadm.yml

# 输出如下
W0620 19:26:13.112999   14321 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
registry.aliyuncs.com/google_containers/kube-apiserver:v1.18.3
registry.aliyuncs.com/google_containers/kube-controller-manager:v1.18.3
registry.aliyuncs.com/google_containers/kube-scheduler:v1.18.3
registry.aliyuncs.com/google_containers/kube-proxy:v1.18.3
registry.aliyuncs.com/google_containers/pause:3.2
registry.aliyuncs.com/google_containers/etcd:3.4.3-0
registry.aliyuncs.com/google_containers/coredns:1.6.7

拉取所需镜像

kubeadm config images pull --config kubeadm.yml

# 输出如下
W0620 19:25:33.889929   14265 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.18.3
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.18.3
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.18.3
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.18.3
[config/images] Pulled registry.aliyuncs.com/google_containers/pause:3.2
[config/images] Pulled registry.aliyuncs.com/google_containers/etcd:3.4.3-0
[config/images] Pulled registry.aliyuncs.com/google_containers/coredns:1.6.7

安装

执行以下命令初始化主节点，该命令指定了初始化时需要使用的配置文件，其中添加 --upload-certs 参数可以在后续执行加入节点时自动分发证书文件。追加的 tee kubeadm-init.log 用以输出日志。

注意： 如果安装 kubernetes 版本和下载的镜像版本不统一则会出现 timed out waiting for the condition 错误。中途失败或是想修改配置可以使用 kubeadm reset 命令重置配置，再做初始化操作即可。

kubeadm init --config=kubeadm.yml --upload-certs | tee kubeadm-init.log

过程会卡在etcd过程，日志如下：

[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.

修改配置

切换ssh会话，修改etcd配置文件

 vi /etc/kubernetes/manifests/etcd.yaml

修改前配置

containers:
  - command:
    - etcd
    - --advertise-client-urls=https://139.196.225.241:2379
    - --cert-file=/etc/kubernetes/pki/etcd/server.crt
    - --client-cert-auth=true
    - --data-dir=/var/lib/etcd
    - --initial-advertise-peer-urls=https://139.196.225.241:2380
    - --initial-cluster=kubernetes-master=https://139.196.225.241:2380
    - --key-file=/etc/kubernetes/pki/etcd/server.key
    - --listen-client-urls=https://127.0.0.1:2379,https://xx.xx.xx.xx:2379
    - --listen-metrics-urls=http://127.0.0.1:2381
    - --listen-peer-urls=https://xx.xx.xx.xx:2380
    - --name=kubernetes-master
    - --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt
    - --peer-client-cert-auth=true
    - --peer-key-file=/etc/kubernetes/pki/etcd/peer.key
    - --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
    - --snapshot-count=10000
    - --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt

etcd参数配置说明

placeholder

需要把–listen-client-urls 和 –listen-peer-urls 地址ip都修改为0.0.0.0

修改后

注意： 需要在安装过程中修改完成，如初始化过程已结束，修改则无效, 可以使用kubeadm reset 命令重置配置。重新尝试安装

按理应该可以刷新etcd配置来同样安装成功，但是博主就没做尝试研究了~~

containers:
  - command:
    - etcd
    - --advertise-client-urls=https://139.196.225.241:2379
    - --cert-file=/etc/kubernetes/pki/etcd/server.crt
    - --client-cert-auth=true
    - --data-dir=/var/lib/etcd
    - --initial-advertise-peer-urls=https://139.196.225.241:2380
    - --initial-cluster=kubernetes-master=https://139.196.225.241:2380
    - --key-file=/etc/kubernetes/pki/etcd/server.key
    # 修改地址为0.0.0.0
    - --listen-client-urls=https://0.0.0.0:2379
    - --listen-metrics-urls=http://127.0.0.1:2381
    # 修改地址为0.0.0.0
    - --listen-peer-urls=https://0.0.0.0:2380
    - --name=kubernetes-master
    - --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt
    - --peer-client-cert-auth=true
    - --peer-key-file=/etc/kubernetes/pki/etcd/peer.key
    - --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
    - --snapshot-count=10000
    - --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt

配置修改完成之后就会提示安装成功

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join xx.xx.xx.xx:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:4768bc9e67cd0b8ae2fa5e63f9d70c7e43352cd4a5325430a45ba0567b45e3as

参考博客

http://www.qfdmy.com/#/courses/lesson/1240365416028405762/1253444164947808257

https://zhuanlan.zhihu.com/p/74134318

https://zhuanlan.zhihu.com/p/75834420

© Shawn Jim. All rights reserved. 本站总访问量 次, 访客数 人次.