待续
Hits: 4257
月度归档： 2023 年 4 月
K8S安装手册 v1.18.0
前情提要，master必须先配置好 native.cgroupdriver=systemd /etc/docker/daemon 否则pod节点可能无法创建沙盘，具体可以查看专题页面
- 关闭防火墙(测试模式下，不关心端口问题)
systemctl stop firewalld && systemctl disable firewalld && firewall-cmd --state
端口开放一览
Master
6443* Kubernetes API server
2379-2380 etcd server client API
10250 kubelet API
10251 Kube-scheduler
10252 Kube-controller-manager
Master
firewall-cmd --add-port=6443/tcp --permanent
firewall-cmd --add-port=10250/tcp --permanent
firewall-cmd --add-port=10251/tcp --permanent
firewall-cmd --add-port=10252/tcp --permanent
firewall-cmd --add-port=2379-2380/tcp --permanent
firewall-cmd --add-port=30000/tcp --permanent
firewall-cmd --reload
firewall-cmd --list-ports
Worker node
10250 kubelet API
30000-32767 NodePort Services†
Node
firewall-cmd --add-port=10250/tcp --permanent
firewall-cmd --add-port=30000-32767/tcp --permanent
firewall-cmd --reload
firewall-cmd --list-ports
- 虚拟机分配
MASTER 192.168.236.160
NODE1 192.168.236.128
NODE2 192.168.236.129
- 关闭SWAP
vi /etc/fstab
#/dev/mapper/centos-swap swap swap defaults 0 0
- 关闭SELINUX
cat /etc/selinux/config
vi /etc/selinux/config
SELINUX=disabled
reboot
- 导入源
rm -rfv /etc/yum.repos.d/*
curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
- 分别设置主机名
hostnamectl set-hostname master1
hostnamectl set-hostname node1
hostnamectl set-hostname node2
more /etc/hostname
Master
- 配置内核参数
cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
理论上每台服务器都需要配置
- 安装常用包
yum install vim bash-completion net-tools gcc -y
- 使用Docker源
yum install -y yum-utils device-mapper-persistent-data lvm2 && yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo && yum -y install docker-ce
或已经配置好docker源（上面不适用于Redhat）
yum install -y yum-utils device-mapper-persistent-data lvm2 && yum -y install docker-ce
- 添加Docker加速器
如需使用内部镜像和缓存请参考其他教程
mkdir -p /etc/docker
tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors":[
"https://fl791z1h.mirror.aliyuncs.com"
],
"insecure-registries":[
"https://harbor.thefunc.com",
"https://192.168.199.179"
],
"exec-opts":[
"native.cgroupdriver=systemd"
]
}
EOF
systemctl daemon-reload && systemctl restart docker && systemctl enable docker
- 装kubectl、kubelet、kubeadm
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
yum -y install kubectl-1.18.0 kubelet-1.18.0 kubeadm-1.18.0
systemctl enable kubelet
12 初始化集群 k8s ，后面章节区分执行节点
Master 节点
初始化集群
kubeadm init --kubernetes-version=1.18.0 \
--apiserver-advertise-address=192.168.236.160 \
--image-repository registry.aliyuncs.com/google_containers \
--service-cidr=10.10.0.0/16 --pod-network-cidr=10.122.0.0/16
Node 执行后记录最后的信息，用于NODE节点执行加入到网络（如果已经加入过重新加入集群的话需要删除配置文件，看下方专题）
kubeadm join 192.168.236.160:6443 --token wiw0iu.2yctn4v9k7dgzugg \
--discovery-token-ca-cert-hash sha256:ec5de369ce10177019e1f37715dd92f011914b6aaa026417efac584ca1f12538
注：
1.kubeadm init后得到的token有效期为24小时，过期后需要重新创建token,
执行：
kubeadm token create
获取新token
2.kubeadm token list 查看token列表，
Master 节点
创建kubectl
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
注：
1.不配置$HOME/.kube/config的话，kubectl命令不可用，
2.node节点写法有点不一样，node节点的这行为：sudo cp -i /etc/kubernetes/kubelet.conf $HOME/.kube/config
NODE 节点
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/kubelet.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
- 安装calico网络（master节点）
这行可能过时
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
可以使用这行
kubectl apply -f https://docs.projectcalico.org/v3.14/manifests/calico.yaml
又或者
kubectl apply -f http://server.thefunc.com/download/k8s/calico/v3.14/manifests/calico.yaml
注：
k8s 18.0 对应的calico版本可以是3.14版本
安装calico网络网络后过一会再输入kubectl get node，可以看到节点的STATUS由NotReady变为Ready
新的方法
cd ~
wget https://docs.projectcalico.org/v3.14/manifests/calico.yaml
kubectl apply -f calico.yaml
NODE 节点
kubeadm join 192.168.236.160:6443 --token wiw0iu.2yctn4v9k7dgzugg \
--discovery-token-ca-cert-hash sha256:ec5de369ce10177019e1f37715dd92f011914b6aaa026417efac584ca1f12538
注：
1.kubeadm init后得到的token有效期为24小时，过期后需要重新创建token,执行：kubeadm token create获取新token
2.kubeadm token list 查看token列表，
只要重新加入了集群都必须重新配置这个脚本，否则访问到历史的节点
rm -rf $HOME/.kube
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/kubelet.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
到了这步，把所有的node节点加入mater节点后，k8s的环境已经安装完成了
[root@localhost ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
master1 Ready master 8m57s v1.18.0
node1 Ready <none> 5m24s v1.18.0
node2 Ready <none> 5m23s v1.18.0
Hits: 4323
在Gitlab中使用Harbor
提示，如果是直接在docker中运行的gitlab-runner需要在docker配置中将域名添加信任。如果是在k8s中运行的gitlab-runner，需要在ci文件中添加信任配置
在.gitlab-ci.yml文件中使用
构建镜像过程中添加如下代码可推送镜像至仓库
- docker build -t $PROJECT .
- docker login -u 'robot$gitlab' -p 'SeQh123vpuJo3iHz' harbor.thefunc.com
- docker tag chatgptwebapi:latest harbor.thefunc.com/chatgpt/chatgptwebapi:latest
- docker push harbor.thefunc.com/chatgpt/chatgptwebapi:latest
- docker logout harbor.thefunc.com
Hits: 4365
Harbor简化版使用
机器人账户
登录
docker login -u 'robot$gitlab' -p 'SeQh123tBCP7ubmMvpuJo3iHz' harbor.thefunc.com
拉取
docker pull harbor.thefunc.com/agilebpm/bpmtest@sha256:f54a58bc1aac5ea1a25d796ae155dc228b3f0e11d046ae276b39c4bf2f13d8c4
推送
docker tag SOURCE_IMAGE[:TAG] harbor.thefunc.com/agilebpm/REPOSITORY[:TAG]
docker push harbor.thefunc.com/agilebpm/REPOSITORY[:TAG]
样例
docker tag hello-world:latest harbor.thefunc.com/agilebpm/bpmtest:v1.0
docker push harbor.thefunc.com/agilebpm/bpmtest:v1.0
登出
docker logout harbor.thefunc.com
Hits: 4229
Harbor跟随系统启动
创建启动文件
cd /etc/systemd/system
vi harbor.service
下列中的 /usr/local/harbor 为harbor所在目录
/usr/bin/docker-compose 为 docker-compose 程序文件，路径可通过命令查找 find / -name docker-compose
[Unit]
Description=harbor
After=docker.service systemd-networkd.service systemd-resolved.service
Requires=docker.service
Documentation=http://github.com/vmware/harbor
[Service]
Type=simple
Restart=on-failure
RestartSec=5
ExecStart=/usr/bin/docker-compose -f /srv/harbor/docker-compose.yml up
ExecStop=/usr/bin/docker-compose -f /srv/harbor/docker-compose.yml down
[Install]
WantedBy=multi-user.target
systemctl enable harbor
systemctl restart harbor
Hits: 4141
Harbor配置维护
例如修改Hostname
从主机名变更为域名以便于外部使用
先停止服务
docker-compose down -v
cd /srv/harbor
修改harbor.yml文件
hostname: harbor.thefunc.com
重新启动部署（如果不生效可能需要先执行./prepare）
./install.sh
再次启动服务
docker-compose up -d
Hits: 4212
Harbor安装
首先关闭SELINUX
其次开放http/https防火墙协议
yum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin docker-compose
systemctl enable docker && systemctl start docker
ll -t
自行官网下载
wget http://server.thefunc.com/download/harbor-offline-installer-v2.7.1.tgz
tar -xvzf harbor-offline-installer-v2.7.1.tgz -C /srv
cd /srv/harbor
ls
docker load -i harbor.v2.7.1.tar.gz
cp harbor.yml.tmpl harbor.yml
harbor.yml 修改hostname为指定服务名（服务名为外部客户端访问所能访问的地址）
hostname
修改hostname为当前主机名
vi harbor.yml
hostname: harbor.thefunc.com
harbor_admin_password: 666666
# Harbor DB configuration
database:
# The password for the root user of Harbor DB. Change this before any production use.
password: 666666
SSL配置
https://goharbor.io/docs/1.10/install-config/configure-https/
mkdir -p /srv/harbor/cert && cd /srv/harbor/cert
openssl genrsa -out ca.key 4096
openssl req -x509 -new -nodes -sha512 -days 3650 \
-subj "/C=CN/ST=GD/L=GZ/O=GGEC/OU=IT/CN=harbor.thefunc.com" \
-key ca.key \
-out ca.crt
openssl genrsa -out harbor.thefunc.com.key 4096
openssl req -sha512 -new \
-subj "/C=CN/ST=GD/L=GZ/O=GGEC/OU=IT/CN=harbor.thefunc.com" \
-key harbor.thefunc.com.key \
-out harbor.thefunc.com.csr
cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1=harbor.thefunc.com
DNS.2=192.168.199.179
DNS.3=REDHAT7-Docker-Harbor-ImageService
EOF
openssl x509 -req -sha512 -days 3650 \
-extfile v3.ext \
-CA ca.crt -CAkey ca.key -CAcreateserial \
-in harbor.thefunc.com.csr \
-out harbor.thefunc.com.crt
openssl x509 -inform PEM -in harbor.thefunc.com.crt -out harbor.thefunc.com.cert
mkdir -p /etc/docker/certs.d/harbor.thefunc.com
cp harbor.thefunc.com.cert /etc/docker/certs.d/harbor.thefunc.com/
cp harbor.thefunc.com.key /etc/docker/certs.d/harbor.thefunc.com/
cp ca.crt /etc/docker/certs.d/harbor.thefunc.com/
systemctl restart docker
修改harbor配置
# https related config
https:
# https port for harbor, default is 443
port: 443
# The path of cert and key files for nginx
certificate: /srv/harbor/cert/harbor.thefunc.com.crt
private_key: /srv/harbor/cert/harbor.thefunc.com.key
安装
./prepare
./install.sh
打开网址访问
https://harbor.thefunc.com
超级密码 666666
Hits: 4201