티스토리 뷰
[Container Management] Kubernetes Woker Node Install & Setting
GodNR 2019. 8. 4. 08:29이번 포스팅에서는 Kubernetes의 Pod들이 배치될 Worker 노드를 설치하고 Master Node와 연결하는 과정을 살펴보겠습니다.
지난 Kubernetes 가이드는 아래를 참고하세요.
[Container Management] Kubernetes Master Node 설치
[Container Management] Kubernetes Dashboard Install & Setting
Kubernetes Worker Node 설치
1. firewall-cmd를 활용하여 오픈할 방화벽 port를 정의합니다.
firewall-cmd --zone=public --permanent --add-port=10250/tcp
firewall-cmd --zone=public --permanent --add-port=30000-32767/tcp
firewall-cmd --reload
Worker Node에서 사용하는 포트는 다음과 같습니다.
|
Protocol |
Direction |
Port Range |
Purpose |
Used By |
|
TCP |
Inbound |
10250 |
Kubelet API |
Self, Control plane |
|
TCP |
Inbound |
30000 - 32767 |
NodePort Services** |
All |
2. Docker 및 Kubernetes 설치
[root@kubeworker ~]# cat /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kube*
[root@kubeworker ~]#kubernetes repository를 추가하여 kubeadm, kubectl, kubelet을 설치합니다.
[root@kubeworker ~]# yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirror.navercorp.com
* extras: mirror.navercorp.com
* updates: mirror.navercorp.com
Resolving Dependencies
--> Running transaction check
---> Package kubeadm.x86_64 0:1.15.1-0 will be installed
--> Processing Dependency: kubernetes-cni >= 0.7.5 for package: kubeadm-1.15.1-0.x86_64
--> Processing Dependency: cri-tools >= 1.13.0 for package: kubeadm-1.15.1-0.x86_64
---> Package kubectl.x86_64 0:1.15.1-0 will be installed
---> Package kubelet.x86_64 0:1.15.1-0 will be installed
--> Processing Dependency: socat for package: kubelet-1.15.1-0.x86_64
--> Processing Dependency: conntrack for package: kubelet-1.15.1-0.x86_64
--> Running transaction check
---> Package conntrack-tools.x86_64 0:1.4.4-4.el7 will be installed
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.1)(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.0)(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0(LIBNETFILTER_CTHELPER_1.0)(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64
--> Processing Dependency: libnetfilter_queue.so.1()(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1()(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0()(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Running transaction check
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-9.el7 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================================================================================================================
Installing:
kubeadm x86_64 1.15.1-0 kubernetes 8.9 M
kubectl x86_64 1.15.1-0 kubernetes 9.5 M
kubelet x86_64 1.15.1-0 kubernetes 22 M
Installing for dependencies:
conntrack-tools x86_64 1.4.4-4.el7 base 186 k
cri-tools x86_64 1.13.0-0 kubernetes 5.1 M
kubernetes-cni x86_64 0.7.5-0 kubernetes 10 M
libnetfilter_cthelper x86_64 1.0.0-9.el7 base 18 k
libnetfilter_cttimeout x86_64 1.0.0-6.el7 base 18 k
libnetfilter_queue x86_64 1.0.2-2.el7_2 base 23 k
socat x86_64 1.7.3.2-2.el7 base 290 k
Transaction Summary
============================================================================================================================================================================================================================================
Install 3 Packages (+7 Dependent packages)
Total download size: 56 M
Installed size: 251 M
Downloading packages:
(1/10): conntrack-tools-1.4.4-4.el7.x86_64.rpm | 186 kB 00:00:00
warning: /var/cache/yum/x86_64/7/kubernetes/packages/14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY 0.0 B/s | 186 kB --:--:-- ETA
Public key for 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm is not installed
(2/10): 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm | 5.1 MB 00:00:01
(3/10): aa386b8f2cac67415283227ccb01dc043d718aec142e32e1a2ba6dbd5173317b-kubeadm-1.15.1-0.x86_64.rpm | 8.9 MB 00:00:02
(4/10): f27b0d7e1770ae83c9fce9ab30a5a7eba4453727cdc53ee96dc4542c8577a464-kubectl-1.15.1-0.x86_64.rpm | 9.5 MB 00:00:01
(5/10): libnetfilter_cthelper-1.0.0-9.el7.x86_64.rpm | 18 kB 00:00:00
(6/10): libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm | 23 kB 00:00:00
(7/10): socat-1.7.3.2-2.el7.x86_64.rpm | 290 kB 00:00:00
(8/10): libnetfilter_cttimeout-1.0.0-6.el7.x86_64.rpm | 18 kB 00:00:00
(9/10): 548a0dcd865c16a50980420ddfa5fbccb8b59621179798e6dc905c9bf8af3b34-kubernetes-cni-0.7.5-0.x86_64.rpm | 10 MB 00:00:01
(10/10): f5edc025972c2d092ac41b05877c89b50cedaa7177978d9e5e49b5a2979dbc85-kubelet-1.15.1-0.x86_64.rpm | 22 MB 00:00:03
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 8.7 MB/s | 56 MB 00:00:06
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
Userid : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>"
Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
From : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Importing GPG key 0x3E1BA8D5:
Userid : "Google Cloud Packages RPM Signing Key <gc-team@google.com>"
Fingerprint: 3749 e1ba 95a8 6ce0 5454 6ed2 f09c 394c 3e1b a8d5
From : https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : socat-1.7.3.2-2.el7.x86_64 1/10
Installing : libnetfilter_cthelper-1.0.0-9.el7.x86_64 2/10
Installing : libnetfilter_queue-1.0.2-2.el7_2.x86_64 3/10
Installing : cri-tools-1.13.0-0.x86_64 4/10
Installing : libnetfilter_cttimeout-1.0.0-6.el7.x86_64 5/10
Installing : conntrack-tools-1.4.4-4.el7.x86_64 6/10
Installing : kubernetes-cni-0.7.5-0.x86_64 7/10
Installing : kubelet-1.15.1-0.x86_64 8/10
Installing : kubectl-1.15.1-0.x86_64 9/10
Installing : kubeadm-1.15.1-0.x86_64 10/10
Verifying : kubectl-1.15.1-0.x86_64 1/10
Verifying : libnetfilter_cttimeout-1.0.0-6.el7.x86_64 2/10
Verifying : cri-tools-1.13.0-0.x86_64 3/10
Verifying : libnetfilter_queue-1.0.2-2.el7_2.x86_64 4/10
Verifying : libnetfilter_cthelper-1.0.0-9.el7.x86_64 5/10
Verifying : kubelet-1.15.1-0.x86_64 6/10
Verifying : kubeadm-1.15.1-0.x86_64 7/10
Verifying : kubernetes-cni-0.7.5-0.x86_64 8/10
Verifying : socat-1.7.3.2-2.el7.x86_64 9/10
Verifying : conntrack-tools-1.4.4-4.el7.x86_64 10/10
Installed:
kubeadm.x86_64 0:1.15.1-0 kubectl.x86_64 0:1.15.1-0 kubelet.x86_64 0:1.15.1-0
Dependency Installed:
conntrack-tools.x86_64 0:1.4.4-4.el7 cri-tools.x86_64 0:1.13.0-0 kubernetes-cni.x86_64 0:0.7.5-0 libnetfilter_cthelper.x86_64 0:1.0.0-9.el7 libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7 libnetfilter_queue.x86_64 0:1.0.2-2.el7_2
socat.x86_64 0:1.7.3.2-2.el7
Complete!
[root@kubeworker ~]#Kubernetes Master Node 연동
1. swapoff -a
[root@kubeworker ~]# swapoff -a
[root@kubeworker ~]#2. kubeadm join
연동 방법은 간단합니다. 기존 Master Node에서 kubeadm init 시 생성된 token 정보를 기반으로 join을 시도합니다.
[root@guruson ~]# kubeadm init
[init] Using Kubernetes version: v1.15.1
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [guruson localhost] and IPs [222.234.124.110 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [guruson localhost] and IPs [222.234.124.110 127.0.0.1 ::1]
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [guruson kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 222.234.124.110]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[apiclient] All control plane components are healthy after 45.506400 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.15" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node guruson as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node guruson as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: deb19a.7yfa212rg0exg0c9
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 222.234.124.110:6443 --token deb19a.7yfa212rg0exg0c9 \
--discovery-token-ca-cert-hash sha256:20d38dd05c158fe88fecd1b219ba9a5e02e5ea66ad612b404678571d104754c3
[root@guruson ~]#
혹시나 키 값을 분실하였거나 Expired 되었을 경우 다음과 같이 확인 또는 재 생성이 가능합니다.
a) master node token 값 확인 (master node에서 명령어 실행)
[root@guruson ~]# kubeadm token list
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
y0ilnu.ospwoi1kbxscy9iz 13h 2019-08-04T20:29:54+09:00 authentication,signing The default bootstrap token generated by 'kubeadm init'. system:bootstrappers:kubeadm:default-node-token
[root@guruson ~]# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
325d28e81247bea994ebf2fa6bccb5af211aa3e6a2a762af800992b0080fb103
[root@guruson ~]#위와 같이 token을 확인하고 Expired 이전 이라면 해당 그대로 사용이 가능합니다.
적용 방법은 kubeadm join --token [TOKEN] --discovery-token-ca-cert-hash sha256:[OPENSSL_ENCRYPT] [MASTER_APISERVER_IP]:6443
[root@guruson ~]# kubeadm join --token y0ilnu.ospwoi1kbxscy9iz --discovery-token-ca-cert-hash sha256:325d28e81247bea994ebf2fa6bccb5af211aa3e6a2a762af800992b0080fb103 ^C
[root@guruson ~]#와 같습니다.
b) master node token이 만료되었을 경우 (master node에서 명령어 실행)
[root@guruson ~]# kubeadm token create
bzgy3x.ltfohixm3ijxm67l
[root@guruson ~]# kubeadm token list
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
bzgy3x.ltfohixm3ijxm67l 23h 2019-08-05T08:15:29+09:00 authentication,signing <none> system:bootstrappers:kubeadm:default-node-token
y0ilnu.ospwoi1kbxscy9iz 12h 2019-08-04T20:29:54+09:00 authentication,signing The default bootstrap token generated by 'kubeadm init'. system:bootstrappers:kubeadm:default-node-token
[root@guruson ~]#위와 같이 신규로 생성이 가능합니다.
3. Master Node 연결 상태 확인
Master Node와 Workder Node의 연결상태를 확인합니다.
[root@guruson ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
guruson Ready master 11h v1.15.1
kubeworker Ready <none> 3m18s v1.15.1
[root@guruson ~]# 연결이 정상적으로 Ready 된 것을 확인할 수 있습니다.
4. Workder Node 상태 확인
[root@kubeworker ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
703c6923dadc weaveworks/weave-npc "/usr/bin/weave-npc" 53 minutes ago Up 53 minutes k8s_weave-npc_weave-net-prcb5_kube-system_35d5017f-b830-48d7-84df-3a40b8aebd5a_0
f9f84fe7f91e weaveworks/weave-kube "/home/weave/launch. 53 minutes ago Up 53 minutes k8s_weave_weave-net-prcb5_kube-system_35d5017f-b830-48d7-84df-3a40b8aebd5a_0
45ae9c7dbfcc k8s.gcr.io/kube-proxy "/usr/local/bin/kube 53 minutes ago Up 53 minutes k8s_kube-proxy_kube-proxy-6bgfw_kube-system_d70f96e3-92ac-483a-96f0-209bedd1bfad_0
1cf6f307fee1 k8s.gcr.io/pause:3.1 "/pause" 53 minutes ago Up 53 minutes k8s_POD_weave-net-prcb5_kube-system_35d5017f-b830-48d7-84df-3a40b8aebd5a_0
fae7fcda7abc k8s.gcr.io/pause:3.1 "/pause" 53 minutes ago Up 53 minutes k8s_POD_kube-proxy-6bgfw_kube-system_d70f96e3-92ac-483a-96f0-209bedd1bfad_0
[root@kubeworker ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
k8s.gcr.io/kube-proxy v1.15.1 89a062da739d 2 weeks ago 82.4MB
weaveworks/weave-kube 2.5.2 f04a043bb67a 2 months ago 148MB
weaveworks/weave-npc 2.5.2 5ce48e0d813c 2 months ago 49.6MB
k8s.gcr.io/pause 3.1 da86e6ba6ca1 19 months ago 742kB
[root@kubeworker ~]# Workder Node는 kube-proxy와 weaveworks Docker 이미지가 기동되어 있습니다.
이번 포스팅에서는 Master Node와 Worker Node를 연동하는 방법에 대해 살펴보았습니다.
다음 포스팅에서는 실제 Pod를 생성하여 애플리케이션을 디플로이하기 위한 구조를 생성해 보도록 하겠습니다.
'③ 클라우드 > ⓚ Kubernetes' 카테고리의 다른 글
| [Container Management] Kubernetes Service 생성 가이드 (0) | 2019.08.04 |
|---|---|
| [Container Management] Kubernetes Pod 생성 가이드 (0) | 2019.08.04 |
| [Container Management] Kubernetes Dashboard Install & Setting (34) | 2019.08.01 |
| [Container Management] Kubernetes Master Node 설치 (11) | 2019.07.24 |
| [Container Management] Kubernetes 개요 (0) | 2019.07.23 |
- Total
- Today
- Yesterday
- Da
- jeus
- k8s
- TA
- SA
- 쿠버네티스
- OpenStack
- aws
- apache
- API Gateway
- aa
- Architecture
- 아키텍처
- JEUS7
- SWA
- kubernetes
- openstack token issue
- JEUS6
- webtob
- node.js
- openstack tenant
- wildfly
- JBoss
- 마이크로서비스 아키텍처
- nodejs
- 오픈스택
- Docker
- git
- 마이크로서비스
- MSA
| 일 | 월 | 화 | 수 | 목 | 금 | 토 |
|---|---|---|---|---|---|---|
| 1 | 2 | |||||
| 3 | 4 | 5 | 6 | 7 | 8 | 9 |
| 10 | 11 | 12 | 13 | 14 | 15 | 16 |
| 17 | 18 | 19 | 20 | 21 | 22 | 23 |
| 24 | 25 | 26 | 27 | 28 | 29 | 30 |