티스토리 뷰
728x90
반응형
본 포스팅에서는 Kubespray를 사용하여 Kubernetes 설치 자동화 구성에 대해 실습해 보도록 하겠습니다.
Kubernetes를 구성하기 위해서는 Native Kubernetes를 직접 다운로드 받아 구성할 수도 있지만, Kubernetes의 다양한 모듈과 이와 연계되는 에코 모듈들을 하나하나 구성하는 것은 왠만한 전문가 수준 이상을 요구하는 기술입니다.
물론 Kubernetes 자동화 모듈을 통해 어느정도 Kubernetes에 대해 익숙해 졌다면 이후에 도전해 볼 만한 내용이기는 하지만, 처음 또는 Kubernetes에 대한 완전한 이해를 갖추지 못한 채로 Navite Kubernetes를 설치하는 것은 매우 어려운 일입니다.
저 같은 경우 약 2년 전 여름 처음 Kubernetes를 접해 봤고 Docker, PaaS, Container 등의 개념 조차 잡히지 않은 상태로 개인적인 관심으로 인해 설치를 진행해 봤는데, 대략 3개월의 시간이 소요됐었던 것 같습니다.
물론, 당시 구성 자료는 홀연히 사라져버려, 제대로 구성을 했는지 여부는 판단하기 어렵지만, Native K8S를 구성하고, Dashboard를 연계하고, Apache HTTPD Page를 호출하는데 까지 걸린 시간입니다. ㅠ.ㅠ
지금 생각해보면, 별거 아닌 내용이지만, 그만큼 처음 접하는 분들에게는 쉽지 않다는 의미로 받아주세요. 물론 모두 그렇지는 않을 거고, 제가 좀 이해가 딸리는 편이라..
그럼 본격적으로 살펴보겠습니다.
Kubernetes 자동화 구성 방법
대표적인 Kubernetes 자동화 구성방법은 앞선 포스팅에서 이미 살펴보았던 kubeadm, kops 그리고 본 포스팅에서 살펴 볼 kubespray가 있습니다.
먼저 kubeadm으로 kubernetes를 설치하기 위한 방법은 아래 포스팅을 참조하시기 바랍니다.
[Container Management] Kubernetes Master Node 설치
[Container Management] Kubernetes Dashboard Install & Setting
[Container Management] Kubernetes Woker Node Install & Setting
[Container Management] Kubernetes Pod 생성 가이드
[Container Management] Kubernetes Service 생성 가이드
[Container Management] Kubernetes 애플리케이션 관리
[Container Management] Kubernetes ingress & ingress controller
1. Kubeadm
Kubeadm은 가장 대표적인 Kubernetes 구성 방식입니다. kubeadm 자동화 Command를 CLI에 입력하는 것만으로도 손쉬운 구성이 가능합니다. Kubernetes Master Node를 구성하고, Worker Node는 Master Node에서 발급된 Token Key 기반으로 kubeadm에 join하는 방식으로 구성됩니다. 매우 쉽기때문에 Kubernetes를 처음 접하는 사용자에게 적합하며, quick start 예제 들이 Kubernetes 홈페이지에 가장 많이 게시되어 있어 따라하기 쉽다는 장점이 있는 반면, Master Node, Worker Node 등에 대한 보다 상세한 자동화를 구성하기 위해서는 Shell Script를 직접 작성하여 운영해야 하는 부분이 발생할 수 있습니다.
2. Kops
Kops는 AWS에 구성할 때 보다 효율적인 방법을 제공합니다. Kubespray와 마찬가지로 terraform을 통해 프로비저닝을 수행할 수 있습니다. Kops는 AWS를 공식적으로 지원하여, S3, Route53, ELB 등을 kubeadm을 통해 쉽게 사용할 수 있습니다.
3. Kubespray
마지막으로 포스팅에서 다룰 Kubespray입니다.
Kubespray로 Kubernetes를 설치하기 위해서는 다음과 요구사항이 선행되어야 합니다.
- Ansible v2.5 이상 + Jinja 2.9 이상 버전 설치 요구
- Firewall에 대한 정책 정의되거나, Disabled
- Swap Memory에 대한 사용을 Disable
- sudo 권한 부여 (add nopassword)
- ssh key create & copy 등
Kubespray는 Ansible 기반으로 구성하게 되는데, 여기서 Ansible에 대해 설명하기에는 부연 설명이 너무 커지는 느낌이 들어 이후에 별도로 Ansible에 대해 포스팅을 준비하도록 하고, Ansible은 간단히 프로비저닝, 구성 관리, 애플리케이션 배포 등을 구성해 주는 도구라고 생각하면 될 것 같습니다.
Ansible의 Inventory Sample을 제공하는데 해당 파일을 본인의 CLOUD_PROJECT로 복사하여 커스터마이징하여 사용합니다. Ansible의 주요 설정은 inventory.ini, all.yml, k8s-cluster.yml, main.yml, docker.yml, cluster.yml 등의 커스터마이징을 진행합니다.
Kubespray의 CNI(Container Network Interface)는 Calico가 Default를 지정되어 있습니다. Calico는 대체로 일반 중견 프로젝트 수준 (대략 50대 미만의 프로젝트)에서 구성할 경우 효과적입니다. 기본 Mesh 형태로 구성되어 수백대의 이커머스 시스템이나, 대국민 서비스에 사용하기에는 구조가 복잡해 질 수 있습니다. Calico의 경우 iptables를 사용하기 때문에 보다 빠른 네트워크 성능을 원할 경우 ipvs 기반의 CNI(본 포스팅에서는 weave를 사용)를 사용하도록 변경합니다.
Kubespray를 활용한 Kubernetes 설치
1. 사전 환경 구성
먼저 Kubespray를 사용하기 위한 환경을 구성해 보도록 하겠습니다.
- /etc/hosts 파일 구성 (Master Node & Worker Node)
Kube-master & Kube-worker Node 대상으로 사용할 Node 정보를 사전에 적용합니다.
ex) /etc/hosts 등록
192.168.56.107 kubemaster
192.168.56.106 kubeworker- ssh key generation & copy (Master Node)
Kubernetes Master Node에서 Worker Node로 Command를 실행할 수 있도록 ssh 접속 key를 생성하고 복사해 줍니다.
ex) ssh 키 생성 (ssh-keygen -t rsa)
[root@kubemaster ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:LMuUvgCERHcDUI7ATtff+RuLnTKCg8BF5pN37WFlSj4 root@kubemaster
The key's randomart image is:
+---[RSA 2048]----+
|++o+oo |
|o+++... . o |
|+.*.. . = = |
| o = . = E |
|. o o = S + |
| o . + o . o |
| . o = o = |
| . + o + = |
| o . o |
+----[SHA256]-----+
[root@kubemaster ~]#비밀번호 없이 사용이 가능하도록 모든 입력 값은 Enter로 패스하도록 합니다.
ex) ssh 키 복사 (ssh-copy-id root@ip)
[root@kubemaster ~]# ssh-copy-id root@192.168.56.107
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.56.107 (192.168.56.107)' can't be established.
ECDSA key fingerprint is SHA256:H12Hgo0dxJZ0si00ji4uRh+XWIP/vi3hUsThwWede+c.
ECDSA key fingerprint is MD5:ce:60:e7:60:00:2c:f2:b6:33:15:94:86:d3:d4:9e:49.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.56.107's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'root@192.168.56.107'"
and check to make sure that only the key(s) you wanted were added.
[root@kubemaster ~]# ssh-copy-id root@192.168.56.106
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.56.106's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'root@192.168.56.106'"
and check to make sure that only the key(s) you wanted were added.
[root@kubemaster ~]#생성한 ssh public key를 통해 인증 받을 수 있도록 key 파일을 master & worker node에 각각 전송합니다.
정상적으로 전송되었는지 여부는 ~/.ssh/known_hosts 파일을 통해 확인할 수 있습니다.
[root@kubeworker .ssh]# cat known_hosts
192.168.56.107 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNpNuaHjPg3u0zhJSaSh6r62Tqxd9nl0m5I6HIV7meiP9f72HPC6DAo4oKHgrDyJIRR0zudJF0Z4H7dCuGp6Ruc=
[root@kubeworker .ssh]#- /proc/sys/net/ipv4/ip_forward 활성화 (Master Node & Worker Node)
가상화 환경에서 서로 다른 네트워크 환경에서 라우팅이 필요한 경우 특히 Kubernetes의 경우 가상 네트워크 환경을 사용해야 하므로 ip_forward를 활성화 해야 합니다.
ex) /proc/sys/net/ipv4/ip_forward enable
[root@kubemaster .ssh]# echo 1 > /proc/sys/net/ipv4/ip_forward
[root@kubemaster .ssh]#1의 경우 enable, 0의 경우 disable을 의미하며, 1을 적용합니다.
- firewalld 중지 및 disable (Master Node & Worker Node)
Kubernetes가 사용하는 네트워크 통신에 대한 정확한 이해가 필요하며, 이에 대한 이해도가 높아지기 전에는 firewalld를 중지하여 구성합니다.
ex) systemctl stop firewalld && systemctl disable firewalld 적용
[root@kubemaster .ssh]# systemctl stop firewalld && systemctl disable firewalld
[root@kubemaster .ssh]# - SELINUX permissive 변경 (Master Node & Worker Node)
firewalld와 마찬가지로 네트워크 정책에 대한 명확한 이해가 생기기 전까지는 disabled 또는 permissive로 변경하여 구성합니다.
ex) /etc/sysconfig/selinux 구성
[root@kubemaster .ssh]# sed -i 's/^SELINUX=.*/SELINUX=disabled/g' /etc/sysconfig/selinux && cat /etc/sysconfig/selinux
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=permissive
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
[root@kubemaster .ssh]# setenforce 0
[root@kubemaster .ssh]#- swap memory 사용 중지 (Master Node & Worker Node)
Kubernetes는 성능 상의 이유로 swap memory 사용을 기본 권장하지 않고 있습니다. swapoff -a 명령어로 swap memory 사용을 중지합니다.
ex) swapoff 적용
[root@kubemaster ~]# free -m
total used free shared buff/cache available
Mem: 1837 154 1402 12 280 1533
Swap: 819 0 819
[root@kubemaster ~]# swapoff -a
[root@kubemaster ~]# free -m
total used free shared buff/cache available
Mem: 1837 155 1401 12 280 1532
Swap: 0 0 0
[root@kubemaster ~]#위와 같이 swap memory 영역이 적용 전 819로 할당되어 있으나, swapoff 적용 후 0으로 변경된 것을 확인할 수 있습니다.
2. Kubespray 실행을 위한 Utility 설치
Kubespray를 기동하기 위해서는 몇몇 Utility들이 설치되어야 합니다.
- python3, pip, wget 설치 (Master Node)
먼저 python3를 설치하기 위한 yum repository를 추가합니다.
ex) ius yum repository 설치
[root@kubemaster ~]# yum install -y https://centos7.iuscommunity.org/ius-release.rpm
Loaded plugins: fastestmirror
ius-release.rpm | 8.2 kB 00:00:00
Examining /var/tmp/yum-root-op7EBf/ius-release.rpm: ius-release-2-1.el7.ius.noarch
Marking /var/tmp/yum-root-op7EBf/ius-release.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package ius-release.noarch 0:2-1.el7.ius will be installed
--> Processing Dependency: epel-release = 7 for package: ius-release-2-1.el7.ius.noarch
Determining fastest mirrors
* base: mirror.navercorp.com
* extras: mirror.navercorp.com
* updates: mirror.navercorp.com
base | 3.6 kB 00:00:00
extras | 2.9 kB 00:00:00
kubernetes/signature | 454 B 00:00:00
kubernetes/signature | 1.4 kB 00:00:00 !!!
updates | 2.9 kB 00:00:00
(1/2): updates/7/x86_64/primary_db | 1.9 MB 00:00:00
(2/2): kubernetes/primary | 57 kB 00:00:00
kubernetes 409/409
--> Running transaction check
---> Package epel-release.noarch 0:7-11 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
====================================================================================================================
Package Arch Version Repository Size
====================================================================================================================
Installing:
ius-release noarch 2-1.el7.ius /ius-release 4.5 k
Installing for dependencies:
epel-release noarch 7-11 extras 15 k
Transaction Summary
====================================================================================================================
Install 1 Package (+1 Dependent package)
Total size: 19 k
Total download size: 15 k
Installed size: 29 k
Downloading packages:
epel-release-7-11.noarch.rpm | 15 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : epel-release-7-11.noarch 1/2
Installing : ius-release-2-1.el7.ius.noarch 2/2
Verifying : ius-release-2-1.el7.ius.noarch 1/2
Verifying : epel-release-7-11.noarch 2/2
Installed:
ius-release.noarch 0:2-1.el7.ius
Dependency Installed:
epel-release.noarch 0:7-11
Complete!
[root@kubemaster ~]#정상적으로 추가가 완료되면, yum repository에 다음과 같이 repo 파일이 추가될 것입니다.
[root@kubemaster yum.repos.d]# ls -la
total 72
drwxr-xr-x. 2 root root 4096 Oct 13 00:15 .
drwxr-xr-x. 88 root root 8192 Oct 12 22:25 ..
-rw-r--r--. 1 root root 1664 Sep 5 22:05 CentOS-Base.repo
-rw-r--r--. 1 root root 1309 Sep 5 22:05 CentOS-CR.repo
-rw-r--r--. 1 root root 649 Sep 5 22:05 CentOS-Debuginfo.repo
-rw-r--r--. 1 root root 314 Sep 5 22:05 CentOS-fasttrack.repo
-rw-r--r--. 1 root root 630 Sep 5 22:05 CentOS-Media.repo
-rw-r--r--. 1 root root 1331 Sep 5 22:05 CentOS-Sources.repo
-rw-r--r--. 1 root root 6639 Sep 5 22:05 CentOS-Vault.repo
-rw-r--r--. 1 root root 951 Oct 3 2017 epel.repo
-rw-r--r--. 1 root root 1050 Oct 3 2017 epel-testing.repo
-rw-r--r--. 1 root root 669 May 2 01:00 ius-archive.repo
-rw-r--r--. 1 root root 591 May 2 01:00 ius.repo
-rw-r--r--. 1 root root 669 May 2 01:00 ius-testing.repo
-rw-r--r--. 1 root root 285 Sep 15 22:55 kubebyinternet.repo
[root@kubemaster yum.repos.d]#명확히 repository를 clear 하기 위해서는 다음과 같이 적용을 진행합니다.
[root@kubemaster yum.repos.d]# yum clean all
Loaded plugins: fastestmirror
Cleaning repos: base epel extras ius kubernetes updates
Cleaning up list of fastest mirrors
[root@kubemaster yum.repos.d]# yum repolist all
Loaded plugins: fastestmirror
Determining fastest mirrors
epel/x86_64/metalink | 5.4 kB 00:00:00
* base: mirror.kakao.com
* epel: hk.mirrors.thegigabit.com
* extras: mirror.kakao.com
* updates: mirror.kakao.com
base | 3.6 kB 00:00:00
epel | 5.3 kB 00:00:00
extras | 2.9 kB 00:00:00
ius | 1.3 kB 00:00:00
kubernetes/signature | 454 B 00:00:00
kubernetes/signature | 1.4 kB 00:00:00 !!!
updates | 2.9 kB 00:00:00
(1/9): base/7/x86_64/group_gz | 165 kB 00:00:00
(2/9): extras/7/x86_64/primary_db | 152 kB 00:00:00
(3/9): epel/x86_64/group_gz | 88 kB 00:00:00
(4/9): epel/x86_64/updateinfo | 1.0 MB 00:00:00
(5/9): base/7/x86_64/primary_db | 6.0 MB 00:00:00
(6/9): updates/7/x86_64/primary_db | 1.9 MB 00:00:00
(7/9): kubernetes/primary | 57 kB 00:00:01
(8/9): ius/x86_64/primary | 121 kB 00:00:05
(9/9): epel/x86_64/primary_db | 6.9 MB 00:03:22
ius 481/481
kubernetes 409/409
repo id repo name status
...
...
...
ius/x86_64 IUS for Enterprise Linux 7 - x86_64 enabled: 481
ius-archive/x86_64 IUS for Enterprise Linux 7 - Archive - x86_64 disabled
ius-archive-debuginfo/x86_64 IUS for Enterprise Linux 7 - Archive - x86_64 - Debug disabled
ius-archive-source IUS for Enterprise Linux 7 - Archive - Source disabled
ius-debuginfo/x86_64 IUS for Enterprise Linux 7 - x86_64 - Debug disabled
ius-source IUS for Enterprise Linux 7 - Source disabled
ius-testing/x86_64 IUS for Enterprise Linux 7 - Testing - x86_64 disabled
ius-testing-debuginfo/x86_64 IUS for Enterprise Linux 7 - Testing - x86_64 - Debug disabled
ius-testing-source IUS for Enterprise Linux 7 - Testing - Source disabled
kubernetes Kubernetes enabled: 8+401
updates/7/x86_64 CentOS-7 - Updates enabled: 332
updates-source/7 CentOS-7 - Updates Sources disabled
repolist: 24,637
[root@kubemaster yum.repos.d]# repository 적용이 완료되면, python3와 pip 그리고 wget을 설치합니다.
ex) yum install -y python36u python36u-pip wget
[root@kubemaster yum.repos.d]# yum install -y python36u python36u-pip wget
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirror.kakao.com
* epel: hk.mirrors.thegigabit.com
* extras: mirror.kakao.com
* updates: mirror.kakao.com
Package python36u is obsoleted by python3, trying to install python3-3.6.8-10.el7.x86_64 instead
Package python36u-pip is obsoleted by python3-pip, trying to install python3-pip-9.0.3-5.el7.noarch instead
Package wget-1.14-18.el7_6.1.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package python3.x86_64 0:3.6.8-10.el7 will be installed
--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64
--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64
--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64
---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed
--> Running transaction check
---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed
--> Processing Dependency: libtirpc.so.1()(64bit) for package: python3-libs-3.6.8-10.el7.x86_64
---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed
--> Running transaction check
---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
====================================================================================================================
Package Arch Version Repository Size
====================================================================================================================
Installing:
python3 x86_64 3.6.8-10.el7 base 69 k
python3-pip noarch 9.0.3-5.el7 base 1.8 M
Installing for dependencies:
libtirpc x86_64 0.2.4-0.16.el7 base 89 k
python3-libs x86_64 3.6.8-10.el7 base 7.0 M
python3-setuptools noarch 39.2.0-10.el7 base 629 k
Transaction Summary
====================================================================================================================
Install 2 Packages (+3 Dependent packages)
Total download size: 9.5 M
Installed size: 48 M
Downloading packages:
(1/5): libtirpc-0.2.4-0.16.el7.x86_64.rpm | 89 kB 00:00:00
(2/5): python3-3.6.8-10.el7.x86_64.rpm | 69 kB 00:00:00
(3/5): python3-setuptools-39.2.0-10.el7.noarch.rpm | 629 kB 00:00:00
(4/5): python3-pip-9.0.3-5.el7.noarch.rpm | 1.8 MB 00:00:00
(5/5): python3-libs-3.6.8-10.el7.x86_64.rpm | 7.0 MB 00:00:01
--------------------------------------------------------------------------------------------------------------------
Total 8.6 MB/s | 9.5 MB 00:00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libtirpc-0.2.4-0.16.el7.x86_64 1/5
Installing : python3-setuptools-39.2.0-10.el7.noarch 2/5
Installing : python3-pip-9.0.3-5.el7.noarch 3/5
Installing : python3-3.6.8-10.el7.x86_64 4/5
Installing : python3-libs-3.6.8-10.el7.x86_64 5/5
Verifying : libtirpc-0.2.4-0.16.el7.x86_64 1/5
Verifying : python3-libs-3.6.8-10.el7.x86_64 2/5
Verifying : python3-setuptools-39.2.0-10.el7.noarch 3/5
Verifying : python3-pip-9.0.3-5.el7.noarch 4/5
Verifying : python3-3.6.8-10.el7.x86_64 5/5
Installed:
python3.x86_64 0:3.6.8-10.el7 python3-pip.noarch 0:9.0.3-5.el7
Dependency Installed:
libtirpc.x86_64 0:0.2.4-0.16.el7 python3-libs.x86_64 0:3.6.8-10.el7 python3-setuptools.noarch 0:39.2.0-10.el7
Complete!
[root@kubemaster yum.repos.d]#- ansible, netaddr, jinja2 설치
설치한 pip python package installer를 이용하여 ansible, netaddr, jinja2를 설치합니다.
ex) pip3.6 install을 이용한 Utility 설치
[root@kubemaster yum.repos.d]# pip3.6 install ansible netaddr jinja2
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3.6 install --user` instead.
Collecting ansible
Downloading https://files.pythonhosted.org/packages/04/25/48fee5f8048360d9375e01846fcf395dda58242ed1f25a2106b6794452eb/ansible-2.8.5.tar.gz (14.4MB)
100% |----------------------------------------------------| 14.4MB 28kB/s
Collecting netaddr
Downloading https://files.pythonhosted.org/packages/ba/97/ce14451a9fd7bdb5a397abf99b24a1a6bb7a1a440b019bebd2e9a0dbec74/netaddr-0.7.19-py2.py3-none-any.whl (1.6MB)
100% |----------------------------------------------------| 1.6MB 33kB/s
Collecting jinja2
Downloading https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl (125kB)
100% |----------------------------------------------------| 133kB 58kB/s
Collecting PyYAML (from ansible)
Downloading https://files.pythonhosted.org/packages/e3/e8/b3212641ee2718d556df0f23f78de8303f068fe29cdaa7a91018849582fe/PyYAML-5.1.2.tar.gz (265kB)
100% |----------------------------------------------------| 266kB 49kB/s
Collecting cryptography (from ansible)
Downloading https://files.pythonhosted.org/packages/97/18/c6557f63a6abde34707196fb2cad1c6dc0dbff25a200d5044922496668a4/cryptography-2.7-cp34-abi3-manylinux1_x86_64.whl (2.3MB)
100% |----------------------------------------------------| 2.3MB 80kB/s
Collecting MarkupSafe>=0.23 (from jinja2)
Downloading https://files.pythonhosted.org/packages/b2/5f/23e0023be6bb885d00ffbefad2942bc51a620328ee910f64abe5a8d18dd1/MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting cffi!=1.11.3,>=1.8 (from cryptography->ansible)
Downloading https://files.pythonhosted.org/packages/5f/bf/6aa1925384c23ffeb579e97a5569eb9abce41b6310b329352b8252cee1c3/cffi-1.12.3-cp36-cp36m-manylinux1_x86_64.whl (430kB)
100% |----------------------------------------------------| 440kB 45kB/s
Collecting six>=1.4.1 (from cryptography->ansible)
Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting asn1crypto>=0.21.0 (from cryptography->ansible)
Downloading https://files.pythonhosted.org/packages/6e/1e/fb0e487b5229e5fb7b15c6d00b4e8082a3414fe62b1da4c9a905b106e672/asn1crypto-1.1.0-py2.py3-none-any.whl (103kB)
100% |----------------------------------------------------| 112kB 28kB/s
Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography->ansible)
Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
100% |----------------------------------------------------| 163kB 69kB/s
Installing collected packages: MarkupSafe, jinja2, PyYAML, pycparser, cffi, six, asn1crypto, cryptography, ansible, netaddr
Running setup.py install for PyYAML ... done
Running setup.py install for pycparser ... done
Running setup.py install for ansible ... done
Successfully installed MarkupSafe-1.1.1 PyYAML-5.1.2 ansible-2.8.5 asn1crypto-1.1.0 cffi-1.12.3 cryptography-2.7 jinja2-2.10.3 netaddr-0.7.19 pycparser-2.19 six-1.12.0
[root@kubemaster yum.repos.d]#3. Kubespray 릴리즈 소스코드 다운로드
Kubernetes는 Kubespray로 구성하기 위한 github 소스코드를 제공합니다.
- github 소스 다운로드
github에 업로드 되어 있는 다음 URL에서 소스코드를 다운로드 받습니다. git clone 사용방법은 다음 URL을 참고합니다.
[Spring Boot] git clone을 활용한 Spring Boot 프로젝트 repository 이관
[root@kubemaster kubespray]# git clone https://github.com/kubernetes-sigs/kubespray
Cloning into 'kubespray'...
remote: Enumerating objects: 38993, done.
remote: Total 38993 (delta 0), reused 0 (delta 0), pack-reused 38993
Receiving objects: 100% (38993/38993), 11.28 MiB | 3.68 MiB/s, done.
Resolving deltas: 100% (21607/21607), done.
[root@kubemaster kubespray]#다운로드가 완료되면 이제 본격적인 설정작업을 진행합니다.
4. Kubespray 설정
Kubespray는 ansible을 기반으로 Kubernetes를 구성하게 되며, 이때 inventory.ini 파일을 생성하여 구성 대상 서버 정보를 정의해야 합니다.
- inventory.ini 파일 구성
[root@kubemaster kubespray]# cp inventory/local/hosts.ini inventory/inventory.ini
[root@kubemaster kubespray]# vi inventory/inventory.ini
kubemaster ansible_ssh_host=192.168.56.107 ip=192.168.56.107
kubeworker ansible_ssh_host=192.168.56.106 ip=192.168.56.106
[kube-master]
kubemaster
[etcd]
kubemaster
[kube-node]
kubeworker
[k8s-cluster:children]
kube-node
kube-master
[root@kubemaster kubespray]# Master Node를 다중화 하여 관리하고자 할 경우 kube-master를 늘려주고, Worker Node를 늘려주고 싶을 경우 kube-node를 늘려주면됩니다.
또한 설치하고자 하는 모듈을 정의합니다.
[root@kubemaster k8s-cluster]# pwd
/root/git_repo/kubespray/inventory/myinventory/group_vars/k8s-cluster
[root@kubemaster k8s-cluster]# vi addons.yml
---
# Kubernetes dashboard
# RBAC required. see docs/getting-started.md for access details.
dashboard_enabled: true
# Helm deployment
helm_enabled: true
#helm_enabled: false
# Registry deployment
registry_enabled: false
# registry_namespace: kube-system
# registry_storage_class: ""
# registry_disk_size: "10Gi"
# Metrics Server deployment
metrics_server_enabled: true
#metrics_server_enabled: false
metrics_server_kubelet_insecure_tls: true
# metrics_server_kubelet_insecure_tls: true
# metrics_server_metric_resolution: 60s
# metrics_server_kubelet_preferred_address_types: "InternalIP"
# Rancher Local Path Provisioner
local_path_provisioner_enabled: false
# local_path_provisioner_namespace: "local-path-storage"
# local_path_provisioner_storage_class: "local-path"
# local_path_provisioner_reclaim_policy: Delete
# local_path_provisioner_claim_root: /opt/local-path-provisioner/
# local_path_provisioner_debug: false
# local_path_provisioner_image_repo: "rancher/local-path-provisioner"
# local_path_provisioner_image_tag: "v0.0.2"
# Local volume provisioner deployment
local_volume_provisioner_enabled: false
# local_volume_provisioner_namespace: kube-system
# local_volume_provisioner_storage_classes:
# local-storage:
# host_dir: /mnt/disks
# mount_dir: /mnt/disks
# volume_mode: Filesystem
# fs_type: ext4
# fast-disks:
# host_dir: /mnt/fast-disks
# mount_dir: /mnt/fast-disks
# block_cleaner_command:
# - "/scripts/shred.sh"
# - "2"
# volume_mode: Filesystem
# fs_type: ext4
# CephFS provisioner deployment
cephfs_provisioner_enabled: false
# cephfs_provisioner_namespace: "cephfs-provisioner"
# cephfs_provisioner_cluster: ceph
# cephfs_provisioner_monitors: "172.24.0.1:6789,172.24.0.2:6789,172.24.0.3:6789"
# cephfs_provisioner_admin_id: admin
# cephfs_provisioner_secret: secret
# cephfs_provisioner_storage_class: cephfs
# cephfs_provisioner_reclaim_policy: Delete
# cephfs_provisioner_claim_root: /volumes
# cephfs_provisioner_deterministic_names: true
# RBD provisioner deployment
rbd_provisioner_enabled: false
# rbd_provisioner_namespace: rbd-provisioner
# rbd_provisioner_replicas: 2
# rbd_provisioner_monitors: "172.24.0.1:6789,172.24.0.2:6789,172.24.0.3:6789"
# rbd_provisioner_pool: kube
# rbd_provisioner_admin_id: admin
# rbd_provisioner_secret_name: ceph-secret-admin
# rbd_provisioner_secret: ceph-key-admin
"addons.yml" 110L, 3660C
# Helm deployment
helm_enabled: true
#helm_enabled: false
# Registry deployment
registry_enabled: false
# registry_namespace: kube-system
# registry_storage_class: ""
# registry_disk_size: "10Gi"
# Metrics Server deployment
metrics_server_enabled: true
#metrics_server_enabled: false
metrics_server_kubelet_insecure_tls: true
# metrics_server_kubelet_insecure_tls: true
# metrics_server_metric_resolution: 60s
# metrics_server_kubelet_preferred_address_types: "InternalIP"
# Rancher Local Path Provisioner
local_path_provisioner_enabled: false
# local_path_provisioner_namespace: "local-path-storage"
# local_path_provisioner_storage_class: "local-path"
# local_path_provisioner_reclaim_policy: Delete
# local_path_provisioner_claim_root: /opt/local-path-provisioner/
# local_path_provisioner_debug: false
# local_path_provisioner_image_repo: "rancher/local-path-provisioner"
# local_path_provisioner_image_tag: "v0.0.2"
# Local volume provisioner deployment
local_volume_provisioner_enabled: false
# local_volume_provisioner_namespace: kube-system
# local_volume_provisioner_storage_classes:
---
# Kubernetes dashboard
# RBAC required. see docs/getting-started.md for access details.
dashboard_enabled: true
# Helm deployment
helm_enabled: true
#helm_enabled: false
# Registry deployment
registry_enabled: false
# registry_namespace: kube-system
# registry_storage_class: ""
# registry_disk_size: "10Gi"
# Metrics Server deployment
metrics_server_enabled: true
#metrics_server_enabled: false
metrics_server_kubelet_insecure_tls: true
# metrics_server_kubelet_insecure_tls: true
# metrics_server_metric_resolution: 60s
# metrics_server_kubelet_preferred_address_types: "InternalIP"
# Rancher Local Path Provisioner
local_path_provisioner_enabled: false
# local_path_provisioner_namespace: "local-path-storage"
# local_path_provisioner_storage_class: "local-path"
# local_path_provisioner_reclaim_policy: Delete
# local_path_provisioner_claim_root: /opt/local-path-provisioner/
# local_path_provisioner_debug: false
# local_path_provisioner_image_repo: "rancher/local-path-provisioner"
# local_path_provisioner_image_tag: "v0.0.2"
# Local volume provisioner deployment
local_volume_provisioner_enabled: false
# local_volume_provisioner_namespace: kube-system
# local_volume_provisioner_storage_classes:
# local-storage:
# host_dir: /mnt/disks
# mount_dir: /mnt/disks
# volume_mode: Filesystem
# fs_type: ext4
# fast-disks:
# host_dir: /mnt/fast-disks
# mount_dir: /mnt/fast-disks
# block_cleaner_command:
# - "/scripts/shred.sh"
# - "2"
# volume_mode: Filesystem
# fs_type: ext4
# CephFS provisioner deployment
cephfs_provisioner_enabled: false
# cephfs_provisioner_namespace: "cephfs-provisioner"
# cephfs_provisioner_cluster: ceph
# cephfs_provisioner_monitors: "172.24.0.1:6789,172.24.0.2:6789,172.24.0.3:6789"
# cephfs_provisioner_admin_id: admin
# cephfs_provisioner_secret: secret
# cephfs_provisioner_storage_class: cephfs
# cephfs_provisioner_reclaim_policy: Delete
# cephfs_provisioner_claim_root: /volumes
# cephfs_provisioner_deterministic_names: true
# RBD provisioner deployment
rbd_provisioner_enabled: false
# rbd_provisioner_namespace: rbd-provisioner
# rbd_provisioner_replicas: 2
# rbd_provisioner_monitors: "172.24.0.1:6789,172.24.0.2:6789,172.24.0.3:6789"
# rbd_provisioner_pool: kube
# rbd_provisioner_admin_id: admin
# rbd_provisioner_secret_name: ceph-secret-admin
# rbd_provisioner_secret: ceph-key-admin
# Helm deployment
helm_enabled: true
#helm_enabled: false
# Registry deployment
registry_enabled: false
# registry_namespace: kube-system
# registry_storage_class: ""
# registry_disk_size: "10Gi"
# Metrics Server deployment
metrics_server_enabled: true
#metrics_server_enabled: false
metrics_server_kubelet_insecure_tls: true
# metrics_server_kubelet_insecure_tls: true
# metrics_server_metric_resolution: 60s
# metrics_server_kubelet_preferred_address_types: "InternalIP"
# Rancher Local Path Provisioner
local_path_provisioner_enabled: false
# local_path_provisioner_namespace: "local-path-storage"
# local_path_provisioner_storage_class: "local-path"
# local_path_provisioner_reclaim_policy: Delete
# local_path_provisioner_claim_root: /opt/local-path-provisioner/
# local_path_provisioner_debug: false
# local_path_provisioner_image_repo: "rancher/local-path-provisioner"
# local_path_provisioner_image_tag: "v0.0.2"
# Local volume provisioner deployment
local_volume_provisioner_enabled: false
# local_volume_provisioner_namespace: kube-system
# local_volume_provisioner_storage_classes:
---
# Kubernetes dashboard
# RBAC required. see docs/getting-started.md for access details.
dashboard_enabled: true
# Helm deployment
helm_enabled: true
#helm_enabled: false
# Registry deployment
registry_enabled: false
# registry_namespace: kube-system
# registry_storage_class: ""
# registry_disk_size: "10Gi"
# Metrics Server deployment
metrics_server_enabled: true
#metrics_server_enabled: false
metrics_server_kubelet_insecure_tls: true
# metrics_server_kubelet_insecure_tls: true
# metrics_server_metric_resolution: 60s
# metrics_server_kubelet_preferred_address_types: "InternalIP"
# Rancher Local Path Provisioner
local_path_provisioner_enabled: false
# local_path_provisioner_namespace: "local-path-storage"
# local_path_provisioner_storage_class: "local-path"
# local_path_provisioner_reclaim_policy: Delete
# local_path_provisioner_claim_root: /opt/local-path-provisioner/
# local_path_provisioner_debug: false
# local_path_provisioner_image_repo: "rancher/local-path-provisioner"
# local_path_provisioner_image_tag: "v0.0.2"
# Local volume provisioner deployment
local_volume_provisioner_enabled: false
# local_volume_provisioner_namespace: kube-system
# local_volume_provisioner_storage_classes:
# local-storage:
# host_dir: /mnt/disks
# mount_dir: /mnt/disks
# volume_mode: Filesystem
# fs_type: ext4
# fast-disks:
# host_dir: /mnt/fast-disks
# mount_dir: /mnt/fast-disks
# block_cleaner_command:
# - "/scripts/shred.sh"
# - "2"
# volume_mode: Filesystem
# fs_type: ext4
# CephFS provisioner deployment
cephfs_provisioner_enabled: false
# cephfs_provisioner_namespace: "cephfs-provisioner"
# cephfs_provisioner_cluster: ceph
# cephfs_provisioner_monitors: "172.24.0.1:6789,172.24.0.2:6789,172.24.0.3:6789"
# cephfs_provisioner_admin_id: admin
# cephfs_provisioner_secret: secret
# cephfs_provisioner_storage_class: cephfs
# cephfs_provisioner_reclaim_policy: Delete
# cephfs_provisioner_claim_root: /volumes
# cephfs_provisioner_deterministic_names: true
# RBD provisioner deployment
rbd_provisioner_enabled: false
# rbd_provisioner_namespace: rbd-provisioner
# rbd_provisioner_replicas: 2
# rbd_provisioner_monitors: "172.24.0.1:6789,172.24.0.2:6789,172.24.0.3:6789"
# rbd_provisioner_pool: kube
# rbd_provisioner_admin_id: admin
# rbd_provisioner_secret_name: ceph-secret-admin
# rbd_provisioner_secret: ceph-key-admin
[root@kubemaster k8s-cluster]# pwd
/root/git_repo/kubespray_kma/inventory/tec/group_vars/k8s-cluster
[root@kubemaster k8s-cluster]# cat addons.yml
---
# Kubernetes dashboard
# RBAC required. see docs/getting-started.md for access details.
dashboard_enabled: true
# Helm deployment
helm_enabled: true
#helm_enabled: false
# Registry deployment
registry_enabled: false
# registry_namespace: kube-system
# registry_storage_class: ""
# registry_disk_size: "10Gi"
# Metrics Server deployment
metrics_server_enabled: true
#metrics_server_enabled: false
metrics_server_kubelet_insecure_tls: true
# metrics_server_kubelet_insecure_tls: true
# metrics_server_metric_resolution: 60s
# metrics_server_kubelet_preferred_address_types: "InternalIP"
# Rancher Local Path Provisioner
local_path_provisioner_enabled: false
# local_path_provisioner_namespace: "local-path-storage"
# local_path_provisioner_storage_class: "local-path"
# local_path_provisioner_reclaim_policy: Delete
# local_path_provisioner_claim_root: /opt/local-path-provisioner/
# local_path_provisioner_debug: false
# local_path_provisioner_image_repo: "rancher/local-path-provisioner"
# local_path_provisioner_image_tag: "v0.0.2"
# Local volume provisioner deployment
local_volume_provisioner_enabled: false
# local_volume_provisioner_namespace: kube-system
# local_volume_provisioner_storage_classes:
# local-storage:
# host_dir: /mnt/disks
# mount_dir: /mnt/disks
# volume_mode: Filesystem
# fs_type: ext4
# fast-disks:
# host_dir: /mnt/fast-disks
# mount_dir: /mnt/fast-disks
# block_cleaner_command:
# - "/scripts/shred.sh"
# - "2"
# volume_mode: Filesystem
# fs_type: ext4
# CephFS provisioner deployment
cephfs_provisioner_enabled: false
# cephfs_provisioner_namespace: "cephfs-provisioner"
# cephfs_provisioner_cluster: ceph
# cephfs_provisioner_monitors: "172.24.0.1:6789,172.24.0.2:6789,172.24.0.3:6789"
# cephfs_provisioner_admin_id: admin
# cephfs_provisioner_secret: secret
# cephfs_provisioner_storage_class: cephfs
# cephfs_provisioner_reclaim_policy: Delete
# cephfs_provisioner_claim_root: /volumes
# cephfs_provisioner_deterministic_names: true
# RBD provisioner deployment
rbd_provisioner_enabled: false
# rbd_provisioner_namespace: rbd-provisioner
# rbd_provisioner_replicas: 2
# rbd_provisioner_monitors: "172.24.0.1:6789,172.24.0.2:6789,172.24.0.3:6789"
# rbd_provisioner_pool: kube
# rbd_provisioner_admin_id: admin
# rbd_provisioner_secret_name: ceph-secret-admin
# rbd_provisioner_secret: ceph-key-admin
# rbd_provisioner_user_id: kube
# rbd_provisioner_user_secret_name: ceph-secret-user
# rbd_provisioner_user_secret: ceph-key-user
# rbd_provisioner_user_secret_namespace: rbd-provisioner
# rbd_provisioner_fs_type: ext4
# rbd_provisioner_image_format: "2"
# rbd_provisioner_image_features: layering
# rbd_provisioner_storage_class: rbd
# rbd_provisioner_reclaim_policy: Delete
# Nginx ingress controller deployment
ingress_nginx_enabled: true
#ingress_nginx_enabled: false
# ingress_nginx_host_network: false
ingress_publish_status_address: ""
# ingress_nginx_nodeselector:
# beta.kubernetes.io/os: "linux"
# ingress_nginx_tolerations:
# - key: "node-role.kubernetes.io/master"
# operator: "Equal"
# value: ""
# effect: "NoSchedule"
# ingress_nginx_namespace: "ingress-nginx"
# ingress_nginx_insecure_port: 80
# ingress_nginx_secure_port: 443
# ingress_nginx_configmap:
# map-hash-bucket-size: "128"
# ssl-protocols: "SSLv2"
# ingress_nginx_configmap_tcp_services:
# 9000: "default/example-go:8080"
# ingress_nginx_configmap_udp_services:
# 53: "kube-system/coredns:53"
# ingress_nginx_extra_args:
# - --default-ssl-certificate=default/foo-tls
# Cert manager deployment
cert_manager_enabled: false
# cert_manager_namespace: "cert-manager"
[root@kubemaster k8s-cluster]# 위와 같이 kubespray에서 등록 가능한 다양한 모듈이 정의되어 있으며, 기본으로 Dashboard가 true로 구성되어 있습니다.
이중 구성을 위해 아래와 같은 모듈을 true로 변경하고 ansible을 기동합니다.
[root@kubemaster k8s-cluster]# diff addons.yml /root/git_repo/kubespray_kma/inventory/tec/group_vars/k8s-cluster/addons.yml
7c7,8
< helm_enabled: false
---
> helm_enabled: true
> #helm_enabled: false
16c17,19
< metrics_server_enabled: false
---
> metrics_server_enabled: true
> #metrics_server_enabled: false
> metrics_server_kubelet_insecure_tls: true
81c84,85
< ingress_nginx_enabled: false
---
> ingress_nginx_enabled: true
> #ingress_nginx_enabled: false
[root@kubemaster k8s-cluster]#helm repo, metrics_server, ingress_controller를 기본 구성에서 위와 같이 true로 변경하고 설치를 진행합니다.
또한, Calico CNI에서 사용하는 IP Rule에 대한 정책은 기본으로 iptables를 선택합니다.
[root@kubemaster k8s-cluster]# pwd
/root/git_repo/kubespray/inventory/tec/group_vars/k8s-cluster
[root@kubemaster k8s-cluster]# cat k8s-cluster.yml
---
# Kubernetes configuration dirs and system namespace.
# Those are where all the additional config stuff goes
# the kubernetes normally puts in /srv/kubernetes.
# This puts them in a sane location and namespace.
# Editing those values will almost surely break something.
kube_config_dir: /etc/kubernetes
kube_script_dir: "{{ bin_dir }}/kubernetes-scripts"
kube_manifest_dir: "{{ kube_config_dir }}/manifests"
# This is where all the cert scripts and certs will be located
kube_cert_dir: "{{ kube_config_dir }}/ssl"
# This is where all of the bearer tokens will be stored
kube_token_dir: "{{ kube_config_dir }}/tokens"
# This is where to save basic auth file
kube_users_dir: "{{ kube_config_dir }}/users"
kube_api_anonymous_auth: true
## Change this to use another Kubernetes version, e.g. a current beta release
kube_version: v1.16.3
# kubernetes image repo define
kube_image_repo: "{{ gcr_image_repo }}/google-containers"
# Where the binaries will be downloaded.
# Note: ensure that you've enough disk space (about 1G)
local_release_dir: "/tmp/releases"
# Random shifts for retrying failed ops like pushing/downloading
retry_stagger: 5
# This is the group that the cert creation scripts chgrp the
# cert files to. Not really changeable...
kube_cert_group: kube-cert
# Cluster Loglevel configuration
kube_log_level: 2
# Directory where credentials will be stored
credentials_dir: "{{ inventory_dir }}/credentials"
# Users to create for basic auth in Kubernetes API via HTTP
# Optionally add groups for user
kube_api_pwd: "{{ lookup('password', credentials_dir + '/kube_user.creds length=15 chars=ascii_letters,digits') }}"
kube_users:
kube:
pass: "{{kube_api_pwd}}"
role: admin
groups:
- system:masters
## It is possible to activate / deactivate selected authentication methods (basic auth, static token auth)
# kube_oidc_auth: false
# kube_basic_auth: false
# kube_token_auth: false
## Variables for OpenID Connect Configuration https://kubernetes.io/docs/admin/authentication/
## To use OpenID you have to deploy additional an OpenID Provider (e.g Dex, Keycloak, ...)
# kube_oidc_url: https:// ...
# kube_oidc_client_id: kubernetes
## Optional settings for OIDC
# kube_oidc_ca_file: "{{ kube_cert_dir }}/ca.pem"
# kube_oidc_username_claim: sub
# kube_oidc_username_prefix: oidc:
# kube_oidc_groups_claim: groups
# kube_oidc_groups_prefix: oidc:
# Choose network plugin (cilium, calico, contiv, weave or flannel. Use cni for generic cni plugin)
# Can also be set to 'cloud', which lets the cloud provider setup appropriate routing
kube_network_plugin: calico
# Setting multi_networking to true will install Multus: https://github.com/intel/multus-cni
kube_network_plugin_multus: false
# Kubernetes internal network for services, unused block of space.
kube_service_addresses: 10.233.0.0/18
# internal network. When used, it will assign IP
# addresses from this range to individual pods.
# This network must be unused in your network infrastructure!
kube_pods_subnet: 10.233.64.0/18
# internal network node size allocation (optional). This is the size allocated
# to each node on your network. With these defaults you should have
# room for 4096 nodes with 254 pods per node.
kube_network_node_prefix: 24
# The port the API Server will be listening on.
kube_apiserver_ip: "{{ kube_service_addresses|ipaddr('net')|ipaddr(1)|ipaddr('address') }}"
kube_apiserver_port: 6443 # (https)
# kube_apiserver_insecure_port: 8080 # (http)
# Set to 0 to disable insecure port - Requires RBAC in authorization_modes and kube_api_anonymous_auth: true
kube_apiserver_insecure_port: 0 # (disabled)
# Kube-proxy proxyMode configuration.
# Can be ipvs, iptables
#kube_proxy_mode: ipvs
kube_proxy_mode: iptables
# configure arp_ignore and arp_announce to avoid answering ARP queries from kube-ipvs0 interface
# must be set to true for MetalLB to work
kube_proxy_strict_arp: false
# A string slice of values which specify the addresses to use for NodePorts.
# Values may be valid IP blocks (e.g. 1.2.3.0/24, 1.2.3.4/32).
# The default empty string slice ([]) means to use all local addresses.
# kube_proxy_nodeport_addresses_cidr is retained for legacy config
kube_proxy_nodeport_addresses: >-
{%- if kube_proxy_nodeport_addresses_cidr is defined -%}
[{{ kube_proxy_nodeport_addresses_cidr }}]
{%- else -%}
[]
{%- endif -%}
# If non-empty, will use this string as identification instead of the actual hostname
# kube_override_hostname: >-
# {%- if cloud_provider is defined and cloud_provider in [ 'aws' ] -%}
# {%- else -%}
# {{ inventory_hostname }}
# {%- endif -%}
## Encrypting Secret Data at Rest (experimental)
kube_encrypt_secret_data: false
# DNS configuration.
# Kubernetes cluster name, also will be used as DNS domain
cluster_name: cluster.local
# Subdomains of DNS domain to be resolved via /etc/resolv.conf for hostnet pods
ndots: 2
# Can be coredns, coredns_dual, manual or none
dns_mode: coredns
# Set manual server if using a custom cluster DNS server
# manual_dns_server: 10.x.x.x
# Enable nodelocal dns cache
enable_nodelocaldns: true
nodelocaldns_ip: 169.254.25.10
nodelocaldns_health_port: 9254
# Enable k8s_external plugin for CoreDNS
enable_coredns_k8s_external: false
coredns_k8s_external_zone: k8s_external.local
# Enable endpoint_pod_names option for kubernetes plugin
enable_coredns_k8s_endpoint_pod_names: false
# Can be docker_dns, host_resolvconf or none
resolvconf_mode: docker_dns
# Deploy netchecker app to verify DNS resolve as an HTTP service
deploy_netchecker: false
# Ip address of the kubernetes skydns service
skydns_server: "{{ kube_service_addresses|ipaddr('net')|ipaddr(3)|ipaddr('address') }}"
skydns_server_secondary: "{{ kube_service_addresses|ipaddr('net')|ipaddr(4)|ipaddr('address') }}"
dns_domain: "{{ cluster_name }}"
## Container runtime
## docker for docker, crio for cri-o and containerd for containerd.
container_manager: docker
## Settings for containerized control plane (etcd/kubelet/secrets)
etcd_deployment_type: docker
kubelet_deployment_type: host
helm_deployment_type: host
# Enable kubeadm experimental control plane
kubeadm_control_plane: false
kubeadm_certificate_key: "{{ lookup('password', credentials_dir + '/kubeadm_certificate_key.creds length=64 chars=hexdigits') | lower }}"
# K8s image pull policy (imagePullPolicy)
k8s_image_pull_policy: IfNotPresent
# audit log for kubernetes
kubernetes_audit: false
# dynamic kubelet configuration
dynamic_kubelet_configuration: false
# define kubelet config dir for dynamic kubelet
# kubelet_config_dir:
default_kubelet_config_dir: "{{ kube_config_dir }}/dynamic_kubelet_dir"
dynamic_kubelet_configuration_dir: "{{ kubelet_config_dir | default(default_kubelet_config_dir) }}"
# pod security policy (RBAC must be enabled either by having 'RBAC' in authorization_modes or kubeadm enabled)
podsecuritypolicy_enabled: false
# Make a copy of kubeconfig on the host that runs Ansible in {{ inventory_dir }}/artifacts
# kubeconfig_localhost: false
# Download kubectl onto the host that runs Ansible in {{ bin_dir }}
# kubectl_localhost: false
# Enable creation of QoS cgroup hierarchy, if true top level QoS and pod cgroups are created. (default true)
# kubelet_cgroups_per_qos: true
# A comma separated list of levels of node allocatable enforcement to be enforced by kubelet.
# Acceptable options are 'pods', 'system-reserved', 'kube-reserved' and ''. Default is "".
# kubelet_enforce_node_allocatable: pods
## Optionally reserve resources for OS system daemons.
# system_reserved: true
## Uncomment to override default values
# system_memory_reserved: 512M
# system_cpu_reserved: 500m
## Reservation for master hosts
# system_master_memory_reserved: 256M
# system_master_cpu_reserved: 250m
# An alternative flexvolume plugin directory
# kubelet_flexvolumes_plugins_dir: /usr/libexec/kubernetes/kubelet-plugins/volume/exec
## Supplementary addresses that can be added in kubernetes ssl keys.
## That can be useful for example to setup a keepalived virtual IP
# supplementary_addresses_in_ssl_keys: [10.0.0.1, 10.0.0.2, 10.0.0.3]
## Running on top of openstack vms with cinder enabled may lead to unschedulable pods due to NoVolumeZoneConflict restriction in kube-scheduler.
## See https://github.com/kubernetes-sigs/kubespray/issues/2141
## Set this variable to true to get rid of this issue
volume_cross_zone_attachment: false
# Add Persistent Volumes Storage Class for corresponding cloud provider ( OpenStack is only supported now )
persistent_volumes_enabled: false
## Container Engine Acceleration
## Enable container acceleration feature, for example use gpu acceleration in containers
# nvidia_accelerator_enabled: true
## Nvidia GPU driver install. Install will by done by a (init) pod running as a daemonset.
## Important: if you use Ubuntu then you should set in all.yml 'docker_storage_options: -s overlay2'
## Array with nvida_gpu_nodes, leave empty or comment if you don't want to install drivers.
## Labels and taints won't be set to nodes if they are not in the array.
# nvidia_gpu_nodes:
# - kube-gpu-001
# nvidia_driver_version: "384.111"
## flavor can be tesla or gtx
# nvidia_gpu_flavor: gtx
## NVIDIA driver installer images. Change them if you have trouble accessing gcr.io.
# nvidia_driver_install_centos_container: atzedevries/nvidia-centos-driver-installer:2
# nvidia_driver_install_ubuntu_container: gcr.io/google-containers/ubuntu-nvidia-driver-installer@sha256:7df76a0f0a17294e86f691c81de6bbb7c04a1b4b3d4ea4e7e2cccdc42e1f6d63
## NVIDIA GPU device plugin image.
# nvidia_gpu_device_plugin_container: "k8s.gcr.io/nvidia-gpu-device-plugin@sha256:0842734032018be107fa2490c98156992911e3e1f2a21e059ff0105b07dd8e9e"
[root@kubemaster k8s-cluster]# 위와 같이 kube_proxy_mode를 iptables로 변경합니다.
5. ansible-playbook을 이용한 kubernetes 설치 시작
ansible의 playbook을 다음과 같이 수행하면, Kubernetes의 설치 작업을 시작하게 됩니다.
- ansible-playbook 실행
ex) ansible-playbook --flush-cache -u root -b -i inventory/inventory.ini cluster.yml -v
[root@kubemaster kubespray]# ansible-playbook --flush-cache -u root -b -i inventory/inventory.ini cluster.yml -v
Using /root/kubespray/kubespray/ansible.cfg as config file
PLAY [localhost] ***************************************************************************************************************************************************************************************************************************
TASK [Check ansible version >=2.7.8] *******************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:58 +0900 (0:00:00.261) 0:00:00.261 ********
ok: [localhost] => {
"changed": false,
"msg": "All assertions passed"
}
[WARNING]: Could not match supplied host pattern, ignoring: bastion
PLAY [bastion[0]] **************************************************************************************************************************************************************************************************************************
skipping: no hosts matched
PLAY [k8s-cluster:etcd] ********************************************************************************************************************************************************************************************************************
TASK [download : prep_download | Set a few facts] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:58 +0900 (0:00:00.070) 0:00:00.332 ********
TASK [download : Set image info command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:58 +0900 (0:00:00.039) 0:00:00.371 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:58 +0900 (0:00:00.039) 0:00:00.411 ********
TASK [download : prep_download | Create staging directory on remote node] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.038) 0:00:00.450 ********
TASK [download : prep_download | Create local cache for files and images] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.049) 0:00:00.499 ********
TASK [download : prep_download | On localhost, check if passwordless root is possible] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.035) 0:00:00.534 ********
TASK [download : prep_download | On localhost, check if user has access to docker without using sudo] **************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.035) 0:00:00.570 ********
TASK [download : prep_download | Parse the outputs of the previous commands] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.028) 0:00:00.599 ********
TASK [download : prep_download | Check that local user is in group or can become root] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.051) 0:00:00.651 ********
TASK [download : include_tasks] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.039) 0:00:00.690 ********
TASK [download : download | Get kubeadm binary and list of required images] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.039) 0:00:00.730 ********
TASK [download : download | Download files / images] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.038) 0:00:00.769 ********
TASK [download : download | Sync files / images from ansible host to nodes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.121) 0:00:00.890 ********
TASK [kubespray-defaults : Configure defaults] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.116) 0:00:01.007 ********
ok: [kubeworker] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
ok: [kubemaster] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
TASK [bootstrap-os : Fetch /etc/os-release] ************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.050) 0:00:01.057 ********
ok: [kubeworker] => {"changed": false, "rc": 0, "stderr": "Shared connection to 192.168.56.106 closed.\r\n", "stderr_lines": ["Shared connection to 192.168.56.106 closed."], "stdout": "NAME=\"CentOS Linux\"\r\nVERSION=\"7 (Core)\"\r\nID=\"centos\"\r\nID_LIKE=\"rhel fedora\"\r\nVERSION_ID=\"7\"\r\nPRETTY_NAME=\"CentOS Linux 7 (Core)\"\r\nANSI_COLOR=\"0;31\"\r\nCPE_NAME=\"cpe:/o:centos:centos:7\"\r\nHOME_URL=\"https://www.centos.org/\"\r\nBUG_REPORT_URL=\"https://bugs.centos.org/\"\r\n\r\nCENTOS_MANTISBT_PROJECT=\"CentOS-7\"\r\nCENTOS_MANTISBT_PROJECT_VERSION=\"7\"\r\nREDHAT_SUPPORT_PRODUCT=\"centos\"\r\nREDHAT_SUPPORT_PRODUCT_VERSION=\"7\"\r\n\r\n", "stdout_lines": ["NAME=\"CentOS Linux\"", "VERSION=\"7 (Core)\"", "ID=\"centos\"", "ID_LIKE=\"rhel fedora\"", "VERSION_ID=\"7\"", "PRETTY_NAME=\"CentOS Linux 7 (Core)\"", "ANSI_COLOR=\"0;31\"", "CPE_NAME=\"cpe:/o:centos:centos:7\"", "HOME_URL=\"https://www.centos.org/\"", "BUG_REPORT_URL=\"https://bugs.centos.org/\"", "", "CENTOS_MANTISBT_PROJECT=\"CentOS-7\"", "CENTOS_MANTISBT_PROJECT_VERSION=\"7\"", "REDHAT_SUPPORT_PRODUCT=\"centos\"", "REDHAT_SUPPORT_PRODUCT_VERSION=\"7\"", ""]}
ok: [kubemaster] => {"changed": false, "rc": 0, "stderr": "Shared connection to 192.168.56.107 closed.\r\n", "stderr_lines": ["Shared connection to 192.168.56.107 closed."], "stdout": "NAME=\"CentOS Linux\"\r\nVERSION=\"7 (Core)\"\r\nID=\"centos\"\r\nID_LIKE=\"rhel fedora\"\r\nVERSION_ID=\"7\"\r\nPRETTY_NAME=\"CentOS Linux 7 (Core)\"\r\nANSI_COLOR=\"0;31\"\r\nCPE_NAME=\"cpe:/o:centos:centos:7\"\r\nHOME_URL=\"https://www.centos.org/\"\r\nBUG_REPORT_URL=\"https://bugs.centos.org/\"\r\n\r\nCENTOS_MANTISBT_PROJECT=\"CentOS-7\"\r\nCENTOS_MANTISBT_PROJECT_VERSION=\"7\"\r\nREDHAT_SUPPORT_PRODUCT=\"centos\"\r\nREDHAT_SUPPORT_PRODUCT_VERSION=\"7\"\r\n\r\n", "stdout_lines": ["NAME=\"CentOS Linux\"", "VERSION=\"7 (Core)\"", "ID=\"centos\"", "ID_LIKE=\"rhel fedora\"", "VERSION_ID=\"7\"", "PRETTY_NAME=\"CentOS Linux 7 (Core)\"", "ANSI_COLOR=\"0;31\"", "CPE_NAME=\"cpe:/o:centos:centos:7\"", "HOME_URL=\"https://www.centos.org/\"", "BUG_REPORT_URL=\"https://bugs.centos.org/\"", "", "CENTOS_MANTISBT_PROJECT=\"CentOS-7\"", "CENTOS_MANTISBT_PROJECT_VERSION=\"7\"", "REDHAT_SUPPORT_PRODUCT=\"centos\"", "REDHAT_SUPPORT_PRODUCT_VERSION=\"7\"", ""]}
TASK [bootstrap-os : include_tasks] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.100) 0:00:01.158 ********
included: /root/kubespray/kubespray/roles/bootstrap-os/tasks/bootstrap-centos.yml for kubeworker, kubemaster
TASK [bootstrap-os : Check if this is an atomic host] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:48:59 +0900 (0:00:00.107) 0:00:01.266 ********
ok: [kubeworker] => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python"}, "changed": false, "stat": {"exists": false}}
ok: [kubemaster] => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python"}, "changed": false, "stat": {"exists": false}}
TASK [bootstrap-os : Store the fact if this is an atomic host] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:00 +0900 (0:00:00.557) 0:00:01.824 ********
ok: [kubeworker] => {"ansible_facts": {"is_atomic": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"is_atomic": false}, "changed": false}
TASK [bootstrap-os : Check presence of fastestmirror.conf] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:00 +0900 (0:00:00.049) 0:00:01.873 ********
ok: [kubemaster] => {"changed": false, "stat": {"atime": 1570896598.611, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "b37d3a9350228164c2c11fafb0326f9a9dd3c68d", "ctime": 1570896596.882, "dev": 64768, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 4305543, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/x-c", "mode": "0644", "mtime": 1570896596.881, "nlink": 1, "path": "/etc/yum/pluginconf.d/fastestmirror.conf", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 279, "uid": 0, "version": "1574015640", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}}
ok: [kubeworker] => {"changed": false, "stat": {"atime": 1570896598.331, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "b37d3a9350228164c2c11fafb0326f9a9dd3c68d", "ctime": 1570896596.583, "dev": 64768, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 4639490, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/x-c", "mode": "0644", "mtime": 1570896596.581, "nlink": 1, "path": "/etc/yum/pluginconf.d/fastestmirror.conf", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 279, "uid": 0, "version": "1263178695", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}}
TASK [bootstrap-os : Disable fastestmirror plugin if requested] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:00 +0900 (0:00:00.209) 0:00:02.083 ********
ok: [kubeworker] => {"backup": "", "changed": false, "msg": ""}
ok: [kubemaster] => {"backup": "", "changed": false, "msg": ""}
TASK [bootstrap-os : Add proxy to /etc/yum.conf if http_proxy is defined] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:01 +0900 (0:00:00.466) 0:00:02.549 ********
TASK [bootstrap-os : Install libselinux python package] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:01 +0900 (0:00:00.038) 0:00:02.588 ********
ok: [kubeworker] => {"changed": false, "msg": "", "rc": 0, "results": ["libselinux-python-2.5-14.1.el7.x86_64 providing libselinux-python is already installed"]}
ok: [kubemaster] => {"changed": false, "msg": "", "rc": 0, "results": ["libselinux-python-2.5-14.1.el7.x86_64 providing libselinux-python is already installed"]}
TASK [bootstrap-os : include_tasks] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:03 +0900 (0:00:01.898) 0:00:04.487 ********
TASK [bootstrap-os : include_tasks] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:03 +0900 (0:00:00.045) 0:00:04.532 ********
TASK [bootstrap-os : include_tasks] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:03 +0900 (0:00:00.038) 0:00:04.571 ********
TASK [bootstrap-os : include_tasks] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:03 +0900 (0:00:00.041) 0:00:04.613 ********
TASK [bootstrap-os : include_tasks] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:03 +0900 (0:00:00.047) 0:00:04.661 ********
TASK [bootstrap-os : include_tasks] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:03 +0900 (0:00:00.040) 0:00:04.702 ********
TASK [bootstrap-os : Create remote_tmp for it is used by another module] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:03 +0900 (0:00:00.042) 0:00:04.745 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0700", "owner": "root", "path": "/root/.ansible/tmp", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 59, "state": "directory", "uid": 0}
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0700", "owner": "root", "path": "/root/.ansible/tmp", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 94, "state": "directory", "uid": 0}
TASK [bootstrap-os : Gather host facts to get ansible_os_family] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:03 +0900 (0:00:00.529) 0:00:05.274 ********
ok: [kubeworker]
ok: [kubemaster]
TASK [bootstrap-os : Assign inventory name to unconfigured hostnames (non-CoreOS, Suse and ClearLinux)] ************************************************************************************************************************************
Sunday 13 October 2019 01:49:04 +0900 (0:00:00.324) 0:00:05.598 ********
ok: [kubeworker] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "kubeworker", "ansible_hostname": "kubeworker", "ansible_nodename": "kubeworker"}, "changed": false, "name": "kubeworker"}
ok: [kubemaster] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "kubemaster", "ansible_hostname": "kubemaster", "ansible_nodename": "kubemaster"}, "changed": false, "name": "kubemaster"}
TASK [bootstrap-os : Assign inventory name to unconfigured hostnames (CoreOS, Suse and ClearLinux only)] ***********************************************************************************************************************************
Sunday 13 October 2019 01:49:05 +0900 (0:00:01.041) 0:00:06.640 ********
TASK [bootstrap-os : Update hostname fact (CoreOS, Suse and ClearLinux only)] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:05 +0900 (0:00:00.062) 0:00:06.702 ********
TASK [bootstrap-os : Install ceph-commmon package] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:05 +0900 (0:00:00.060) 0:00:06.763 ********
TASK [bootstrap-os : Ensure bash_completion.d folder exists] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:05 +0900 (0:00:00.061) 0:00:06.824 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/bash_completion.d/", "secontext": "system_u:object_r:etc_t:s0", "size": 55, "state": "directory", "uid": 0}
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/bash_completion.d/", "secontext": "system_u:object_r:etc_t:s0", "size": 68, "state": "directory", "uid": 0}
PLAY [k8s-cluster:etcd] ********************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] *********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:05 +0900 (0:00:00.443) 0:00:07.268 ********
ok: [kubeworker]
ok: [kubemaster]
TASK [download : prep_download | Set a few facts] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:06 +0900 (0:00:00.946) 0:00:08.215 ********
TASK [download : Set image info command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:06 +0900 (0:00:00.136) 0:00:08.351 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:07 +0900 (0:00:00.134) 0:00:08.485 ********
TASK [download : prep_download | Create staging directory on remote node] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:07 +0900 (0:00:00.124) 0:00:08.610 ********
TASK [download : prep_download | Create local cache for files and images] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:07 +0900 (0:00:00.132) 0:00:08.742 ********
TASK [download : prep_download | On localhost, check if passwordless root is possible] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:49:07 +0900 (0:00:00.081) 0:00:08.824 ********
TASK [download : prep_download | On localhost, check if user has access to docker without using sudo] **************************************************************************************************************************************
Sunday 13 October 2019 01:49:07 +0900 (0:00:00.091) 0:00:08.915 ********
TASK [download : prep_download | Parse the outputs of the previous commands] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:07 +0900 (0:00:00.073) 0:00:08.989 ********
TASK [download : prep_download | Check that local user is in group or can become root] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:49:07 +0900 (0:00:00.148) 0:00:09.138 ********
TASK [download : include_tasks] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:07 +0900 (0:00:00.129) 0:00:09.268 ********
TASK [download : download | Get kubeadm binary and list of required images] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:07 +0900 (0:00:00.128) 0:00:09.397 ********
TASK [download : download | Download files / images] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:08 +0900 (0:00:00.136) 0:00:09.534 ********
TASK [download : download | Sync files / images from ansible host to nodes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:09 +0900 (0:00:01.359) 0:00:10.894 ********
TASK [kubespray-defaults : Configure defaults] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:10 +0900 (0:00:01.348) 0:00:12.243 ********
ok: [kubeworker] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
ok: [kubemaster] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
TASK [adduser : User | Create User Group] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:10 +0900 (0:00:00.155) 0:00:12.399 ********
ok: [kubemaster] => {"changed": false, "gid": 994, "name": "kube-cert", "state": "present", "system": true}
ok: [kubeworker] => {"changed": false, "gid": 994, "name": "kube-cert", "state": "present", "system": true}
TASK [adduser : User | Create User] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:11 +0900 (0:00:00.623) 0:00:13.022 ********
ok: [kubeworker] => {"append": false, "changed": false, "comment": "Kubernetes user", "group": 994, "home": "/home/kube", "move_home": false, "name": "kube", "shell": "/sbin/nologin", "state": "present", "uid": 997}
ok: [kubemaster] => {"append": false, "changed": false, "comment": "Kubernetes user", "group": 994, "home": "/home/kube", "move_home": false, "name": "kube", "shell": "/sbin/nologin", "state": "present", "uid": 997}
TASK [kubernetes/preinstall : Remove swapfile from /etc/fstab] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:12 +0900 (0:00:00.730) 0:00:13.752 ********
ok: [kubeworker] => (item=swap) => {"ansible_loop_var": "item", "changed": false, "dump": "0", "fstab": "/etc/fstab", "fstype": "swap", "item": "swap", "name": "swap", "opts": "defaults", "passno": "0"}
ok: [kubemaster] => (item=swap) => {"ansible_loop_var": "item", "changed": false, "dump": "0", "fstab": "/etc/fstab", "fstype": "swap", "item": "swap", "name": "swap", "opts": "defaults", "passno": "0"}
ok: [kubemaster] => (item=none) => {"ansible_loop_var": "item", "changed": false, "dump": "0", "fstab": "/etc/fstab", "fstype": "swap", "item": "none", "name": "none", "opts": "defaults", "passno": "0"}
ok: [kubeworker] => (item=none) => {"ansible_loop_var": "item", "changed": false, "dump": "0", "fstab": "/etc/fstab", "fstype": "swap", "item": "none", "name": "none", "opts": "defaults", "passno": "0"}
TASK [kubernetes/preinstall : check swap] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:13 +0900 (0:00:00.820) 0:00:14.573 ********
ok: [kubemaster] => {"changed": false, "cmd": ["/sbin/swapon", "-s"], "delta": "0:00:00.017323", "end": "2019-10-13 01:49:13.633769", "rc": 0, "start": "2019-10-13 01:49:13.616446", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
ok: [kubeworker] => {"changed": false, "cmd": ["/sbin/swapon", "-s"], "delta": "0:00:00.002776", "end": "2019-10-13 01:49:13.360675", "rc": 0, "start": "2019-10-13 01:49:13.357899", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
TASK [kubernetes/preinstall : Disable swap] ************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:13 +0900 (0:00:00.575) 0:00:15.148 ********
TASK [kubernetes/preinstall : Stop if either kube-master, kube-node or etcd is empty] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:49:13 +0900 (0:00:00.145) 0:00:15.294 ********
ok: [kubeworker] => (item=kube-master) => {
"ansible_loop_var": "item",
"changed": false,
"item": "kube-master",
"msg": "All assertions passed"
}
ok: [kubeworker] => (item=kube-node) => {
"ansible_loop_var": "item",
"changed": false,
"item": "kube-node",
"msg": "All assertions passed"
}
ok: [kubeworker] => (item=etcd) => {
"ansible_loop_var": "item",
"changed": false,
"item": "etcd",
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if non systemd OS type] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:14 +0900 (0:00:00.244) 0:00:15.538 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
ok: [kubemaster] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if unknown OS] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:14 +0900 (0:00:00.164) 0:00:15.703 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
ok: [kubemaster] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if unknown network plugin] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:14 +0900 (0:00:00.180) 0:00:15.883 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
ok: [kubemaster] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if incompatible network plugin and cloudprovider] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:49:14 +0900 (0:00:00.177) 0:00:16.061 ********
TASK [kubernetes/preinstall : Stop if unsupported version of Kubernetes] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:14 +0900 (0:00:00.160) 0:00:16.221 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
ok: [kubemaster] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if known booleans are set as strings (Use JSON format on CLI: -e "{'key': true }")] *********************************************************************************************************************
Sunday 13 October 2019 01:49:14 +0900 (0:00:00.179) 0:00:16.401 ********
ok: [kubeworker] => (item={'name': 'download_run_once', 'value': False}) => {
"ansible_loop_var": "item",
"changed": false,
"item": {
"name": "download_run_once",
"value": false
},
"msg": "All assertions passed"
}
ok: [kubeworker] => (item={'name': 'deploy_netchecker', 'value': False}) => {
"ansible_loop_var": "item",
"changed": false,
"item": {
"name": "deploy_netchecker",
"value": false
},
"msg": "All assertions passed"
}
ok: [kubeworker] => (item={'name': 'download_always_pull', 'value': False}) => {
"ansible_loop_var": "item",
"changed": false,
"item": {
"name": "download_always_pull",
"value": false
},
"msg": "All assertions passed"
}
ok: [kubeworker] => (item={'name': 'helm_enabled', 'value': False}) => {
"ansible_loop_var": "item",
"changed": false,
"item": {
"name": "helm_enabled",
"value": false
},
"msg": "All assertions passed"
}
ok: [kubeworker] => (item={'name': 'openstack_lbaas_enabled', 'value': False}) => {
"ansible_loop_var": "item",
"changed": false,
"item": {
"name": "openstack_lbaas_enabled",
"value": false
},
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if even number of etcd hosts] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:15 +0900 (0:00:00.354) 0:00:16.756 ********
ok: [kubemaster] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if memory is too small for masters] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:15 +0900 (0:00:00.168) 0:00:16.924 ********
ok: [kubemaster] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if memory is too small for nodes] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:15 +0900 (0:00:00.178) 0:00:17.103 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Guarantee that enough network address space is available for all pods] ***************************************************************************************************************************************
Sunday 13 October 2019 01:49:15 +0900 (0:00:00.134) 0:00:17.237 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if ip var does not match local ips] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:15 +0900 (0:00:00.141) 0:00:17.378 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
ok: [kubemaster] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if access_ip is not pingable] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:16 +0900 (0:00:00.177) 0:00:17.556 ********
TASK [kubernetes/preinstall : Stop if RBAC is not enabled when dashboard is enabled] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:49:16 +0900 (0:00:00.131) 0:00:17.688 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
ok: [kubemaster] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if RBAC is not enabled when OCI cloud controller is enabled] ********************************************************************************************************************************************
Sunday 13 October 2019 01:49:16 +0900 (0:00:00.183) 0:00:17.871 ********
TASK [kubernetes/preinstall : Stop if RBAC and anonymous-auth are not enabled when insecure port is disabled] ******************************************************************************************************************************
Sunday 13 October 2019 01:49:16 +0900 (0:00:00.145) 0:00:18.016 ********
ok: [kubemaster] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if kernel version is too low] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:16 +0900 (0:00:00.194) 0:00:18.211 ********
TASK [kubernetes/preinstall : Stop if bad hostname] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:16 +0900 (0:00:00.157) 0:00:18.369 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
ok: [kubemaster] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : check cloud_provider value] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:17 +0900 (0:00:00.174) 0:00:18.544 ********
TASK [kubernetes/preinstall : Ensure minimum calico version] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:17 +0900 (0:00:00.129) 0:00:18.673 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Get current version of calico cluster version] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:17 +0900 (0:00:00.114) 0:00:18.788 ********
ok: [kubeworker] => {"changed": false, "cmd": "/usr/local/bin/calicoctl.sh version | grep 'Cluster Version:' | awk '{ print $3}'", "delta": "0:00:00.004006", "end": "2019-10-13 01:49:17.255581", "failed_when_result": false, "rc": 0, "start": "2019-10-13 01:49:17.251575", "stderr": "/bin/sh: /usr/local/bin/calicoctl.sh: No such file or directory", "stderr_lines": ["/bin/sh: /usr/local/bin/calicoctl.sh: No such file or directory"], "stdout": "", "stdout_lines": []}
TASK [kubernetes/preinstall : Check that calico version is enough for upgrade] *************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:17 +0900 (0:00:00.261) 0:00:19.049 ********
TASK [kubernetes/preinstall : Check that cluster_id is set if calico_rr enabled] ***********************************************************************************************************************************************************
Sunday 13 October 2019 01:49:17 +0900 (0:00:00.110) 0:00:19.160 ********
TASK [kubernetes/preinstall : Check that calico_rr nodes are in k8s-cluster group] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:49:17 +0900 (0:00:00.091) 0:00:19.252 ********
TASK [kubernetes/preinstall : Check that kube_service_addresses is a network range] ********************************************************************************************************************************************************
Sunday 13 October 2019 01:49:17 +0900 (0:00:00.134) 0:00:19.386 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Check that kube_pods_subnet is a network range] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:18 +0900 (0:00:00.118) 0:00:19.505 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Check that kube_pods_subnet does not collide with kube_service_addresses] ************************************************************************************************************************************
Sunday 13 October 2019 01:49:18 +0900 (0:00:00.127) 0:00:19.633 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if unknown dns mode] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:18 +0900 (0:00:00.122) 0:00:19.755 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if unknown kube proxy mode] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:18 +0900 (0:00:00.116) 0:00:19.872 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if vault is chose] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:18 +0900 (0:00:00.131) 0:00:20.003 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if unknown cert_management] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:18 +0900 (0:00:00.125) 0:00:20.129 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if unknown resolvconf_mode] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:18 +0900 (0:00:00.111) 0:00:20.241 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [kubernetes/preinstall : Stop if kubeadm etcd mode is enabled but experimental control plane is not] **********************************************************************************************************************************
Sunday 13 October 2019 01:49:18 +0900 (0:00:00.135) 0:00:20.376 ********
TASK [kubernetes/preinstall : Pre-upgrade | check if old credential dir exists] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:19 +0900 (0:00:00.151) 0:00:20.528 ********
ok: [kubeworker -> localhost] => {"changed": false, "stat": {"exists": false}}
TASK [kubernetes/preinstall : Pre-upgrade | check if new credential dir exists] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:19 +0900 (0:00:00.334) 0:00:20.862 ********
TASK [kubernetes/preinstall : Pre-upgrade | move data from old credential dir to new] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:49:19 +0900 (0:00:00.085) 0:00:20.947 ********
TASK [kubernetes/preinstall : set architecture_groups] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:19 +0900 (0:00:00.078) 0:00:21.026 ********
ok: [kubeworker] => {"ansible_facts": {"architecture_groups": {"aarch64": "arm64", "armv7l": "arm", "x86_64": "amd64"}}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"architecture_groups": {"aarch64": "arm64", "armv7l": "arm", "x86_64": "amd64"}}, "changed": false}
TASK [kubernetes/preinstall : ansible_architecture_rename] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:19 +0900 (0:00:00.160) 0:00:21.186 ********
ok: [kubeworker] => {"ansible_facts": {"host_architecture": "amd64"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"host_architecture": "amd64"}, "changed": false}
TASK [kubernetes/preinstall : Force binaries directory for Container Linux by CoreOS] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:49:19 +0900 (0:00:00.171) 0:00:21.357 ********
TASK [kubernetes/preinstall : check if atomic host] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:20 +0900 (0:00:00.130) 0:00:21.487 ********
ok: [kubeworker] => {"changed": false, "stat": {"exists": false}}
ok: [kubemaster] => {"changed": false, "stat": {"exists": false}}
TASK [kubernetes/preinstall : set is_atomic] ***********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:20 +0900 (0:00:00.296) 0:00:21.784 ********
ok: [kubeworker] => {"ansible_facts": {"is_atomic": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"is_atomic": false}, "changed": false}
TASK [kubernetes/preinstall : set kube_cert_group on atomic hosts] *************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:20 +0900 (0:00:00.163) 0:00:21.947 ********
TASK [kubernetes/preinstall : check resolvconf] ********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:20 +0900 (0:00:00.129) 0:00:22.077 ********
ok: [kubeworker] => {"changed": false, "cmd": "which resolvconf", "delta": "0:00:00.003075", "end": "2019-10-13 01:49:20.554818", "failed_when_result": false, "msg": "non-zero return code", "rc": 1, "start": "2019-10-13 01:49:20.551743", "stderr": "which: no resolvconf in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)", "stderr_lines": ["which: no resolvconf in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)"], "stdout": "", "stdout_lines": []}
ok: [kubemaster] => {"changed": false, "cmd": "which resolvconf", "delta": "0:00:00.003880", "end": "2019-10-13 01:49:20.877410", "failed_when_result": false, "msg": "non-zero return code", "rc": 1, "start": "2019-10-13 01:49:20.873530", "stderr": "which: no resolvconf in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)", "stderr_lines": ["which: no resolvconf in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)"], "stdout": "", "stdout_lines": []}
TASK [kubernetes/preinstall : set dns facts] ***********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:20 +0900 (0:00:00.299) 0:00:22.376 ********
ok: [kubeworker] => {"ansible_facts": {"bogus_domains": "cluster.local.default.svc.cluster.local./default.svc.cluster.local.default.svc.cluster.local./com.default.svc.cluster.local./cluster.local.svc.cluster.local./svc.cluster.local.svc.cluster.local./com.svc.cluster.local./", "cloud_resolver": [], "resolvconf": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"bogus_domains": "cluster.local.default.svc.cluster.local./default.svc.cluster.local.default.svc.cluster.local./com.default.svc.cluster.local./cluster.local.svc.cluster.local./svc.cluster.local.svc.cluster.local./com.svc.cluster.local./", "cloud_resolver": [], "resolvconf": false}, "changed": false}
TASK [kubernetes/preinstall : check if kubelet is configured] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:21 +0900 (0:00:00.177) 0:00:22.553 ********
ok: [kubeworker] => {"changed": false, "stat": {"exists": false}}
ok: [kubemaster] => {"changed": false, "stat": {"exists": false}}
TASK [kubernetes/preinstall : check if early DNS configuration stage] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:21 +0900 (0:00:00.309) 0:00:22.863 ********
ok: [kubeworker] => {"ansible_facts": {"dns_early": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"dns_early": true}, "changed": false}
TASK [kubernetes/preinstall : target resolv.conf files] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:21 +0900 (0:00:00.154) 0:00:23.018 ********
ok: [kubeworker] => {"ansible_facts": {"base": "", "head": "", "resolvconffile": "/etc/resolv.conf"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"base": "", "head": "", "resolvconffile": "/etc/resolv.conf"}, "changed": false}
TASK [kubernetes/preinstall : target temporary resolvconf cloud init file (Container Linux by CoreOS)] *************************************************************************************************************************************
Sunday 13 October 2019 01:49:21 +0900 (0:00:00.175) 0:00:23.193 ********
TASK [kubernetes/preinstall : check if /etc/dhclient.conf exists] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:21 +0900 (0:00:00.126) 0:00:23.319 ********
ok: [kubeworker] => {"changed": false, "stat": {"exists": false}}
ok: [kubemaster] => {"changed": false, "stat": {"exists": false}}
TASK [kubernetes/preinstall : target dhclient conf file for /etc/dhclient.conf] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:22 +0900 (0:00:00.282) 0:00:23.602 ********
TASK [kubernetes/preinstall : check if /etc/dhcp/dhclient.conf exists] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:22 +0900 (0:00:00.137) 0:00:23.739 ********
ok: [kubeworker] => {"changed": false, "stat": {"exists": false}}
ok: [kubemaster] => {"changed": false, "stat": {"exists": false}}
TASK [kubernetes/preinstall : target dhclient conf file for /etc/dhcp/dhclient.conf] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:49:22 +0900 (0:00:00.302) 0:00:24.041 ********
TASK [kubernetes/preinstall : target dhclient hook file for Red Hat family] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:22 +0900 (0:00:00.125) 0:00:24.167 ********
ok: [kubeworker] => {"ansible_facts": {"dhclienthookfile": "/etc/dhcp/dhclient.d/zdnsupdate.sh"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"dhclienthookfile": "/etc/dhcp/dhclient.d/zdnsupdate.sh"}, "changed": false}
TASK [kubernetes/preinstall : target dhclient hook file for Debian family] *****************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:22 +0900 (0:00:00.179) 0:00:24.347 ********
TASK [kubernetes/preinstall : generate search domains to resolvconf] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:23 +0900 (0:00:00.139) 0:00:24.486 ********
ok: [kubeworker] => {"ansible_facts": {"domainentry": "domain cluster.local", "searchentries": "search default.svc.cluster.local svc.cluster.local", "supersede_domain": "supersede domain-name \"cluster.local\";", "supersede_search": "supersede domain-search \"default.svc.cluster.local\", \"svc.cluster.local\";"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"domainentry": "domain cluster.local", "searchentries": "search default.svc.cluster.local svc.cluster.local", "supersede_domain": "supersede domain-name \"cluster.local\";", "supersede_search": "supersede domain-search \"default.svc.cluster.local\", \"svc.cluster.local\";"}, "changed": false}
TASK [kubernetes/preinstall : pick coredns cluster IP or default resolver] *****************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:23 +0900 (0:00:00.162) 0:00:24.649 ********
ok: [kubeworker] => {"ansible_facts": {"coredns_server": []}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"coredns_server": []}, "changed": false}
TASK [kubernetes/preinstall : generate nameservers to resolvconf] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:23 +0900 (0:00:00.175) 0:00:24.824 ********
ok: [kubeworker] => {"ansible_facts": {"nameserverentries": "nameserver ", "supersede_nameserver": "supersede domain-name-servers ;"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"nameserverentries": "nameserver ", "supersede_nameserver": "supersede domain-name-servers ;"}, "changed": false}
TASK [kubernetes/preinstall : gather os specific variables] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:23 +0900 (0:00:00.192) 0:00:25.016 ********
ok: [kubeworker] => (item=/root/kubespray/kubespray/roles/kubernetes/preinstall/vars/../vars/centos.yml) => {"ansible_facts": {"required_pkgs": ["{{ ( (ansible_facts.distribution_major_version | default(0) | int) < 8) | ternary('libselinux-python','python3-libselinux') }}", "device-mapper-libs", "ebtables", "nss"]}, "ansible_included_var_files": ["/root/kubespray/kubespray/roles/kubernetes/preinstall/vars/centos.yml"], "ansible_loop_var": "item", "changed": false, "item": "/root/kubespray/kubespray/roles/kubernetes/preinstall/vars/../vars/centos.yml"}
ok: [kubemaster] => (item=/root/kubespray/kubespray/roles/kubernetes/preinstall/vars/../vars/centos.yml) => {"ansible_facts": {"required_pkgs": ["{{ ( (ansible_facts.distribution_major_version | default(0) | int) < 8) | ternary('libselinux-python','python3-libselinux') }}", "device-mapper-libs", "ebtables", "nss"]}, "ansible_included_var_files": ["/root/kubespray/kubespray/roles/kubernetes/preinstall/vars/centos.yml"], "ansible_loop_var": "item", "changed": false, "item": "/root/kubespray/kubespray/roles/kubernetes/preinstall/vars/../vars/centos.yml"}
TASK [kubernetes/preinstall : override kube_proxy_mode to ipvs if kube_proxy_remove is set, as ipvs won't require kube-proxy cleanup when kube-proxy daemonset gets deleted] ***************************************************************
Sunday 13 October 2019 01:49:23 +0900 (0:00:00.197) 0:00:25.214 ********
TASK [kubernetes/preinstall : set etcd vars if using kubeadm mode] *************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:23 +0900 (0:00:00.120) 0:00:25.335 ********
TASK [kubernetes/preinstall : check /usr readonly] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:24 +0900 (0:00:00.122) 0:00:25.458 ********
ok: [kubeworker] => {"changed": false, "stat": {"atime": 1523422795.0, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 0, "charset": "binary", "ctime": 1568211001.8961053, "dev": 64768, "device_type": 0, "executable": true, "exists": true, "gid": 0, "gr_name": "root", "inode": 8409528, "isblk": false, "ischr": false, "isdir": true, "isfifo": false, "isgid": false, "islnk": false, "isreg": false, "issock": false, "isuid": false, "mimetype": "inode/directory", "mode": "0755", "mtime": 1568211001.8961053, "nlink": 13, "path": "/usr", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 155, "uid": 0, "version": "174472261", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}}
ok: [kubemaster] => {"changed": false, "stat": {"atime": 1569252953.707, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 0, "charset": "binary", "ctime": 1568209711.5469654, "dev": 64768, "device_type": 0, "executable": true, "exists": true, "gid": 0, "gr_name": "root", "inode": 8409528, "isblk": false, "ischr": false, "isdir": true, "isfifo": false, "isgid": false, "islnk": false, "isreg": false, "issock": false, "isuid": false, "mimetype": "inode/directory", "mode": "0755", "mtime": 1568209711.5469654, "nlink": 13, "path": "/usr", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 155, "uid": 0, "version": "18446744072689059142", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}}
TASK [kubernetes/preinstall : set alternate flexvolume path] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:24 +0900 (0:00:00.302) 0:00:25.760 ********
TASK [kubernetes/preinstall : Create kubernetes directories] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:24 +0900 (0:00:00.132) 0:00:25.893 ********
ok: [kubeworker] => (item=/usr/local/bin) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/usr/local/bin", "mode": "0755", "owner": "kube", "path": "/usr/local/bin", "secontext": "system_u:object_r:bin_t:s0", "size": 58, "state": "directory", "uid": 997}
ok: [kubemaster] => (item=/usr/local/bin) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/usr/local/bin", "mode": "0755", "owner": "kube", "path": "/usr/local/bin", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 997}
ok: [kubeworker] => (item=/etc/kubernetes) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/kubernetes", "mode": "0755", "owner": "kube", "path": "/etc/kubernetes", "secontext": "system_u:object_r:etc_t:s0", "size": 45, "state": "directory", "uid": 997}
ok: [kubemaster] => (item=/etc/kubernetes) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/kubernetes", "mode": "0755", "owner": "kube", "path": "/etc/kubernetes", "secontext": "system_u:object_r:etc_t:s0", "size": 72, "state": "directory", "uid": 997}
ok: [kubeworker] => (item=/etc/kubernetes/ssl) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/kubernetes/ssl", "mode": "0755", "owner": "kube", "path": "/etc/kubernetes/ssl", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 6, "state": "directory", "uid": 997}
ok: [kubemaster] => (item=/etc/kubernetes/ssl) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/kubernetes/ssl", "mode": "0755", "owner": "kube", "path": "/etc/kubernetes/ssl", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 6, "state": "directory", "uid": 997}
ok: [kubeworker] => (item=/etc/kubernetes/manifests) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/kubernetes/manifests", "mode": "0755", "owner": "kube", "path": "/etc/kubernetes/manifests", "secontext": "system_u:object_r:etc_t:s0", "size": 6, "state": "directory", "uid": 997}
ok: [kubemaster] => (item=/etc/kubernetes/manifests) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/kubernetes/manifests", "mode": "0755", "owner": "kube", "path": "/etc/kubernetes/manifests", "secontext": "system_u:object_r:etc_t:s0", "size": 6, "state": "directory", "uid": 997}
ok: [kubeworker] => (item=/usr/local/bin/kubernetes-scripts) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/usr/local/bin/kubernetes-scripts", "mode": "0755", "owner": "kube", "path": "/usr/local/bin/kubernetes-scripts", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 6, "state": "directory", "uid": 997}
ok: [kubemaster] => (item=/usr/local/bin/kubernetes-scripts) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/usr/local/bin/kubernetes-scripts", "mode": "0755", "owner": "kube", "path": "/usr/local/bin/kubernetes-scripts", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 6, "state": "directory", "uid": 997}
ok: [kubeworker] => (item=/usr/libexec/kubernetes/kubelet-plugins/volume/exec) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/usr/libexec/kubernetes/kubelet-plugins/volume/exec", "mode": "0755", "owner": "kube", "path": "/usr/libexec/kubernetes/kubelet-plugins/volume/exec", "secontext": "system_u:object_r:bin_t:s0", "size": 6, "state": "directory", "uid": 997}
ok: [kubemaster] => (item=/usr/libexec/kubernetes/kubelet-plugins/volume/exec) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/usr/libexec/kubernetes/kubelet-plugins/volume/exec", "mode": "0755", "owner": "kube", "path": "/usr/libexec/kubernetes/kubelet-plugins/volume/exec", "secontext": "system_u:object_r:bin_t:s0", "size": 6, "state": "directory", "uid": 997}
TASK [kubernetes/preinstall : Check if kubernetes kubeadm compat cert dir exists] **********************************************************************************************************************************************************
Sunday 13 October 2019 01:49:25 +0900 (0:00:01.428) 0:00:27.321 ********
ok: [kubeworker] => {"changed": false, "stat": {"atime": 1570877753.192, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 0, "charset": "binary", "ctime": 1570877753.192, "dev": 64768, "device_type": 0, "executable": true, "exists": true, "gid": 0, "gr_name": "root", "inode": 395681, "isblk": false, "ischr": false, "isdir": true, "isfifo": false, "isgid": false, "islnk": false, "isreg": false, "issock": false, "isuid": false, "mimetype": "inode/directory", "mode": "0755", "mtime": 1570877753.192, "nlink": 2, "path": "/etc/kubernetes/pki", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 6, "uid": 0, "version": "18446744072143035779", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}}
ok: [kubemaster] => {"changed": false, "stat": {"atime": 1569252953.282, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 0, "charset": "binary", "ctime": 1569253101.459, "dev": 64768, "device_type": 0, "executable": true, "exists": true, "gid": 0, "gr_name": "root", "inode": 747148, "isblk": false, "ischr": false, "isdir": true, "isfifo": false, "isgid": false, "islnk": false, "isreg": false, "issock": false, "isuid": false, "mimetype": "inode/directory", "mode": "0755", "mtime": 1569253101.459, "nlink": 2, "path": "/etc/kubernetes/pki", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 6, "uid": 0, "version": "18446744073221252429", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}}
TASK [kubernetes/preinstall : Create kubernetes kubeadm compat cert dir (kubernetes/kubeadm issue 1498)] ***********************************************************************************************************************************
Sunday 13 October 2019 01:49:26 +0900 (0:00:00.339) 0:00:27.660 ********
TASK [kubernetes/preinstall : Create cni directories] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:26 +0900 (0:00:00.139) 0:00:27.800 ********
ok: [kubeworker] => (item=/etc/cni/net.d) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/cni/net.d", "mode": "0755", "owner": "kube", "path": "/etc/cni/net.d", "secontext": "system_u:object_r:etc_t:s0", "size": 6, "state": "directory", "uid": 997}
ok: [kubemaster] => (item=/etc/cni/net.d) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/cni/net.d", "mode": "0755", "owner": "kube", "path": "/etc/cni/net.d", "secontext": "system_u:object_r:etc_t:s0", "size": 6, "state": "directory", "uid": 997}
ok: [kubeworker] => (item=/opt/cni/bin) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/opt/cni/bin", "mode": "0755", "owner": "kube", "path": "/opt/cni/bin", "secontext": "system_u:object_r:bin_t:s0", "size": 256, "state": "directory", "uid": 997}
ok: [kubemaster] => (item=/opt/cni/bin) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/opt/cni/bin", "mode": "0755", "owner": "kube", "path": "/opt/cni/bin", "secontext": "system_u:object_r:bin_t:s0", "size": 256, "state": "directory", "uid": 997}
ok: [kubeworker] => (item=/var/lib/calico) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/var/lib/calico", "mode": "0755", "owner": "kube", "path": "/var/lib/calico", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 6, "state": "directory", "uid": 997}
ok: [kubemaster] => (item=/var/lib/calico) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/var/lib/calico", "mode": "0755", "owner": "kube", "path": "/var/lib/calico", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 6, "state": "directory", "uid": 997}
TASK [kubernetes/preinstall : Create local volume provisioner directories] *****************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:27 +0900 (0:00:00.752) 0:00:28.553 ********
TASK [kubernetes/preinstall : create temporary resolveconf cloud init file] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:27 +0900 (0:00:00.156) 0:00:28.710 ********
TASK [kubernetes/preinstall : Add domain/search/nameservers/options to resolv.conf] ********************************************************************************************************************************************************
Sunday 13 October 2019 01:49:27 +0900 (0:00:00.139) 0:00:28.849 ********
TASK [kubernetes/preinstall : Remove search/domain/nameserver options before block] ********************************************************************************************************************************************************
Sunday 13 October 2019 01:49:27 +0900 (0:00:00.130) 0:00:28.979 ********
TASK [kubernetes/preinstall : Remove search/domain/nameserver options after block] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:49:27 +0900 (0:00:00.222) 0:00:29.201 ********
TASK [kubernetes/preinstall : get temporary resolveconf cloud init file content] ***********************************************************************************************************************************************************
Sunday 13 October 2019 01:49:27 +0900 (0:00:00.210) 0:00:29.412 ********
TASK [kubernetes/preinstall : persist resolvconf cloud init file] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:28 +0900 (0:00:00.150) 0:00:29.562 ********
TASK [kubernetes/preinstall : Update package management cache (zypper) - SUSE] *************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:28 +0900 (0:00:00.140) 0:00:29.703 ********
TASK [kubernetes/preinstall : Update package management cache (APT)] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:28 +0900 (0:00:00.162) 0:00:29.866 ********
TASK [kubernetes/preinstall : Install python-dnf for latest RedHat versions] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:28 +0900 (0:00:00.132) 0:00:29.999 ********
TASK [kubernetes/preinstall : Install epel-release on RedHat/CentOS] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:28 +0900 (0:00:00.126) 0:00:30.125 ********
TASK [kubernetes/preinstall : Update common_required_pkgs with ipvsadm when kube_proxy_mode is ipvs] ***************************************************************************************************************************************
Sunday 13 October 2019 01:49:28 +0900 (0:00:00.143) 0:00:30.269 ********
ok: [kubeworker] => {"ansible_facts": {"common_required_pkgs": ["openssl", "curl", "rsync", "socat", "unzip", "e2fsprogs", "xfsprogs", "ipvsadm", "ipset"]}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"common_required_pkgs": ["openssl", "curl", "rsync", "socat", "unzip", "e2fsprogs", "xfsprogs", "ipvsadm", "ipset"]}, "changed": false}
TASK [kubernetes/preinstall : Install packages requirements] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:29 +0900 (0:00:00.184) 0:00:30.453 ********
ok: [kubeworker] => {"attempts": 1, "changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing libselinux-python are up to date", "All packages providing device-mapper-libs are up to date", "All packages providing ebtables are up to date", "All packages providing nss are up to date", "All packages providing openssl are up to date", "All packages providing curl are up to date", "All packages providing rsync are up to date", "All packages providing socat are up to date", "All packages providing unzip are up to date", "All packages providing e2fsprogs are up to date", "All packages providing xfsprogs are up to date", "All packages providing ipvsadm are up to date", "All packages providing ipset are up to date", ""]}
ok: [kubemaster] => {"attempts": 1, "changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing libselinux-python are up to date", "All packages providing device-mapper-libs are up to date", "All packages providing ebtables are up to date", "All packages providing nss are up to date", "All packages providing openssl are up to date", "All packages providing curl are up to date", "All packages providing rsync are up to date", "All packages providing socat are up to date", "All packages providing unzip are up to date", "All packages providing e2fsprogs are up to date", "All packages providing xfsprogs are up to date", "All packages providing ipvsadm are up to date", "All packages providing ipset are up to date", ""]}
TASK [kubernetes/preinstall : Install ipvsadm for ClearLinux] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:34 +0900 (0:00:05.553) 0:00:36.007 ********
TASK [kubernetes/preinstall : Confirm selinux deployed] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:34 +0900 (0:00:00.144) 0:00:36.152 ********
ok: [kubeworker] => {"changed": false, "stat": {"atime": 1570896680.39, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "d6afd3111474b2b59faed44e0c597dae55e1d5b5", "ctime": 1570896679.476, "dev": 64768, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 4639507, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1570896679.476, "nlink": 1, "path": "/etc/selinux/config", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 544, "uid": 0, "version": "928273048", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}}
ok: [kubemaster] => {"changed": false, "stat": {"atime": 1570896680.736, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "d6afd3111474b2b59faed44e0c597dae55e1d5b5", "ctime": 1570896679.762, "dev": 64768, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 4305546, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1570896679.762, "nlink": 1, "path": "/etc/selinux/config", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 544, "uid": 0, "version": "241450752", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}}
TASK [kubernetes/preinstall : Set selinux policy] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:35 +0900 (0:00:00.360) 0:00:36.513 ********
ok: [kubeworker] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"}
ok: [kubemaster] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"}
TASK [kubernetes/preinstall : Disable IPv6 DNS lookup] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:36 +0900 (0:00:01.050) 0:00:37.563 ********
TASK [kubernetes/preinstall : Stat sysctl file configuration] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:36 +0900 (0:00:00.137) 0:00:37.701 ********
ok: [kubeworker] => {"changed": false, "stat": {"atime": 1570877664.89, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 0, "charset": "binary", "ctime": 1568211018.2321053, "dev": 64768, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 8556414, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": true, "isreg": false, "issock": false, "isuid": false, "lnk_source": "/etc/sysctl.conf", "lnk_target": "../sysctl.conf", "mimetype": "inode/symlink", "mode": "0777", "mtime": 1568211018.2321053, "nlink": 1, "path": "/etc/sysctl.d/99-sysctl.conf", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 14, "uid": 0, "version": null, "wgrp": true, "woth": true, "writeable": true, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}}
ok: [kubemaster] => {"changed": false, "stat": {"atime": 1570877662.169, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 0, "charset": "binary", "ctime": 1569070088.548, "dev": 64768, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 8564476, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": true, "isreg": false, "issock": false, "isuid": false, "lnk_source": "/etc/sysctl.conf", "lnk_target": "../sysctl.conf", "mimetype": "inode/symlink", "mode": "0777", "mtime": 1569070088.547, "nlink": 1, "path": "/etc/sysctl.d/99-sysctl.conf", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 14, "uid": 0, "version": null, "wgrp": true, "woth": true, "writeable": true, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}}
TASK [kubernetes/preinstall : Change sysctl file path to link source if linked] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:36 +0900 (0:00:00.335) 0:00:38.036 ********
ok: [kubeworker] => {"ansible_facts": {"sysctl_file_path": "/etc/sysctl.conf"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"sysctl_file_path": "/etc/sysctl.conf"}, "changed": false}
TASK [kubernetes/preinstall : Make sure sysctl file path folder exists] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:36 +0900 (0:00:00.205) 0:00:38.241 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc", "secontext": "system_u:object_r:etc_t:s0", "size": 8192, "state": "directory", "uid": 0}
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc", "secontext": "system_u:object_r:etc_t:s0", "size": 8192, "state": "directory", "uid": 0}
TASK [kubernetes/preinstall : Enable ip forwarding] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:37 +0900 (0:00:00.306) 0:00:38.548 ********
[WARNING]: The value 1 (type int) in a string field was converted to u'1' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change.
ok: [kubeworker] => {"changed": false}
ok: [kubemaster] => {"changed": false}
TASK [kubernetes/preinstall : Hosts | populate inventory into hosts file] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:37 +0900 (0:00:00.502) 0:00:39.051 ********
ok: [kubemaster] => {"changed": false, "msg": ""}
ok: [kubeworker] => {"changed": false, "msg": ""}
TASK [kubernetes/preinstall : Hosts | populate kubernetes loadbalancer address into hosts file] ********************************************************************************************************************************************
Sunday 13 October 2019 01:49:38 +0900 (0:00:01.273) 0:00:40.324 ********
TASK [kubernetes/preinstall : Hosts | Retrieve hosts file content] *************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:39 +0900 (0:00:00.147) 0:00:40.472 ********
ok: [kubemaster] => {"changed": false, "content": "MTI3LjAuMC4xIGxvY2FsaG9zdCBsb2NhbGhvc3QubG9jYWxkb21haW4gbG9jYWxob3N0NCBsb2NhbGhvc3Q0LmxvY2FsZG9tYWluNAo6OjEgbG9jYWxob3N0NiBsb2NhbGhvc3Q2LmxvY2FsZG9tYWluNiBsb2NhbGhvc3Q2LmxvY2FsZG9tYWluCjE5Mi4xNjguNTYuMTA3CWt1YmVtYXN0ZXIKMTkyLjE2OC41Ni4xMDYJa3ViZXdvcmtlcgojIEFuc2libGUgaW52ZW50b3J5IGhvc3RzIEJFR0lOCjE5Mi4xNjguNTYuMTA2IGt1YmV3b3JrZXIuY2x1c3Rlci5sb2NhbCBrdWJld29ya2VyCjE5Mi4xNjguNTYuMTA3IGt1YmVtYXN0ZXIuY2x1c3Rlci5sb2NhbCBrdWJlbWFzdGVyCiMgQW5zaWJsZSBpbnZlbnRvcnkgaG9zdHMgRU5ECg==", "encoding": "base64", "source": "/etc/hosts"}
ok: [kubeworker] => {"changed": false, "content": "MTI3LjAuMC4xIGxvY2FsaG9zdCBsb2NhbGhvc3QubG9jYWxkb21haW4gbG9jYWxob3N0NCBsb2NhbGhvc3Q0LmxvY2FsZG9tYWluNAo6OjEgbG9jYWxob3N0NiBsb2NhbGhvc3Q2LmxvY2FsZG9tYWluNiBsb2NhbGhvc3Q2LmxvY2FsZG9tYWluCjE5Mi4xNjguNTYuMTA3ICBrdWJlbWFzdGVyCjE5Mi4xNjguNTYuMTA2ICBrdWJld29ya2VyCiMgQW5zaWJsZSBpbnZlbnRvcnkgaG9zdHMgQkVHSU4KMTkyLjE2OC41Ni4xMDYga3ViZXdvcmtlci5jbHVzdGVyLmxvY2FsIGt1YmV3b3JrZXIKMTkyLjE2OC41Ni4xMDcga3ViZW1hc3Rlci5jbHVzdGVyLmxvY2FsIGt1YmVtYXN0ZXIKIyBBbnNpYmxlIGludmVudG9yeSBob3N0cyBFTkQK", "encoding": "base64", "source": "/etc/hosts"}
TASK [kubernetes/preinstall : Hosts | Extract existing entries for localhost from hosts file] **********************************************************************************************************************************************
Sunday 13 October 2019 01:49:39 +0900 (0:00:00.520) 0:00:40.993 ********
ok: [kubeworker] => (item=127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4) => {"ansible_facts": {"etc_hosts_localhosts_dict": {"127.0.0.1": ["localhost", "localhost.localdomain", "localhost4", "localhost4.localdomain4"]}}, "ansible_loop_var": "item", "changed": false, "item": "127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4"}
ok: [kubemaster] => (item=127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4) => {"ansible_facts": {"etc_hosts_localhosts_dict": {"127.0.0.1": ["localhost", "localhost.localdomain", "localhost4", "localhost4.localdomain4"]}}, "ansible_loop_var": "item", "changed": false, "item": "127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4"}
ok: [kubeworker] => (item=::1 localhost6 localhost6.localdomain6 localhost6.localdomain) => {"ansible_facts": {"etc_hosts_localhosts_dict": {"127.0.0.1": ["localhost", "localhost.localdomain", "localhost4", "localhost4.localdomain4"], "::1": ["localhost6", "localhost6.localdomain6", "localhost6.localdomain"]}}, "ansible_loop_var": "item", "changed": false, "item": "::1 localhost6 localhost6.localdomain6 localhost6.localdomain"}
ok: [kubemaster] => (item=::1 localhost6 localhost6.localdomain6 localhost6.localdomain) => {"ansible_facts": {"etc_hosts_localhosts_dict": {"127.0.0.1": ["localhost", "localhost.localdomain", "localhost4", "localhost4.localdomain4"], "::1": ["localhost6", "localhost6.localdomain6", "localhost6.localdomain"]}}, "ansible_loop_var": "item", "changed": false, "item": "::1 localhost6 localhost6.localdomain6 localhost6.localdomain"}
TASK [kubernetes/preinstall : Hosts | Update target hosts file entries dict with required entries] *****************************************************************************************************************************************
Sunday 13 October 2019 01:49:39 +0900 (0:00:00.431) 0:00:41.424 ********
ok: [kubeworker] => (item={'key': '127.0.0.1', 'value': {'expected': ['localhost', 'localhost.localdomain']}}) => {"ansible_facts": {"etc_hosts_localhosts_dict_target": {"127.0.0.1": ["localhost", "localhost.localdomain", "localhost4", "localhost4.localdomain4"]}}, "ansible_loop_var": "item", "changed": false, "item": {"key": "127.0.0.1", "value": {"expected": ["localhost", "localhost.localdomain"]}}}
ok: [kubeworker] => (item={'key': '::1', 'value': {'expected': ['localhost6', 'localhost6.localdomain'], 'unexpected': ['localhost', 'localhost.localdomain']}}) => {"ansible_facts": {"etc_hosts_localhosts_dict_target": {"127.0.0.1": ["localhost", "localhost.localdomain", "localhost4", "localhost4.localdomain4"], "::1": ["localhost6", "localhost6.localdomain6", "localhost6.localdomain"]}}, "ansible_loop_var": "item", "changed": false, "item": {"key": "::1", "value": {"expected": ["localhost6", "localhost6.localdomain"], "unexpected": ["localhost", "localhost.localdomain"]}}}
ok: [kubemaster] => (item={'key': '127.0.0.1', 'value': {'expected': ['localhost', 'localhost.localdomain']}}) => {"ansible_facts": {"etc_hosts_localhosts_dict_target": {"127.0.0.1": ["localhost", "localhost.localdomain", "localhost4", "localhost4.localdomain4"]}}, "ansible_loop_var": "item", "changed": false, "item": {"key": "127.0.0.1", "value": {"expected": ["localhost", "localhost.localdomain"]}}}
ok: [kubemaster] => (item={'key': '::1', 'value': {'expected': ['localhost6', 'localhost6.localdomain'], 'unexpected': ['localhost', 'localhost.localdomain']}}) => {"ansible_facts": {"etc_hosts_localhosts_dict_target": {"127.0.0.1": ["localhost", "localhost.localdomain", "localhost4", "localhost4.localdomain4"], "::1": ["localhost6", "localhost6.localdomain6", "localhost6.localdomain"]}}, "ansible_loop_var": "item", "changed": false, "item": {"key": "::1", "value": {"expected": ["localhost6", "localhost6.localdomain"], "unexpected": ["localhost", "localhost.localdomain"]}}}
TASK [kubernetes/preinstall : Hosts | Update (if necessary) hosts file] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:40 +0900 (0:00:00.259) 0:00:41.683 ********
ok: [kubeworker] => (item={'key': '127.0.0.1', 'value': ['localhost', 'localhost.localdomain', 'localhost4', 'localhost4.localdomain4']}) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": {"key": "127.0.0.1", "value": ["localhost", "localhost.localdomain", "localhost4", "localhost4.localdomain4"]}, "msg": ""}
ok: [kubemaster] => (item={'key': '127.0.0.1', 'value': ['localhost', 'localhost.localdomain', 'localhost4', 'localhost4.localdomain4']}) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": {"key": "127.0.0.1", "value": ["localhost", "localhost.localdomain", "localhost4", "localhost4.localdomain4"]}, "msg": ""}
ok: [kubeworker] => (item={'key': '::1', 'value': ['localhost6', 'localhost6.localdomain6', 'localhost6.localdomain']}) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": {"key": "::1", "value": ["localhost6", "localhost6.localdomain6", "localhost6.localdomain"]}, "msg": ""}
ok: [kubemaster] => (item={'key': '::1', 'value': ['localhost6', 'localhost6.localdomain6', 'localhost6.localdomain']}) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": {"key": "::1", "value": ["localhost6", "localhost6.localdomain6", "localhost6.localdomain"]}, "msg": ""}
TASK [kubernetes/preinstall : Configure dhclient to supersede search/domain/nameservers] ***************************************************************************************************************************************************
Sunday 13 October 2019 01:49:40 +0900 (0:00:00.563) 0:00:42.246 ********
TASK [kubernetes/preinstall : Configure dhclient hooks for resolv.conf (non-RH)] ***********************************************************************************************************************************************************
Sunday 13 October 2019 01:49:40 +0900 (0:00:00.130) 0:00:42.376 ********
TASK [kubernetes/preinstall : Configure dhclient hooks for resolv.conf (RH-only)] **********************************************************************************************************************************************************
Sunday 13 October 2019 01:49:41 +0900 (0:00:00.129) 0:00:42.506 ********
TASK [kubernetes/preinstall : Remove kubespray specific config from dhclient config] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:49:41 +0900 (0:00:00.128) 0:00:42.634 ********
TASK [kubernetes/preinstall : Remove kubespray specific dhclient hook] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:41 +0900 (0:00:00.135) 0:00:42.769 ********
ok: [kubeworker] => {"changed": false, "path": "/etc/dhcp/dhclient.d/zdnsupdate.sh", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/etc/dhcp/dhclient.d/zdnsupdate.sh", "state": "absent"}
TASK [kubernetes/preinstall : Check if we are running inside a Azure VM] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:41 +0900 (0:00:00.332) 0:00:43.102 ********
ok: [kubeworker] => {"changed": false, "stat": {"exists": false}}
ok: [kubemaster] => {"changed": false, "stat": {"exists": false}}
TASK [kubernetes/preinstall : install growpart] ********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:41 +0900 (0:00:00.310) 0:00:43.412 ********
TASK [kubernetes/preinstall : check if growpart needs to be run] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:42 +0900 (0:00:00.131) 0:00:43.544 ********
TASK [kubernetes/preinstall : check fs type] ***********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:42 +0900 (0:00:00.130) 0:00:43.674 ********
TASK [kubernetes/preinstall : run growpart] ************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:42 +0900 (0:00:00.130) 0:00:43.805 ********
TASK [kubernetes/preinstall : run xfs_growfs] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:42 +0900 (0:00:00.133) 0:00:43.938 ********
TASK [container-engine/cri-o : gather os specific variables] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:42 +0900 (0:00:00.137) 0:00:44.075 ********
TASK [container-engine/cri-o : Add OpenShift Origin repository] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:42 +0900 (0:00:00.169) 0:00:44.245 ********
TASK [container-engine/cri-o : Add CRI-O PPA] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:42 +0900 (0:00:00.154) 0:00:44.399 ********
TASK [container-engine/cri-o : Add CRI-O PPA] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:43 +0900 (0:00:00.131) 0:00:44.531 ********
TASK [container-engine/cri-o : Install crictl] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:43 +0900 (0:00:00.128) 0:00:44.659 ********
TASK [container-engine/cri-o : Make sure needed folders exist in the system] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:43 +0900 (0:00:00.143) 0:00:44.803 ********
TASK [container-engine/cri-o : Install cri-o packages] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:43 +0900 (0:00:00.166) 0:00:44.970 ********
TASK [container-engine/cri-o : Install cri-o config] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:43 +0900 (0:00:00.137) 0:00:45.107 ********
TASK [container-engine/cri-o : Copy mounts.conf] *******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:43 +0900 (0:00:00.162) 0:00:45.270 ********
TASK [container-engine/cri-o : Create directory for oci hooks] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:43 +0900 (0:00:00.161) 0:00:45.432 ********
TASK [container-engine/cri-o : Install cri-o service] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:44 +0900 (0:00:00.131) 0:00:45.563 ********
TASK [container-engine/containerd : Fail containerd setup if distribution is not supported] ************************************************************************************************************************************************
Sunday 13 October 2019 01:49:44 +0900 (0:00:00.133) 0:00:45.697 ********
TASK [container-engine/containerd : gather os specific variables] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:44 +0900 (0:00:00.147) 0:00:45.844 ********
TASK [container-engine/containerd : include_tasks] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:44 +0900 (0:00:00.176) 0:00:46.021 ********
TASK [container-engine/containerd : ensure containerd config directory] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:44 +0900 (0:00:00.132) 0:00:46.154 ********
TASK [container-engine/containerd : Copy containerd config file] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:44 +0900 (0:00:00.153) 0:00:46.307 ********
TASK [container-engine/containerd : Set containerd pin priority to apt_preferences on Debian family] ***************************************************************************************************************************************
Sunday 13 October 2019 01:49:45 +0900 (0:00:00.152) 0:00:46.460 ********
TASK [container-engine/containerd : ensure containerd packages are installed] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:45 +0900 (0:00:00.139) 0:00:46.599 ********
TASK [container-engine/containerd : Check if runc is installed] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:45 +0900 (0:00:00.153) 0:00:46.753 ********
TASK [container-engine/containerd : Install runc package if necessary] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:45 +0900 (0:00:00.131) 0:00:46.884 ********
TASK [container-engine/containerd : include_tasks] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:45 +0900 (0:00:00.142) 0:00:47.026 ********
TASK [container-engine/docker-storage : docker-storage-setup | install git and make] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:49:45 +0900 (0:00:00.139) 0:00:47.166 ********
TASK [container-engine/docker-storage : docker-storage-setup | docker-storage-setup sysconfig template] ************************************************************************************************************************************
Sunday 13 October 2019 01:49:45 +0900 (0:00:00.192) 0:00:47.359 ********
TASK [container-engine/docker-storage : docker-storage-override-directory | docker service storage-setup override dir] *********************************************************************************************************************
Sunday 13 October 2019 01:49:46 +0900 (0:00:00.172) 0:00:47.531 ********
TASK [container-engine/docker-storage : docker-storage-override | docker service storage-setup override file] ******************************************************************************************************************************
Sunday 13 October 2019 01:49:46 +0900 (0:00:00.134) 0:00:47.666 ********
TASK [container-engine/docker-storage : docker-storage-setup | install lvm2] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:46 +0900 (0:00:00.141) 0:00:47.807 ********
TASK [container-engine/docker-storage : docker-storage-setup | install and run container-storage-setup] ************************************************************************************************************************************
Sunday 13 October 2019 01:49:46 +0900 (0:00:00.131) 0:00:47.939 ********
TASK [container-engine/docker : check if atomic host] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:46 +0900 (0:00:00.153) 0:00:48.092 ********
ok: [kubeworker] => {"changed": false, "stat": {"exists": false}}
ok: [kubemaster] => {"changed": false, "stat": {"exists": false}}
TASK [container-engine/docker : set is_atomic] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:46 +0900 (0:00:00.313) 0:00:48.406 ********
ok: [kubeworker] => {"ansible_facts": {"is_atomic": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"is_atomic": false}, "changed": false}
TASK [container-engine/docker : gather os specific variables] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:47 +0900 (0:00:00.166) 0:00:48.572 ********
ok: [kubeworker] => (item=/root/kubespray/kubespray/roles/container-engine/docker/vars/../vars/redhat.yml) => {"ansible_facts": {"docker_kernel_min_version": "0", "docker_package_info": {"pkg_mgr": "yum", "pkgs": "{%- if docker_version is version('17.04', '<') -%}\n{{ docker_pkgs_use_docker_ce }}\n{%- else -%}\n{{ docker_pkgs }}\n{%- endif %}"}, "docker_pkgs": [{"name": "{{ docker_versioned_pkg[docker_version | string] }}", "yum_conf": "{{ docker_yum_conf }}"}], "docker_pkgs_use_docker_ce": [{"name": "{{ docker_selinux_versioned_pkg[docker_selinux_version | string] }}", "yum_conf": "{{ docker_yum_conf }}"}, {"name": "{{ docker_versioned_pkg[docker_version | string] }}", "yum_conf": "{{ docker_yum_conf }}"}], "docker_repo_info": {"pkg_repo": "", "repos": []}, "docker_repo_key_info": {"pkg_key": "", "repo_keys": []}, "docker_selinux_versioned_pkg": {"1.13": "docker-engine-selinux-1.13.1-1.el7.centos", "17.03": "docker-ce-selinux-17.03.2.ce-1.el7.centos", "edge": "docker-ce-selinux-17.03.2.ce-1.el7.centos", "latest": "docker-ce-selinux", "stable": "docker-ce-selinux-17.03.2.ce-1.el7.centos"}, "docker_versioned_pkg": {"1.13": "docker-engine-1.13.1-1.el7.centos", "17.03": "docker-ce-17.03.2.ce-1.el7.centos", "17.09": "docker-ce-17.09.0.ce-1.el7.centos", "17.12": "docker-ce-17.12.1.ce-1.el7.centos", "18.03": "docker-ce-18.03.1.ce-1.el7.centos", "18.06": "docker-ce-18.06.2.ce-3.el7", "18.09": "docker-ce-18.09.7-3.el7", "edge": "docker-ce-18.09.7-3.el7", "latest": "docker-ce", "stable": "docker-ce-18.09.7-3.el7"}}, "ansible_included_var_files": ["/root/kubespray/kubespray/roles/container-engine/docker/vars/redhat.yml"], "ansible_loop_var": "item", "changed": false, "item": "/root/kubespray/kubespray/roles/container-engine/docker/vars/../vars/redhat.yml"}
ok: [kubemaster] => (item=/root/kubespray/kubespray/roles/container-engine/docker/vars/../vars/redhat.yml) => {"ansible_facts": {"docker_kernel_min_version": "0", "docker_package_info": {"pkg_mgr": "yum", "pkgs": "{%- if docker_version is version('17.04', '<') -%}\n{{ docker_pkgs_use_docker_ce }}\n{%- else -%}\n{{ docker_pkgs }}\n{%- endif %}"}, "docker_pkgs": [{"name": "{{ docker_versioned_pkg[docker_version | string] }}", "yum_conf": "{{ docker_yum_conf }}"}], "docker_pkgs_use_docker_ce": [{"name": "{{ docker_selinux_versioned_pkg[docker_selinux_version | string] }}", "yum_conf": "{{ docker_yum_conf }}"}, {"name": "{{ docker_versioned_pkg[docker_version | string] }}", "yum_conf": "{{ docker_yum_conf }}"}], "docker_repo_info": {"pkg_repo": "", "repos": []}, "docker_repo_key_info": {"pkg_key": "", "repo_keys": []}, "docker_selinux_versioned_pkg": {"1.13": "docker-engine-selinux-1.13.1-1.el7.centos", "17.03": "docker-ce-selinux-17.03.2.ce-1.el7.centos", "edge": "docker-ce-selinux-17.03.2.ce-1.el7.centos", "latest": "docker-ce-selinux", "stable": "docker-ce-selinux-17.03.2.ce-1.el7.centos"}, "docker_versioned_pkg": {"1.13": "docker-engine-1.13.1-1.el7.centos", "17.03": "docker-ce-17.03.2.ce-1.el7.centos", "17.09": "docker-ce-17.09.0.ce-1.el7.centos", "17.12": "docker-ce-17.12.1.ce-1.el7.centos", "18.03": "docker-ce-18.03.1.ce-1.el7.centos", "18.06": "docker-ce-18.06.2.ce-3.el7", "18.09": "docker-ce-18.09.7-3.el7", "edge": "docker-ce-18.09.7-3.el7", "latest": "docker-ce", "stable": "docker-ce-18.09.7-3.el7"}}, "ansible_included_var_files": ["/root/kubespray/kubespray/roles/container-engine/docker/vars/redhat.yml"], "ansible_loop_var": "item", "changed": false, "item": "/root/kubespray/kubespray/roles/container-engine/docker/vars/../vars/redhat.yml"}
TASK [container-engine/docker : Warn about Docker version on SUSE] *************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:47 +0900 (0:00:00.227) 0:00:48.800 ********
TASK [container-engine/docker : include_tasks] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:47 +0900 (0:00:00.138) 0:00:48.939 ********
included: /root/kubespray/kubespray/roles/container-engine/docker/tasks/set_facts_dns.yml for kubeworker, kubemaster
TASK [container-engine/docker : set dns server for docker] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:47 +0900 (0:00:00.301) 0:00:49.241 ********
ok: [kubeworker] => {"ansible_facts": {"docker_dns_servers": ["10.233.0.3"]}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"docker_dns_servers": ["10.233.0.3"]}, "changed": false}
TASK [container-engine/docker : show docker_dns_servers] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:48 +0900 (0:00:00.198) 0:00:49.439 ********
ok: [kubeworker] => {
"msg": [
"10.233.0.3"
]
}
ok: [kubemaster] => {
"msg": [
"10.233.0.3"
]
}
TASK [container-engine/docker : set base docker dns facts] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:48 +0900 (0:00:00.197) 0:00:49.637 ********
ok: [kubeworker] => {"ansible_facts": {"docker_dns_options": ["ndots:2", "timeout:2", "attempts:2"], "docker_dns_search_domains": ["default.svc.cluster.local", "svc.cluster.local"]}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"docker_dns_options": ["ndots:2", "timeout:2", "attempts:2"], "docker_dns_search_domains": ["default.svc.cluster.local", "svc.cluster.local"]}, "changed": false}
TASK [container-engine/docker : add upstream dns servers] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:48 +0900 (0:00:00.210) 0:00:49.848 ********
ok: [kubeworker] => {"ansible_facts": {"docker_dns_servers": ["10.233.0.3"]}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"docker_dns_servers": ["10.233.0.3"]}, "changed": false}
TASK [container-engine/docker : add global searchdomains] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:48 +0900 (0:00:00.193) 0:00:50.042 ********
ok: [kubeworker] => {"ansible_facts": {"docker_dns_search_domains": ["default.svc.cluster.local", "svc.cluster.local"]}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"docker_dns_search_domains": ["default.svc.cluster.local", "svc.cluster.local"]}, "changed": false}
TASK [container-engine/docker : check system nameservers] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:48 +0900 (0:00:00.185) 0:00:50.228 ********
ok: [kubeworker] => {"changed": false, "cmd": "grep \"^nameserver\" /etc/resolv.conf | sed 's/^nameserver\\s*//'", "delta": "0:00:00.003877", "end": "2019-10-13 01:49:48.715230", "rc": 0, "start": "2019-10-13 01:49:48.711353", "stderr": "", "stderr_lines": [], "stdout": "210.220.163.82\n219.250.36.130", "stdout_lines": ["210.220.163.82", "219.250.36.130"]}
ok: [kubemaster] => {"changed": false, "cmd": "grep \"^nameserver\" /etc/resolv.conf | sed 's/^nameserver\\s*//'", "delta": "0:00:00.017663", "end": "2019-10-13 01:49:49.053169", "rc": 0, "start": "2019-10-13 01:49:49.035506", "stderr": "", "stderr_lines": [], "stdout": "210.220.163.82\n219.250.36.130", "stdout_lines": ["210.220.163.82", "219.250.36.130"]}
TASK [container-engine/docker : check system search domains] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:49 +0900 (0:00:00.330) 0:00:50.558 ********
ok: [kubeworker] => {"changed": false, "cmd": "grep \"^search\" /etc/resolv.conf | sed 's/^search\\s*//'", "delta": "0:00:00.003787", "end": "2019-10-13 01:49:49.044436", "rc": 0, "start": "2019-10-13 01:49:49.040649", "stderr": "", "stderr_lines": [], "stdout": "skbroadband", "stdout_lines": ["skbroadband"]}
ok: [kubemaster] => {"changed": false, "cmd": "grep \"^search\" /etc/resolv.conf | sed 's/^search\\s*//'", "delta": "0:00:00.003424", "end": "2019-10-13 01:49:49.384451", "rc": 0, "start": "2019-10-13 01:49:49.381027", "stderr": "", "stderr_lines": [], "stdout": "skbroadband", "stdout_lines": ["skbroadband"]}
TASK [container-engine/docker : add system nameservers to docker options] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:49 +0900 (0:00:00.333) 0:00:50.891 ********
ok: [kubeworker] => {"ansible_facts": {"docker_dns_servers": ["10.233.0.3", "210.220.163.82", "219.250.36.130"]}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"docker_dns_servers": ["10.233.0.3", "210.220.163.82", "219.250.36.130"]}, "changed": false}
TASK [container-engine/docker : add system search domains to docker options] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:49 +0900 (0:00:00.217) 0:00:51.109 ********
ok: [kubeworker] => {"ansible_facts": {"docker_dns_search_domains": ["default.svc.cluster.local", "svc.cluster.local", "skbroadband"]}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"docker_dns_search_domains": ["default.svc.cluster.local", "svc.cluster.local", "skbroadband"]}, "changed": false}
TASK [container-engine/docker : check number of nameservers] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:49 +0900 (0:00:00.183) 0:00:51.292 ********
TASK [container-engine/docker : rtrim number of nameservers to 3] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:50 +0900 (0:00:00.150) 0:00:51.442 ********
TASK [container-engine/docker : check number of search domains] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:50 +0900 (0:00:00.150) 0:00:51.593 ********
TASK [container-engine/docker : check length of search domains] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:50 +0900 (0:00:00.143) 0:00:51.737 ********
TASK [container-engine/docker : check for minimum kernel version] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:50 +0900 (0:00:00.150) 0:00:51.888 ********
TASK [container-engine/docker : Ensure old versions of Docker are not installed. | Debian] *************************************************************************************************************************************************
Sunday 13 October 2019 01:49:50 +0900 (0:00:00.139) 0:00:52.027 ********
TASK [container-engine/docker : Ensure old versions of Docker are not installed. | RedHat] *************************************************************************************************************************************************
Sunday 13 October 2019 01:49:50 +0900 (0:00:00.167) 0:00:52.195 ********
ok: [kubeworker] => {"changed": false, "msg": "", "rc": 0, "results": ["docker is not installed", "docker-common is not installed", "docker-engine is not installed", "docker-selinux.noarch is not installed", "docker-client is not installed", "docker-client-latest is not installed", "docker-latest is not installed", "docker-latest-logrotate is not installed", "docker-logrotate is not installed", "docker-engine-selinux.noarch is not installed"]}
ok: [kubemaster] => {"changed": false, "msg": "", "rc": 0, "results": ["docker is not installed", "docker-common is not installed", "docker-engine is not installed", "docker-selinux.noarch is not installed", "docker-client is not installed", "docker-client-latest is not installed", "docker-latest is not installed", "docker-latest-logrotate is not installed", "docker-logrotate is not installed", "docker-engine-selinux.noarch is not installed"]}
TASK [container-engine/docker : ensure docker-ce repository public key is installed] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:49:52 +0900 (0:00:01.280) 0:00:53.476 ********
TASK [container-engine/docker : ensure docker-ce repository is enabled] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:52 +0900 (0:00:00.122) 0:00:53.598 ********
TASK [container-engine/docker : ensure docker-engine repository public key is installed] ***************************************************************************************************************************************************
Sunday 13 October 2019 01:49:52 +0900 (0:00:00.127) 0:00:53.725 ********
TASK [container-engine/docker : ensure docker-engine repository is enabled] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:52 +0900 (0:00:00.166) 0:00:53.892 ********
TASK [container-engine/docker : Configure docker repository on Fedora] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:52 +0900 (0:00:00.160) 0:00:54.052 ********
TASK [container-engine/docker : Configure docker repository on RedHat/CentOS/Oracle Linux] *************************************************************************************************************************************************
Sunday 13 October 2019 01:49:52 +0900 (0:00:00.132) 0:00:54.185 ********
ok: [kubeworker] => {"changed": false, "checksum": "622b3b0948e26cd21562f028e9c04b8a98777367", "dest": "/etc/yum.repos.d/docker.repo", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/docker.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 369, "state": "file", "uid": 0}
ok: [kubemaster] => {"changed": false, "checksum": "622b3b0948e26cd21562f028e9c04b8a98777367", "dest": "/etc/yum.repos.d/docker.repo", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/docker.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 369, "state": "file", "uid": 0}
TASK [container-engine/docker : check if container-selinux is available] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:53 +0900 (0:00:00.572) 0:00:54.757 ********
ok: [kubeworker] => {"changed": false, "results": [{"arch": "noarch", "envra": "2:container-selinux-2.107-1.el7_6.noarch", "epoch": "2", "name": "container-selinux", "release": "1.el7_6", "repo": "extras", "version": "2.107", "yumstate": "available"}, {"arch": "noarch", "envra": "2:container-selinux-2.107-1.el7_6.noarch", "epoch": "2", "name": "container-selinux", "release": "1.el7_6", "repo": "installed", "version": "2.107", "yumstate": "installed"}, {"arch": "noarch", "envra": "2:container-selinux-2.107-3.el7.noarch", "epoch": "2", "name": "container-selinux", "release": "3.el7", "repo": "extras", "version": "2.107", "yumstate": "available"}]}
ok: [kubemaster] => {"changed": false, "results": [{"arch": "noarch", "envra": "2:container-selinux-2.107-1.el7_6.noarch", "epoch": "2", "name": "container-selinux", "release": "1.el7_6", "repo": "extras", "version": "2.107", "yumstate": "available"}, {"arch": "noarch", "envra": "2:container-selinux-2.107-3.el7.noarch", "epoch": "2", "name": "container-selinux", "release": "3.el7", "repo": "extras", "version": "2.107", "yumstate": "available"}, {"arch": "noarch", "envra": "2:container-selinux-2.107-3.el7.noarch", "epoch": "2", "name": "container-selinux", "release": "3.el7", "repo": "installed", "version": "2.107", "yumstate": "installed"}]}
TASK [container-engine/docker : Configure extras repository on RedHat/CentOS if container-selinux is not available in current repos] *******************************************************************************************************
Sunday 13 October 2019 01:49:54 +0900 (0:00:00.842) 0:00:55.599 ********
TASK [container-engine/docker : Copy yum.conf for editing] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:54 +0900 (0:00:00.134) 0:00:55.734 ********
changed: [kubeworker] => {"changed": true, "checksum": "e0637e631f4ab0aaebef1a6b8822a36f031f332e", "dest": "/etc/yum_docker.conf", "gid": 0, "group": "root", "md5sum": "a7dc0d7b8902e9c8c096c93eb431d19e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 970, "src": "/etc/yum.conf", "state": "file", "uid": 0}
changed: [kubemaster] => {"changed": true, "checksum": "e0637e631f4ab0aaebef1a6b8822a36f031f332e", "dest": "/etc/yum_docker.conf", "gid": 0, "group": "root", "md5sum": "a7dc0d7b8902e9c8c096c93eb431d19e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 970, "src": "/etc/yum.conf", "state": "file", "uid": 0}
TASK [container-engine/docker : Edit copy of yum.conf to set obsoletes=0] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:54 +0900 (0:00:00.639) 0:00:56.374 ********
changed: [kubeworker] => {"backup": "", "changed": true, "msg": "line replaced"}
changed: [kubemaster] => {"backup": "", "changed": true, "msg": "line replaced"}
TASK [container-engine/docker : ensure docker packages are installed] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:55 +0900 (0:00:00.327) 0:00:56.701 ********
ok: [kubeworker] => (item={'name': 'docker-ce-18.09.7-3.el7', 'yum_conf': '/etc/yum_docker.conf'}) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"name": "docker-ce-18.09.7-3.el7", "yum_conf": "/etc/yum_docker.conf"}, "msg": "", "rc": 0, "results": ["3:docker-ce-18.09.7-3.el7.x86_64 providing docker-ce-18.09.7-3.el7 is already installed"]}
ok: [kubemaster] => (item={'name': 'docker-ce-18.09.7-3.el7', 'yum_conf': '/etc/yum_docker.conf'}) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"name": "docker-ce-18.09.7-3.el7", "yum_conf": "/etc/yum_docker.conf"}, "msg": "", "rc": 0, "results": ["3:docker-ce-18.09.7-3.el7.x86_64 providing docker-ce-18.09.7-3.el7 is already installed"]}
TASK [container-engine/docker : Ensure docker packages are installed] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:56 +0900 (0:00:01.033) 0:00:57.735 ********
TASK [container-engine/docker : get available packages on Ubuntu] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:56 +0900 (0:00:00.200) 0:00:57.936 ********
TASK [container-engine/docker : show available packages on ubuntu] *************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:56 +0900 (0:00:00.172) 0:00:58.108 ********
TASK [container-engine/docker : Tell Debian hosts not to change the docker version with apt upgrade] ***************************************************************************************************************************************
Sunday 13 October 2019 01:49:56 +0900 (0:00:00.141) 0:00:58.249 ********
TASK [container-engine/docker : ensure service is started if docker packages are already present] ******************************************************************************************************************************************
Sunday 13 October 2019 01:49:56 +0900 (0:00:00.137) 0:00:58.387 ********
ok: [kubemaster] => {"changed": false, "name": "docker", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-10-13 01:12:36 KST", "ActiveEnterTimestampMonotonic": "19097362081", "ActiveExitTimestamp": "Sun 2019-10-13 01:12:36 KST", "ActiveExitTimestampMonotonic": "19097223451", "ActiveState": "active", "After": "systemd-journald.socket docker.socket containerd.service system.slice network.target basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-10-13 01:12:36 KST", "AssertTimestampMonotonic": "19097238642", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-10-13 01:12:36 KST", "ConditionTimestampMonotonic": "19097238642", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlGroup": "/system.slice/docker.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "http://docs.docker.com", "DropInPaths": "/etc/systemd/system/docker.service.d/docker-dns.conf /etc/systemd/system/docker.service.d/docker-options.conf", "Environment": "GOTRACEBACK=crash DOCKER_DNS_OPTIONS= --dns 10.233.0.3 --dns 210.220.163.82 --dns 219.250.36.130 --dns-search default.svc.cluster.local --dns-search svc.cluster.local --dns-search skbroadband --dns-opt ndots:2 --dns-opt timeout:2 --dns-opt attempts:2 DOCKER_OPTS= --iptables=false --data-root=/var/lib/docker --log-opt max-size=50m --log-opt max-file=5", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "18824", "ExecMainStartTimestamp": "Sun 2019-10-13 01:12:36 KST", "ExecMainStartTimestampMonotonic": "19097242693", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd $DOCKER_OPTS $DOCKER_STORAGE_OPTIONS $DOCKER_NETWORK_OPTIONS $DOCKER_DNS_OPTIONS $INSECURE_REGISTRY ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/etc/systemd/system/docker.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2019-10-13 01:12:36 KST", "InactiveEnterTimestampMonotonic": "19097227203", "InactiveExitTimestamp": "Sun 2019-10-13 01:12:36 KST", "InactiveExitTimestampMonotonic": "19097242725", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "process", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "1048576", "LimitNPROC": "1048576", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7259", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "18824", "MemoryAccounting": "no", "MemoryCurrent": "653475840", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "system.slice basic.target", "Restart": "on-failure", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitInterval": "60000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "11", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sun 2019-10-13 01:12:36 KST", "WatchdogTimestampMonotonic": "19097362056", "WatchdogUSec": "0"}}
ok: [kubeworker] => {"changed": false, "name": "docker", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-10-13 01:12:36 KST", "ActiveEnterTimestampMonotonic": "19094006356", "ActiveExitTimestamp": "Sun 2019-10-13 01:12:35 KST", "ActiveExitTimestampMonotonic": "19093860883", "ActiveState": "active", "After": "system.slice docker.socket containerd.service network.target basic.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-10-13 01:12:35 KST", "AssertTimestampMonotonic": "19093874084", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-10-13 01:12:35 KST", "ConditionTimestampMonotonic": "19093874084", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlGroup": "/system.slice/docker.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "http://docs.docker.com", "DropInPaths": "/etc/systemd/system/docker.service.d/docker-dns.conf /etc/systemd/system/docker.service.d/docker-options.conf", "Environment": "GOTRACEBACK=crash DOCKER_DNS_OPTIONS=\\x20\\x20\\x20\\x20\\x20--dns\\x2010.233.0.3\\x20--dns\\x20210.220.163.82\\x20--dns\\x20219.250.36.130\\x20\\x20\\x20\\x20\\x20\\x20\\x20--dns-search\\x20default.svc.cluster.local\\x20--dns-search\\x20svc.cluster.local\\x20--dns-search\\x20skbroadband\\x20\\x20\\x20\\x20\\x20\\x20\\x20--dns-opt\\x20ndots:2\\x20--dns-opt\\x20timeout:2\\x20--dns-opt\\x20attempts:2\\x20\\x20\\x20 DOCKER_OPTS=\\x20--iptables=false\\x20\\x20\\x20\\x20\\x20\\x20--data-root=/var/lib/docker\\x20\\x20--log-opt\\x20max-size=50m\\x20--log-opt\\x20max-file=5\\x20\\x20", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "16118", "ExecMainStartTimestamp": "Sun 2019-10-13 01:12:35 KST", "ExecMainStartTimestampMonotonic": "19093878806", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd $DOCKER_OPTS $DOCKER_STORAGE_OPTIONS $DOCKER_NETWORK_OPTIONS $DOCKER_DNS_OPTIONS $INSECURE_REGISTRY ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/etc/systemd/system/docker.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2019-10-13 01:12:35 KST", "InactiveEnterTimestampMonotonic": "19093864882", "InactiveExitTimestamp": "Sun 2019-10-13 01:12:35 KST", "InactiveExitTimestampMonotonic": "19093878876", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "process", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "1048576", "LimitNPROC": "1048576", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7257", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "16118", "MemoryAccounting": "no", "MemoryCurrent": "32612352", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "on-failure", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitInterval": "60000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "10", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Sun 2019-10-13 01:12:36 KST", "WatchdogTimestampMonotonic": "19094006329", "WatchdogUSec": "0"}}
[WARNING]: flush_handlers task does not support when conditional
TASK [container-engine/docker : set fact for docker_version] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:58 +0900 (0:00:01.083) 0:00:59.470 ********
ok: [kubeworker] => {"changed": false, "cmd": ["docker", "version", "-f", "{{.Client.Version}}"], "delta": "0:00:00.047992", "end": "2019-10-13 01:49:57.986903", "rc": 0, "start": "2019-10-13 01:49:57.938911", "stderr": "", "stderr_lines": [], "stdout": "19.03.3", "stdout_lines": ["19.03.3"]}
ok: [kubemaster] => {"changed": false, "cmd": ["docker", "version", "-f", "{{.Client.Version}}"], "delta": "0:00:00.046324", "end": "2019-10-13 01:49:58.335474", "rc": 0, "start": "2019-10-13 01:49:58.289150", "stderr": "", "stderr_lines": [], "stdout": "19.03.3", "stdout_lines": ["19.03.3"]}
TASK [container-engine/docker : check minimum docker version for docker_dns mode. You need at least docker version >= 1.12 for resolvconf_mode=docker_dns] *********************************************************************************
Sunday 13 October 2019 01:49:58 +0900 (0:00:00.366) 0:00:59.837 ********
TASK [container-engine/docker : include_tasks] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:58 +0900 (0:00:00.161) 0:00:59.999 ********
TASK [container-engine/docker : Create docker service systemd directory if it doesn't exist] ***********************************************************************************************************************************************
Sunday 13 October 2019 01:49:58 +0900 (0:00:00.121) 0:01:00.120 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/systemd/system/docker.service.d", "secontext": "unconfined_u:object_r:systemd_unit_file_t:s0", "size": 56, "state": "directory", "uid": 0}
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/systemd/system/docker.service.d", "secontext": "unconfined_u:object_r:systemd_unit_file_t:s0", "size": 56, "state": "directory", "uid": 0}
TASK [container-engine/docker : Write docker proxy drop-in] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:59 +0900 (0:00:00.348) 0:01:00.469 ********
TASK [container-engine/docker : get systemd version] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:59 +0900 (0:00:00.153) 0:01:00.622 ********
ok: [kubeworker] => {"changed": false, "cmd": "systemctl --version | head -n 1 | cut -d \" \" -f 2", "delta": "0:00:00.004595", "end": "2019-10-13 01:49:59.108551", "rc": 0, "start": "2019-10-13 01:49:59.103956", "stderr": "", "stderr_lines": [], "stdout": "219", "stdout_lines": ["219"]}
ok: [kubemaster] => {"changed": false, "cmd": "systemctl --version | head -n 1 | cut -d \" \" -f 2", "delta": "0:00:00.017025", "end": "2019-10-13 01:49:59.460280", "rc": 0, "start": "2019-10-13 01:49:59.443255", "stderr": "", "stderr_lines": [], "stdout": "219", "stdout_lines": ["219"]}
TASK [container-engine/docker : Write docker.service systemd file] *************************************************************************************************************************************************************************
Sunday 13 October 2019 01:49:59 +0900 (0:00:00.344) 0:01:00.967 ********
ok: [kubeworker] => {"changed": false, "checksum": "7cb5a8e186a8a691b26ece4dc040d3c88d4727b0", "dest": "/etc/systemd/system/docker.service", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/systemd/system/docker.service", "secontext": "system_u:object_r:container_unit_file_t:s0", "size": 671, "state": "file", "uid": 0}
ok: [kubemaster] => {"changed": false, "checksum": "7cb5a8e186a8a691b26ece4dc040d3c88d4727b0", "dest": "/etc/systemd/system/docker.service", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/systemd/system/docker.service", "secontext": "system_u:object_r:container_unit_file_t:s0", "size": 671, "state": "file", "uid": 0}
TASK [container-engine/docker : Write docker options systemd drop-in] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:00 +0900 (0:00:00.595) 0:01:01.563 ********
ok: [kubeworker] => {"changed": false, "checksum": "41c70f3516177f25bd9821c3ffcd64d9ace12640", "dest": "/etc/systemd/system/docker.service.d/docker-options.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/systemd/system/docker.service.d/docker-options.conf", "secontext": "system_u:object_r:container_unit_file_t:s0", "size": 140, "state": "file", "uid": 0}
ok: [kubemaster] => {"changed": false, "checksum": "41c70f3516177f25bd9821c3ffcd64d9ace12640", "dest": "/etc/systemd/system/docker.service.d/docker-options.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/systemd/system/docker.service.d/docker-options.conf", "secontext": "system_u:object_r:container_unit_file_t:s0", "size": 140, "state": "file", "uid": 0}
TASK [container-engine/docker : Write docker dns systemd drop-in] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:00 +0900 (0:00:00.564) 0:01:02.127 ********
ok: [kubeworker] => {"changed": false, "checksum": "d7e56ca18d817a15190cc40b7210b99628a2f690", "dest": "/etc/systemd/system/docker.service.d/docker-dns.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/systemd/system/docker.service.d/docker-dns.conf", "secontext": "system_u:object_r:container_unit_file_t:s0", "size": 279, "state": "file", "uid": 0}
ok: [kubemaster] => {"changed": false, "checksum": "d7e56ca18d817a15190cc40b7210b99628a2f690", "dest": "/etc/systemd/system/docker.service.d/docker-dns.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/systemd/system/docker.service.d/docker-dns.conf", "secontext": "system_u:object_r:container_unit_file_t:s0", "size": 279, "state": "file", "uid": 0}
TASK [container-engine/docker : Copy docker orphan clean up script to the node] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:01 +0900 (0:00:00.562) 0:01:02.690 ********
TASK [container-engine/docker : Write docker orphan clean up systemd drop-in] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:01 +0900 (0:00:00.149) 0:01:02.840 ********
TASK [container-engine/docker : ensure docker service is started and enabled] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:01 +0900 (0:00:00.155) 0:01:02.996 ********
ok: [kubeworker] => (item=docker) => {"ansible_loop_var": "item", "changed": false, "enabled": true, "item": "docker", "name": "docker", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-10-13 01:12:36 KST", "ActiveEnterTimestampMonotonic": "19094006356", "ActiveExitTimestamp": "Sun 2019-10-13 01:12:35 KST", "ActiveExitTimestampMonotonic": "19093860883", "ActiveState": "active", "After": "system.slice docker.socket containerd.service network.target basic.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-10-13 01:12:35 KST", "AssertTimestampMonotonic": "19093874084", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-10-13 01:12:35 KST", "ConditionTimestampMonotonic": "19093874084", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlGroup": "/system.slice/docker.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "http://docs.docker.com", "DropInPaths": "/etc/systemd/system/docker.service.d/docker-dns.conf /etc/systemd/system/docker.service.d/docker-options.conf", "Environment": "GOTRACEBACK=crash DOCKER_DNS_OPTIONS=\\x20\\x20\\x20\\x20\\x20--dns\\x2010.233.0.3\\x20--dns\\x20210.220.163.82\\x20--dns\\x20219.250.36.130\\x20\\x20\\x20\\x20\\x20\\x20\\x20--dns-search\\x20default.svc.cluster.local\\x20--dns-search\\x20svc.cluster.local\\x20--dns-search\\x20skbroadband\\x20\\x20\\x20\\x20\\x20\\x20\\x20--dns-opt\\x20ndots:2\\x20--dns-opt\\x20timeout:2\\x20--dns-opt\\x20attempts:2\\x20\\x20\\x20 DOCKER_OPTS=\\x20--iptables=false\\x20\\x20\\x20\\x20\\x20\\x20--data-root=/var/lib/docker\\x20\\x20--log-opt\\x20max-size=50m\\x20--log-opt\\x20max-file=5\\x20\\x20", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "16118", "ExecMainStartTimestamp": "Sun 2019-10-13 01:12:35 KST", "ExecMainStartTimestampMonotonic": "19093878806", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd $DOCKER_OPTS $DOCKER_STORAGE_OPTIONS $DOCKER_NETWORK_OPTIONS $DOCKER_DNS_OPTIONS $INSECURE_REGISTRY ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/etc/systemd/system/docker.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2019-10-13 01:12:35 KST", "InactiveEnterTimestampMonotonic": "19093864882", "InactiveExitTimestamp": "Sun 2019-10-13 01:12:35 KST", "InactiveExitTimestampMonotonic": "19093878876", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "process", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "1048576", "LimitNPROC": "1048576", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7257", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "16118", "MemoryAccounting": "no", "MemoryCurrent": "32657408", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "on-failure", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitInterval": "60000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "10", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Sun 2019-10-13 01:12:36 KST", "WatchdogTimestampMonotonic": "19094006329", "WatchdogUSec": "0"}}
ok: [kubemaster] => (item=docker) => {"ansible_loop_var": "item", "changed": false, "enabled": true, "item": "docker", "name": "docker", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-10-13 01:12:36 KST", "ActiveEnterTimestampMonotonic": "19097362081", "ActiveExitTimestamp": "Sun 2019-10-13 01:12:36 KST", "ActiveExitTimestampMonotonic": "19097223451", "ActiveState": "active", "After": "systemd-journald.socket docker.socket containerd.service system.slice network.target basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-10-13 01:12:36 KST", "AssertTimestampMonotonic": "19097238642", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-10-13 01:12:36 KST", "ConditionTimestampMonotonic": "19097238642", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlGroup": "/system.slice/docker.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "http://docs.docker.com", "DropInPaths": "/etc/systemd/system/docker.service.d/docker-dns.conf /etc/systemd/system/docker.service.d/docker-options.conf", "Environment": "GOTRACEBACK=crash DOCKER_DNS_OPTIONS= --dns 10.233.0.3 --dns 210.220.163.82 --dns 219.250.36.130 --dns-search default.svc.cluster.local --dns-search svc.cluster.local --dns-search skbroadband --dns-opt ndots:2 --dns-opt timeout:2 --dns-opt attempts:2 DOCKER_OPTS= --iptables=false --data-root=/var/lib/docker --log-opt max-size=50m --log-opt max-file=5", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "18824", "ExecMainStartTimestamp": "Sun 2019-10-13 01:12:36 KST", "ExecMainStartTimestampMonotonic": "19097242693", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd $DOCKER_OPTS $DOCKER_STORAGE_OPTIONS $DOCKER_NETWORK_OPTIONS $DOCKER_DNS_OPTIONS $INSECURE_REGISTRY ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/etc/systemd/system/docker.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2019-10-13 01:12:36 KST", "InactiveEnterTimestampMonotonic": "19097227203", "InactiveExitTimestamp": "Sun 2019-10-13 01:12:36 KST", "InactiveExitTimestampMonotonic": "19097242725", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "process", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "1048576", "LimitNPROC": "1048576", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7259", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "18824", "MemoryAccounting": "no", "MemoryCurrent": "653475840", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "system.slice basic.target", "Restart": "on-failure", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitInterval": "60000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "11", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sun 2019-10-13 01:12:36 KST", "WatchdogTimestampMonotonic": "19097362056", "WatchdogUSec": "0"}}
TASK [download : prep_download | Set a few facts] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:02 +0900 (0:00:00.474) 0:01:03.471 ********
ok: [kubeworker] => {"ansible_facts": {"download_force_cache": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"download_force_cache": false}, "changed": false}
TASK [download : Set image info command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:02 +0900 (0:00:00.166) 0:01:03.637 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:02 +0900 (0:00:00.132) 0:01:03.769 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : prep_download | Create staging directory on remote node] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:02 +0900 (0:00:00.420) 0:01:04.190 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/releases/images", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 6, "state": "directory", "uid": 0}
changed: [kubemaster] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/releases/images", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 6, "state": "directory", "uid": 0}
TASK [download : prep_download | Create local cache for files and images] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:03 +0900 (0:00:00.339) 0:01:04.529 ********
changed: [kubeworker -> localhost] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/kubespray_cache/images", "size": 6, "state": "directory", "uid": 0}
TASK [download : prep_download | On localhost, check if passwordless root is possible] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:50:03 +0900 (0:00:00.338) 0:01:04.868 ********
TASK [download : prep_download | On localhost, check if user has access to docker without using sudo] **************************************************************************************************************************************
Sunday 13 October 2019 01:50:03 +0900 (0:00:00.085) 0:01:04.954 ********
TASK [download : prep_download | Parse the outputs of the previous commands] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:03 +0900 (0:00:00.107) 0:01:05.061 ********
TASK [download : prep_download | Check that local user is in group or can become root] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:50:03 +0900 (0:00:00.136) 0:01:05.197 ********
TASK [download : include_tasks] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:03 +0900 (0:00:00.134) 0:01:05.332 ********
TASK [download : download | Get kubeadm binary and list of required images] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:04 +0900 (0:00:00.133) 0:01:05.465 ********
included: /root/kubespray/kubespray/roles/download/tasks/prep_kubeadm_images.yml for kubemaster
TASK [download : prep_kubeadm_images | Download kubeadm binary] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:04 +0900 (0:00:00.250) 0:01:05.716 ********
included: /root/kubespray/kubespray/roles/download/tasks/download_file.yml for kubemaster
TASK [download : download_file | Starting download of file] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:04 +0900 (0:00:00.595) 0:01:06.312 ********
ok: [kubemaster] => {
"msg": "https://storage.googleapis.com/kubernetes-release/release/v1.16.0/bin/linux/amd64/kubeadm"
}
TASK [download : download_file | Set pathname of cached file] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:05 +0900 (0:00:00.453) 0:01:06.766 ********
ok: [kubemaster] => {"ansible_facts": {"file_path_cached": "/tmp/kubespray_cache/kubeadm-v1.16.0-amd64"}, "changed": false}
TASK [download : download_file | Create dest directory on node] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:05 +0900 (0:00:00.489) 0:01:07.255 ********
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/releases", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 20, "state": "directory", "uid": 0}
TASK [download : download_file | Create local cache directory] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:06 +0900 (0:00:00.619) 0:01:07.875 ********
ok: [kubemaster -> localhost] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/kubespray_cache", "size": 20, "state": "directory", "uid": 0}
TASK [download : download_file | Check if file is available in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:06 +0900 (0:00:00.376) 0:01:08.251 ********
TASK [download : download_file | Set file_is_cached fact based on previous task] ***********************************************************************************************************************************************************
Sunday 13 October 2019 01:50:06 +0900 (0:00:00.092) 0:01:08.344 ********
TASK [download : download_file | Copy file from cache to nodes, if it is available] ********************************************************************************************************************************************************
Sunday 13 October 2019 01:50:06 +0900 (0:00:00.077) 0:01:08.422 ********
TASK [download : download_file | Set mode and owner] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:07 +0900 (0:00:00.079) 0:01:08.502 ********
TASK [download : download_file | Download item] ********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:07 +0900 (0:00:00.125) 0:01:08.627 ********
changed: [kubemaster -> None] => {"attempts": 1, "changed": true, "checksum_dest": null, "checksum_src": "2d0d544584409cdaacbaf203801a9957069df092", "dest": "/tmp/releases/kubeadm-v1.16.0-amd64", "elapsed": 4, "gid": 0, "group": "root", "md5sum": "1dfa41977e0ef0b4bff7a8bcc18ce1e0", "mode": "0755", "msg": "OK (44244800 bytes)", "owner": "root", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 44244800, "src": "/root/.ansible/tmp/ansible-moduletmp-1570899008.16-iZz3yq/tmp4tLdFb", "state": "file", "status_code": 200, "uid": 0, "url": "https://storage.googleapis.com/kubernetes-release/release/v1.16.0/bin/linux/amd64/kubeadm"}
TASK [download : download_file | Extract file archives] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:13 +0900 (0:00:06.269) 0:01:14.897 ********
included: /root/kubespray/kubespray/roles/download/tasks/extract_file.yml for kubemaster
TASK [download : extract_file | Unpacking archive] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:13 +0900 (0:00:00.164) 0:01:15.062 ********
[WARNING]: noop task does not support when conditional
TASK [download : download_file | Copy file back to ansible host file cache] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:14 +0900 (0:00:00.424) 0:01:15.486 ********
TASK [download : prep_kubeadm_images | Sync kubeadm binary to nodes] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:14 +0900 (0:00:00.079) 0:01:15.566 ********
TASK [download : prep_kubeadm_images | Create kubeadm config] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:14 +0900 (0:00:00.430) 0:01:15.997 ********
ok: [kubemaster] => {"changed": false, "checksum": "057b27e8be70c510af6afaea2152360c538bb28d", "dest": "/etc/kubernetes/kubeadm-images.yaml", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/kubernetes/kubeadm-images.yaml", "secontext": "system_u:object_r:etc_t:s0", "size": 422, "state": "file", "uid": 0}
TASK [download : prep_kubeadm_images | Copy kubeadm binary from download dir to system path] ***********************************************************************************************************************************************
Sunday 13 October 2019 01:50:15 +0900 (0:00:00.818) 0:01:16.815 ********
changed: [kubemaster -> None] => {"changed": true, "cmd": "/usr/bin/rsync --delay-updates -F --archive --no-owner --no-group --out-format=<<CHANGED>>%i %n%L /tmp/releases/kubeadm-v1.16.0-amd64 /usr/local/bin/kubeadm", "msg": ">f..t...... kubeadm-v1.16.0-amd64\n", "rc": 0, "stdout_lines": [">f..t...... kubeadm-v1.16.0-amd64"]}
TASK [download : prep_kubeadm_images | Set kubeadm binary permissions] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:16 +0900 (0:00:00.812) 0:01:17.627 ********
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/local/bin/kubeadm", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 44244800, "state": "file", "uid": 0}
TASK [download : prep_kubeadm_images | Generate list of required images] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:16 +0900 (0:00:00.270) 0:01:17.898 ********
ok: [kubemaster] => {"changed": false, "cmd": ["/usr/local/bin/kubeadm", "config", "images", "list", "--config=/etc/kubernetes/kubeadm-images.yaml"], "delta": "0:00:00.045104", "end": "2019-10-13 01:50:16.705316", "rc": 0, "start": "2019-10-13 01:50:16.660212", "stderr": "", "stderr_lines": [], "stdout": "gcr.io/google-containers/kube-apiserver:v1.16.0\ngcr.io/google-containers/kube-controller-manager:v1.16.0\ngcr.io/google-containers/kube-scheduler:v1.16.0\ngcr.io/google-containers/kube-proxy:v1.16.0\ngcr.io/google-containers/pause:3.1\ndocker.io/coredns/coredns:1.6.0", "stdout_lines": ["gcr.io/google-containers/kube-apiserver:v1.16.0", "gcr.io/google-containers/kube-controller-manager:v1.16.0", "gcr.io/google-containers/kube-scheduler:v1.16.0", "gcr.io/google-containers/kube-proxy:v1.16.0", "gcr.io/google-containers/pause:3.1", "docker.io/coredns/coredns:1.6.0"]}
TASK [download : prep_kubeadm_images | Parse list of images] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:16 +0900 (0:00:00.313) 0:01:18.211 ********
ok: [kubemaster] => (item=gcr.io/google-containers/kube-apiserver:v1.16.0) => {"ansible_facts": {"kubeadm_image": {"key": "kubeadm_kube-apiserver", "value": {"container": true, "enabled": true, "groups": "k8s-cluster", "repo": "gcr.io/google-containers/kube-apiserver", "tag": "v1.16.0"}}}, "ansible_loop_var": "item", "changed": false, "item": "gcr.io/google-containers/kube-apiserver:v1.16.0"}
ok: [kubemaster] => (item=gcr.io/google-containers/kube-controller-manager:v1.16.0) => {"ansible_facts": {"kubeadm_image": {"key": "kubeadm_kube-controller-manager", "value": {"container": true, "enabled": true, "groups": "k8s-cluster", "repo": "gcr.io/google-containers/kube-controller-manager", "tag": "v1.16.0"}}}, "ansible_loop_var": "item", "changed": false, "item": "gcr.io/google-containers/kube-controller-manager:v1.16.0"}
ok: [kubemaster] => (item=gcr.io/google-containers/kube-scheduler:v1.16.0) => {"ansible_facts": {"kubeadm_image": {"key": "kubeadm_kube-scheduler", "value": {"container": true, "enabled": true, "groups": "k8s-cluster", "repo": "gcr.io/google-containers/kube-scheduler", "tag": "v1.16.0"}}}, "ansible_loop_var": "item", "changed": false, "item": "gcr.io/google-containers/kube-scheduler:v1.16.0"}
ok: [kubemaster] => (item=gcr.io/google-containers/kube-proxy:v1.16.0) => {"ansible_facts": {"kubeadm_image": {"key": "kubeadm_kube-proxy", "value": {"container": true, "enabled": true, "groups": "k8s-cluster", "repo": "gcr.io/google-containers/kube-proxy", "tag": "v1.16.0"}}}, "ansible_loop_var": "item", "changed": false, "item": "gcr.io/google-containers/kube-proxy:v1.16.0"}
ok: [kubemaster] => (item=gcr.io/google-containers/pause:3.1) => {"ansible_facts": {"kubeadm_image": {"key": "kubeadm_pause", "value": {"container": true, "enabled": true, "groups": "k8s-cluster", "repo": "gcr.io/google-containers/pause", "tag": "3.1"}}}, "ansible_loop_var": "item", "changed": false, "item": "gcr.io/google-containers/pause:3.1"}
ok: [kubemaster] => (item=docker.io/coredns/coredns:1.6.0) => {"ansible_facts": {"kubeadm_image": {"key": "kubeadm_coredns", "value": {"container": true, "enabled": true, "groups": "k8s-cluster", "repo": "docker.io/coredns/coredns", "tag": "1.6.0"}}}, "ansible_loop_var": "item", "changed": false, "item": "docker.io/coredns/coredns:1.6.0"}
TASK [download : prep_kubeadm_images | Convert list of images to dict for later use] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:50:17 +0900 (0:00:00.443) 0:01:18.655 ********
ok: [kubemaster] => {"ansible_facts": {"kubeadm_images": {"kubeadm_coredns": {"container": true, "enabled": true, "groups": "k8s-cluster", "repo": "docker.io/coredns/coredns", "tag": "1.6.0"}, "kubeadm_kube-apiserver": {"container": true, "enabled": true, "groups": "k8s-cluster", "repo": "gcr.io/google-containers/kube-apiserver", "tag": "v1.16.0"}, "kubeadm_kube-controller-manager": {"container": true, "enabled": true, "groups": "k8s-cluster", "repo": "gcr.io/google-containers/kube-controller-manager", "tag": "v1.16.0"}, "kubeadm_kube-proxy": {"container": true, "enabled": true, "groups": "k8s-cluster", "repo": "gcr.io/google-containers/kube-proxy", "tag": "v1.16.0"}, "kubeadm_kube-scheduler": {"container": true, "enabled": true, "groups": "k8s-cluster", "repo": "gcr.io/google-containers/kube-scheduler", "tag": "v1.16.0"}, "kubeadm_pause": {"container": true, "enabled": true, "groups": "k8s-cluster", "repo": "gcr.io/google-containers/pause", "tag": "3.1"}}}, "changed": false}
TASK [download : download | Download files / images] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:17 +0900 (0:00:00.123) 0:01:18.778 ********
included: /root/kubespray/kubespray/roles/download/tasks/download_file.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_file.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_file.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_file.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubeworker, kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubemaster
included: /root/kubespray/kubespray/roles/download/tasks/download_container.yml for kubemaster
TASK [download : download_file | Starting download of file] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:21 +0900 (0:00:04.277) 0:01:23.056 ********
ok: [kubeworker] => {
"msg": "https://github.com/containernetworking/plugins/releases/download/v0.8.1/cni-plugins-linux-amd64-v0.8.1.tgz"
}
ok: [kubemaster] => {
"msg": "https://github.com/containernetworking/plugins/releases/download/v0.8.1/cni-plugins-linux-amd64-v0.8.1.tgz"
}
TASK [download : download_file | Set pathname of cached file] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:21 +0900 (0:00:00.180) 0:01:23.236 ********
ok: [kubeworker] => {"ansible_facts": {"file_path_cached": "/tmp/kubespray_cache/cni-plugins-linux-amd64-v0.8.1.tgz"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"file_path_cached": "/tmp/kubespray_cache/cni-plugins-linux-amd64-v0.8.1.tgz"}, "changed": false}
TASK [download : download_file | Create dest directory on node] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:21 +0900 (0:00:00.178) 0:01:23.415 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/releases", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 139, "state": "directory", "uid": 0}
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/releases", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 49, "state": "directory", "uid": 0}
TASK [download : download_file | Create local cache directory] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:22 +0900 (0:00:00.321) 0:01:23.737 ********
ok: [kubeworker -> localhost] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/kubespray_cache", "size": 20, "state": "directory", "uid": 0}
TASK [download : download_file | Check if file is available in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:22 +0900 (0:00:00.336) 0:01:24.073 ********
TASK [download : download_file | Set file_is_cached fact based on previous task] ***********************************************************************************************************************************************************
Sunday 13 October 2019 01:50:22 +0900 (0:00:00.072) 0:01:24.146 ********
TASK [download : download_file | Copy file from cache to nodes, if it is available] ********************************************************************************************************************************************************
Sunday 13 October 2019 01:50:22 +0900 (0:00:00.137) 0:01:24.283 ********
TASK [download : download_file | Set mode and owner] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:22 +0900 (0:00:00.134) 0:01:24.418 ********
TASK [download : download_file | Download item] ********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:50:23 +0900 (0:00:00.190) 0:01:24.608 ********
ok: [kubeworker -> None] => {"attempts": 1, "changed": false, "checksum_dest": null, "checksum_src": null, "dest": "/tmp/releases/cni-plugins-linux-amd64-v0.8.1.tgz", "elapsed": 0, "gid": 0, "group": "root", "mode": "0755", "msg": "file already exists", "owner": "root", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 36646402, "state": "file", "uid": 0, "url": "https://github.com/containernetworking/plugins/releases/download/v0.8.1/cni-plugins-linux-amd64-v0.8.1.tgz"}
changed: [kubemaster -> None] => {"attempts": 1, "changed": true, "checksum_dest": null, "checksum_src": "662d4c4a3234ca36f5dd594878fa48b7a365df76", "dest": "/tmp/releases/cni-plugins-linux-amd64-v0.8.1.tgz", "elapsed": 113, "gid": 0, "group": "root", "md5sum": "d32fc1ca45ce3264fb439c69e678d08e", "mode": "0755", "msg": "OK (36646402 bytes)", "owner": "root", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 36646402, "src": "/root/.ansible/tmp/ansible-moduletmp-1570899023.54-MmeNxL/tmpBhPz2_", "state": "file", "status_code": 200, "uid": 0, "url": "https://github.com/containernetworking/plugins/releases/download/v0.8.1/cni-plugins-linux-amd64-v0.8.1.tgz"}
TASK [download : download_file | Extract file archives] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:17 +0900 (0:01:54.514) 0:03:19.122 ********
included: /root/kubespray/kubespray/roles/download/tasks/extract_file.yml for kubeworker, kubemaster
TASK [download : extract_file | Unpacking archive] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:17 +0900 (0:00:00.270) 0:03:19.392 ********
TASK [download : download_file | Copy file back to ansible host file cache] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:18 +0900 (0:00:00.139) 0:03:19.532 ********
TASK [download : download_file | Starting download of file] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:18 +0900 (0:00:00.129) 0:03:19.661 ********
ok: [kubeworker] => {
"msg": "https://storage.googleapis.com/kubernetes-release/release/v1.16.0/bin/linux/amd64/kubeadm"
}
ok: [kubemaster] => {
"msg": "https://storage.googleapis.com/kubernetes-release/release/v1.16.0/bin/linux/amd64/kubeadm"
}
TASK [download : download_file | Set pathname of cached file] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:18 +0900 (0:00:00.227) 0:03:19.889 ********
ok: [kubeworker] => {"ansible_facts": {"file_path_cached": "/tmp/kubespray_cache/kubeadm-v1.16.0-amd64"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"file_path_cached": "/tmp/kubespray_cache/kubeadm-v1.16.0-amd64"}, "changed": false}
TASK [download : download_file | Create dest directory on node] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:18 +0900 (0:00:00.177) 0:03:20.067 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/releases", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 139, "state": "directory", "uid": 0}
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/releases", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 91, "state": "directory", "uid": 0}
TASK [download : download_file | Create local cache directory] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:18 +0900 (0:00:00.322) 0:03:20.390 ********
ok: [kubeworker -> localhost] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/kubespray_cache", "size": 20, "state": "directory", "uid": 0}
TASK [download : download_file | Check if file is available in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:19 +0900 (0:00:00.355) 0:03:20.746 ********
TASK [download : download_file | Set file_is_cached fact based on previous task] ***********************************************************************************************************************************************************
Sunday 13 October 2019 01:52:19 +0900 (0:00:00.090) 0:03:20.836 ********
TASK [download : download_file | Copy file from cache to nodes, if it is available] ********************************************************************************************************************************************************
Sunday 13 October 2019 01:52:19 +0900 (0:00:00.160) 0:03:20.997 ********
TASK [download : download_file | Set mode and owner] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:19 +0900 (0:00:00.130) 0:03:21.128 ********
TASK [download : download_file | Download item] ********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:19 +0900 (0:00:00.180) 0:03:21.308 ********
ok: [kubeworker -> None] => {"attempts": 1, "changed": false, "checksum_dest": null, "checksum_src": null, "dest": "/tmp/releases/kubeadm-v1.16.0-amd64", "elapsed": 0, "gid": 0, "group": "root", "mode": "0755", "msg": "file already exists", "owner": "root", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 44244800, "state": "file", "uid": 0, "url": "https://storage.googleapis.com/kubernetes-release/release/v1.16.0/bin/linux/amd64/kubeadm"}
ok: [kubemaster -> None] => {"attempts": 1, "changed": false, "checksum_dest": null, "checksum_src": null, "dest": "/tmp/releases/kubeadm-v1.16.0-amd64", "elapsed": 0, "gid": 0, "group": "root", "mode": "0755", "msg": "file already exists", "owner": "root", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 44244800, "state": "file", "uid": 0, "url": "https://storage.googleapis.com/kubernetes-release/release/v1.16.0/bin/linux/amd64/kubeadm"}
TASK [download : download_file | Extract file archives] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:20 +0900 (0:00:00.567) 0:03:21.876 ********
included: /root/kubespray/kubespray/roles/download/tasks/extract_file.yml for kubeworker, kubemaster
TASK [download : extract_file | Unpacking archive] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:20 +0900 (0:00:00.301) 0:03:22.178 ********
TASK [download : download_file | Copy file back to ansible host file cache] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:20 +0900 (0:00:00.159) 0:03:22.337 ********
TASK [download : download_file | Starting download of file] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:21 +0900 (0:00:00.140) 0:03:22.477 ********
ok: [kubeworker] => {
"msg": "https://storage.googleapis.com/kubernetes-release/release/v1.16.0/bin/linux/amd64/hyperkube"
}
ok: [kubemaster] => {
"msg": "https://storage.googleapis.com/kubernetes-release/release/v1.16.0/bin/linux/amd64/hyperkube"
}
TASK [download : download_file | Set pathname of cached file] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:21 +0900 (0:00:00.180) 0:03:22.658 ********
ok: [kubeworker] => {"ansible_facts": {"file_path_cached": "/tmp/kubespray_cache/hyperkube-v1.16.0-amd64"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"file_path_cached": "/tmp/kubespray_cache/hyperkube-v1.16.0-amd64"}, "changed": false}
TASK [download : download_file | Create dest directory on node] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:21 +0900 (0:00:00.170) 0:03:22.828 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/releases", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 139, "state": "directory", "uid": 0}
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/releases", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 91, "state": "directory", "uid": 0}
TASK [download : download_file | Create local cache directory] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:21 +0900 (0:00:00.346) 0:03:23.174 ********
ok: [kubeworker -> localhost] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/kubespray_cache", "size": 20, "state": "directory", "uid": 0}
TASK [download : download_file | Check if file is available in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:22 +0900 (0:00:00.356) 0:03:23.531 ********
TASK [download : download_file | Set file_is_cached fact based on previous task] ***********************************************************************************************************************************************************
Sunday 13 October 2019 01:52:22 +0900 (0:00:00.072) 0:03:23.603 ********
TASK [download : download_file | Copy file from cache to nodes, if it is available] ********************************************************************************************************************************************************
Sunday 13 October 2019 01:52:22 +0900 (0:00:00.136) 0:03:23.740 ********
TASK [download : download_file | Set mode and owner] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:22 +0900 (0:00:00.131) 0:03:23.872 ********
TASK [download : download_file | Download item] ********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:22 +0900 (0:00:00.195) 0:03:24.067 ********
ok: [kubeworker -> None] => {"attempts": 1, "changed": false, "checksum_dest": null, "checksum_src": null, "dest": "/tmp/releases/hyperkube-v1.16.0-amd64", "elapsed": 0, "gid": 0, "group": "root", "mode": "0755", "msg": "file already exists", "owner": "root", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 212345584, "state": "file", "uid": 0, "url": "https://storage.googleapis.com/kubernetes-release/release/v1.16.0/bin/linux/amd64/hyperkube"}
changed: [kubemaster -> None] => {"attempts": 1, "changed": true, "checksum_dest": null, "checksum_src": "c46d81ebdc9891ac5f25e5196dfc8e9279a27cd0", "dest": "/tmp/releases/hyperkube-v1.16.0-amd64", "elapsed": 20, "gid": 0, "group": "root", "md5sum": "cccfd10a3e15112e99e9ba8e58f15f80", "mode": "0755", "msg": "OK (212345584 bytes)", "owner": "root", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 212345584, "src": "/root/.ansible/tmp/ansible-moduletmp-1570899143.03-cemQMo/tmplUJdxN", "state": "file", "status_code": 200, "uid": 0, "url": "https://storage.googleapis.com/kubernetes-release/release/v1.16.0/bin/linux/amd64/hyperkube"}
TASK [download : download_file | Extract file archives] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:44 +0900 (0:00:21.888) 0:03:45.956 ********
included: /root/kubespray/kubespray/roles/download/tasks/extract_file.yml for kubeworker, kubemaster
TASK [download : extract_file | Unpacking archive] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:44 +0900 (0:00:00.273) 0:03:46.230 ********
TASK [download : download_file | Copy file back to ansible host file cache] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:44 +0900 (0:00:00.133) 0:03:46.363 ********
TASK [download : download_file | Starting download of file] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:45 +0900 (0:00:00.123) 0:03:46.487 ********
ok: [kubeworker] => {
"msg": "https://github.com/projectcalico/calicoctl/releases/download/v3.7.3/calicoctl-linux-amd64"
}
ok: [kubemaster] => {
"msg": "https://github.com/projectcalico/calicoctl/releases/download/v3.7.3/calicoctl-linux-amd64"
}
TASK [download : download_file | Set pathname of cached file] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:45 +0900 (0:00:00.200) 0:03:46.688 ********
ok: [kubeworker] => {"ansible_facts": {"file_path_cached": "/tmp/kubespray_cache/calicoctl"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"file_path_cached": "/tmp/kubespray_cache/calicoctl"}, "changed": false}
TASK [download : download_file | Create dest directory on node] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:45 +0900 (0:00:00.173) 0:03:46.861 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/releases", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 139, "state": "directory", "uid": 0}
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/releases", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 122, "state": "directory", "uid": 0}
TASK [download : download_file | Create local cache directory] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:45 +0900 (0:00:00.349) 0:03:47.211 ********
ok: [kubeworker -> localhost] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/tmp/kubespray_cache", "size": 20, "state": "directory", "uid": 0}
TASK [download : download_file | Check if file is available in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:46 +0900 (0:00:00.343) 0:03:47.555 ********
TASK [download : download_file | Set file_is_cached fact based on previous task] ***********************************************************************************************************************************************************
Sunday 13 October 2019 01:52:46 +0900 (0:00:00.072) 0:03:47.628 ********
TASK [download : download_file | Copy file from cache to nodes, if it is available] ********************************************************************************************************************************************************
Sunday 13 October 2019 01:52:46 +0900 (0:00:00.122) 0:03:47.750 ********
TASK [download : download_file | Set mode and owner] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:46 +0900 (0:00:00.136) 0:03:47.887 ********
TASK [download : download_file | Download item] ********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:52:46 +0900 (0:00:00.183) 0:03:48.070 ********
ok: [kubeworker -> None] => {"attempts": 1, "changed": false, "checksum_dest": null, "checksum_src": null, "dest": "/tmp/releases/calicoctl", "elapsed": 0, "gid": 0, "group": "root", "mode": "0755", "msg": "file already exists", "owner": "root", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 35861536, "state": "file", "uid": 0, "url": "https://github.com/projectcalico/calicoctl/releases/download/v3.7.3/calicoctl-linux-amd64"}
changed: [kubemaster -> None] => {"attempts": 1, "changed": true, "checksum_dest": null, "checksum_src": "84f9e061d85fddf6437700a90af18084654cbb21", "dest": "/tmp/releases/calicoctl", "elapsed": 148, "gid": 0, "group": "root", "md5sum": "7840346b42d3df44cecb256266104e88", "mode": "0755", "msg": "OK (35861536 bytes)", "owner": "root", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 35861536, "src": "/root/.ansible/tmp/ansible-moduletmp-1570899167.04-143naF/tmpluw3eV", "state": "file", "status_code": 200, "uid": 0, "url": "https://github.com/projectcalico/calicoctl/releases/download/v3.7.3/calicoctl-linux-amd64"}
TASK [download : download_file | Extract file archives] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:15 +0900 (0:02:29.068) 0:06:17.139 ********
included: /root/kubespray/kubespray/roles/download/tasks/extract_file.yml for kubeworker, kubemaster
TASK [download : extract_file | Unpacking archive] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:15 +0900 (0:00:00.277) 0:06:17.417 ********
TASK [download : download_file | Copy file back to ansible host file cache] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:16 +0900 (0:00:00.133) 0:06:17.550 ********
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:55:16 +0900 (0:00:00.125) 0:06:17.676 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker, kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:16 +0900 (0:00:00.300) 0:06:17.977 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:16 +0900 (0:00:00.194) 0:06:18.171 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "docker.io/calico/node:v3.7.3"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_args": "docker.io/calico/node:v3.7.3"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:16 +0900 (0:00:00.183) 0:06:18.354 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:17 +0900 (0:00:00.137) 0:06:18.492 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:17 +0900 (0:00:00.434) 0:06:18.926 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:17 +0900 (0:00:00.201) 0:06:19.128 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:55:17 +0900 (0:00:00.230) 0:06:19.358 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:55:18 +0900 (0:00:00.158) 0:06:19.517 ********
ok: [kubeworker] => {
"msg": "docker.io/calico/node"
}
ok: [kubemaster] => {
"msg": "docker.io/calico/node"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:55:18 +0900 (0:00:00.175) 0:06:19.692 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:18 +0900 (0:00:00.182) 0:06:19.875 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "docker.io/calico/node:v3.7.3"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "docker.io/calico/node:v3.7.3"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:18 +0900 (0:00:00.187) 0:06:20.062 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "docker.io_calico_node_v3.7.3.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_filename": "docker.io_calico_node_v3.7.3.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:18 +0900 (0:00:00.169) 0:06:20.232 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/docker.io_calico_node_v3.7.3.tar", "image_path_final": "/tmp/releases/images/docker.io_calico_node_v3.7.3.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/docker.io_calico_node_v3.7.3.tar", "image_path_final": "/tmp/releases/images/docker.io_calico_node_v3.7.3.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:18 +0900 (0:00:00.177) 0:06:20.410 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:19 +0900 (0:00:00.139) 0:06:20.550 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:19 +0900 (0:00:00.127) 0:06:20.677 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:19 +0900 (0:00:00.190) 0:06:20.868 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:19 +0900 (0:00:00.224) 0:06:21.092 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker, kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:55:20 +0900 (0:00:00.398) 0:06:21.491 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:55:20 +0900 (0:00:00.490) 0:06:21.981 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:55:20 +0900 (0:00:00.186) 0:06:22.168 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:20 +0900 (0:00:00.163) 0:06:22.332 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: True"
}
ok: [kubemaster] => {
"msg": "XXX Pull required is: True"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:21 +0900 (0:00:00.247) 0:06:22.579 ********
changed: [kubeworker -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "docker.io/calico/node:v3.7.3"], "delta": "0:00:05.440722", "end": "2019-10-13 01:55:26.789636", "rc": 0, "start": "2019-10-13 01:55:21.348914", "stderr": "", "stderr_lines": [], "stdout": "v3.7.3: Pulling from calico/node\nDigest: sha256:a2782b53500c96e35299b8af729eaf39423f9ffd903d9fda675073f4a063502a\nStatus: Image is up to date for calico/node:v3.7.3\ndocker.io/calico/node:v3.7.3", "stdout_lines": ["v3.7.3: Pulling from calico/node", "Digest: sha256:a2782b53500c96e35299b8af729eaf39423f9ffd903d9fda675073f4a063502a", "Status: Image is up to date for calico/node:v3.7.3", "docker.io/calico/node:v3.7.3"]}
changed: [kubemaster -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "docker.io/calico/node:v3.7.3"], "delta": "0:00:05.824908", "end": "2019-10-13 01:55:27.298334", "rc": 0, "start": "2019-10-13 01:55:21.473426", "stderr": "", "stderr_lines": [], "stdout": "v3.7.3: Pulling from calico/node\nDigest: sha256:a2782b53500c96e35299b8af729eaf39423f9ffd903d9fda675073f4a063502a\nStatus: Image is up to date for calico/node:v3.7.3\ndocker.io/calico/node:v3.7.3", "stdout_lines": ["v3.7.3: Pulling from calico/node", "Digest: sha256:a2782b53500c96e35299b8af729eaf39423f9ffd903d9fda675073f4a063502a", "Status: Image is up to date for calico/node:v3.7.3", "docker.io/calico/node:v3.7.3"]}
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:27 +0900 (0:00:06.223) 0:06:28.803 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:27 +0900 (0:00:00.129) 0:06:28.932 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:27 +0900 (0:00:00.088) 0:06:29.021 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:27 +0900 (0:00:00.139) 0:06:29.161 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/docker.io_calico_node_v3.7.3.tar", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/docker.io_calico_node_v3.7.3.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:55:28 +0900 (0:00:00.338) 0:06:29.499 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker, kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:28 +0900 (0:00:00.274) 0:06:29.774 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:28 +0900 (0:00:00.175) 0:06:29.950 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "docker.io/calico/cni:v3.7.3"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_args": "docker.io/calico/cni:v3.7.3"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:28 +0900 (0:00:00.180) 0:06:30.131 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:28 +0900 (0:00:00.143) 0:06:30.274 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:29 +0900 (0:00:00.416) 0:06:30.691 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:29 +0900 (0:00:00.196) 0:06:30.887 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:55:29 +0900 (0:00:00.232) 0:06:31.120 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:55:29 +0900 (0:00:00.144) 0:06:31.265 ********
ok: [kubeworker] => {
"msg": "docker.io/calico/cni"
}
ok: [kubemaster] => {
"msg": "docker.io/calico/cni"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:55:30 +0900 (0:00:00.176) 0:06:31.441 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:30 +0900 (0:00:00.182) 0:06:31.624 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "docker.io/calico/cni:v3.7.3"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "docker.io/calico/cni:v3.7.3"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:30 +0900 (0:00:00.169) 0:06:31.793 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "docker.io_calico_cni_v3.7.3.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_filename": "docker.io_calico_cni_v3.7.3.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:30 +0900 (0:00:00.178) 0:06:31.972 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/docker.io_calico_cni_v3.7.3.tar", "image_path_final": "/tmp/releases/images/docker.io_calico_cni_v3.7.3.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/docker.io_calico_cni_v3.7.3.tar", "image_path_final": "/tmp/releases/images/docker.io_calico_cni_v3.7.3.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:30 +0900 (0:00:00.207) 0:06:32.180 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:30 +0900 (0:00:00.144) 0:06:32.325 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:31 +0900 (0:00:00.140) 0:06:32.465 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:31 +0900 (0:00:00.188) 0:06:32.654 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:31 +0900 (0:00:00.209) 0:06:32.863 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker, kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:55:31 +0900 (0:00:00.353) 0:06:33.217 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:55:32 +0900 (0:00:00.490) 0:06:33.707 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:55:32 +0900 (0:00:00.197) 0:06:33.904 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:32 +0900 (0:00:00.131) 0:06:34.036 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: True"
}
ok: [kubemaster] => {
"msg": "XXX Pull required is: True"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:32 +0900 (0:00:00.230) 0:06:34.267 ********
changed: [kubeworker -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "docker.io/calico/cni:v3.7.3"], "delta": "0:00:05.235396", "end": "2019-10-13 01:55:38.325042", "rc": 0, "start": "2019-10-13 01:55:33.089646", "stderr": "", "stderr_lines": [], "stdout": "v3.7.3: Pulling from calico/cni\nDigest: sha256:258a0cb3c25022e44ebda3606112c40865adb67b8fb7be3d119f960957301ad6\nStatus: Image is up to date for calico/cni:v3.7.3\ndocker.io/calico/cni:v3.7.3", "stdout_lines": ["v3.7.3: Pulling from calico/cni", "Digest: sha256:258a0cb3c25022e44ebda3606112c40865adb67b8fb7be3d119f960957301ad6", "Status: Image is up to date for calico/cni:v3.7.3", "docker.io/calico/cni:v3.7.3"]}
changed: [kubemaster -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "docker.io/calico/cni:v3.7.3"], "delta": "0:00:06.064484", "end": "2019-10-13 01:55:39.228828", "rc": 0, "start": "2019-10-13 01:55:33.164344", "stderr": "", "stderr_lines": [], "stdout": "v3.7.3: Pulling from calico/cni\nDigest: sha256:258a0cb3c25022e44ebda3606112c40865adb67b8fb7be3d119f960957301ad6\nStatus: Image is up to date for calico/cni:v3.7.3\ndocker.io/calico/cni:v3.7.3", "stdout_lines": ["v3.7.3: Pulling from calico/cni", "Digest: sha256:258a0cb3c25022e44ebda3606112c40865adb67b8fb7be3d119f960957301ad6", "Status: Image is up to date for calico/cni:v3.7.3", "docker.io/calico/cni:v3.7.3"]}
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:39 +0900 (0:00:06.464) 0:06:40.732 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:39 +0900 (0:00:00.141) 0:06:40.874 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:39 +0900 (0:00:00.086) 0:06:40.960 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:39 +0900 (0:00:00.146) 0:06:41.107 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/docker.io_calico_cni_v3.7.3.tar", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/docker.io_calico_cni_v3.7.3.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:55:40 +0900 (0:00:00.336) 0:06:41.443 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker, kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:40 +0900 (0:00:00.293) 0:06:41.736 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:40 +0900 (0:00:00.186) 0:06:41.923 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "docker.io/calico/kube-controllers:v3.7.3"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_args": "docker.io/calico/kube-controllers:v3.7.3"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:40 +0900 (0:00:00.178) 0:06:42.101 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:40 +0900 (0:00:00.152) 0:06:42.254 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:41 +0900 (0:00:00.436) 0:06:42.690 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:41 +0900 (0:00:00.189) 0:06:42.880 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:55:41 +0900 (0:00:00.217) 0:06:43.098 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:55:41 +0900 (0:00:00.144) 0:06:43.242 ********
ok: [kubeworker] => {
"msg": "docker.io/calico/kube-controllers"
}
ok: [kubemaster] => {
"msg": "docker.io/calico/kube-controllers"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:55:41 +0900 (0:00:00.184) 0:06:43.427 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:42 +0900 (0:00:00.176) 0:06:43.604 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "docker.io/calico/kube-controllers:v3.7.3"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "docker.io/calico/kube-controllers:v3.7.3"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:42 +0900 (0:00:00.188) 0:06:43.792 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "docker.io_calico_kube-controllers_v3.7.3.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_filename": "docker.io_calico_kube-controllers_v3.7.3.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:42 +0900 (0:00:00.181) 0:06:43.973 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/docker.io_calico_kube-controllers_v3.7.3.tar", "image_path_final": "/tmp/releases/images/docker.io_calico_kube-controllers_v3.7.3.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/docker.io_calico_kube-controllers_v3.7.3.tar", "image_path_final": "/tmp/releases/images/docker.io_calico_kube-controllers_v3.7.3.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:42 +0900 (0:00:00.191) 0:06:44.165 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:42 +0900 (0:00:00.152) 0:06:44.317 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:43 +0900 (0:00:00.140) 0:06:44.458 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:43 +0900 (0:00:00.180) 0:06:44.639 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:43 +0900 (0:00:00.202) 0:06:44.842 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker, kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:55:43 +0900 (0:00:00.354) 0:06:45.196 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:55:44 +0900 (0:00:00.501) 0:06:45.697 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:55:44 +0900 (0:00:00.209) 0:06:45.907 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:44 +0900 (0:00:00.131) 0:06:46.038 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: True"
}
ok: [kubemaster] => {
"msg": "XXX Pull required is: True"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:44 +0900 (0:00:00.235) 0:06:46.273 ********
changed: [kubeworker -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "docker.io/calico/kube-controllers:v3.7.3"], "delta": "0:00:05.558305", "end": "2019-10-13 01:55:50.592093", "rc": 0, "start": "2019-10-13 01:55:45.033788", "stderr": "", "stderr_lines": [], "stdout": "v3.7.3: Pulling from calico/kube-controllers\nDigest: sha256:6bc3fd181fabb580df33442d728b39d4eeccae0883a50d85b1a26cac796a2999\nStatus: Image is up to date for calico/kube-controllers:v3.7.3\ndocker.io/calico/kube-controllers:v3.7.3", "stdout_lines": ["v3.7.3: Pulling from calico/kube-controllers", "Digest: sha256:6bc3fd181fabb580df33442d728b39d4eeccae0883a50d85b1a26cac796a2999", "Status: Image is up to date for calico/kube-controllers:v3.7.3", "docker.io/calico/kube-controllers:v3.7.3"]}
changed: [kubemaster -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "docker.io/calico/kube-controllers:v3.7.3"], "delta": "0:00:05.566259", "end": "2019-10-13 01:55:50.722388", "rc": 0, "start": "2019-10-13 01:55:45.156129", "stderr": "", "stderr_lines": [], "stdout": "v3.7.3: Pulling from calico/kube-controllers\nDigest: sha256:6bc3fd181fabb580df33442d728b39d4eeccae0883a50d85b1a26cac796a2999\nStatus: Image is up to date for calico/kube-controllers:v3.7.3\ndocker.io/calico/kube-controllers:v3.7.3", "stdout_lines": ["v3.7.3: Pulling from calico/kube-controllers", "Digest: sha256:6bc3fd181fabb580df33442d728b39d4eeccae0883a50d85b1a26cac796a2999", "Status: Image is up to date for calico/kube-controllers:v3.7.3", "docker.io/calico/kube-controllers:v3.7.3"]}
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:50 +0900 (0:00:05.960) 0:06:52.233 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:50 +0900 (0:00:00.140) 0:06:52.374 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:51 +0900 (0:00:00.100) 0:06:52.474 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:51 +0900 (0:00:00.143) 0:06:52.618 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/docker.io_calico_kube-controllers_v3.7.3.tar", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/docker.io_calico_kube-controllers_v3.7.3.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:55:51 +0900 (0:00:00.348) 0:06:52.967 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker, kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:51 +0900 (0:00:00.311) 0:06:53.279 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:52 +0900 (0:00:00.192) 0:06:53.471 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "gcr.io/google_containers/pause-amd64:3.1"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_args": "gcr.io/google_containers/pause-amd64:3.1"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:52 +0900 (0:00:00.186) 0:06:53.657 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:52 +0900 (0:00:00.132) 0:06:53.790 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:52 +0900 (0:00:00.434) 0:06:54.224 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:52 +0900 (0:00:00.191) 0:06:54.415 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:55:53 +0900 (0:00:00.242) 0:06:54.658 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:55:53 +0900 (0:00:00.151) 0:06:54.810 ********
ok: [kubeworker] => {
"msg": "gcr.io/google_containers/pause-amd64"
}
ok: [kubemaster] => {
"msg": "gcr.io/google_containers/pause-amd64"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:55:53 +0900 (0:00:00.195) 0:06:55.005 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:53 +0900 (0:00:00.177) 0:06:55.183 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "gcr.io/google_containers/pause-amd64:3.1"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "gcr.io/google_containers/pause-amd64:3.1"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:53 +0900 (0:00:00.187) 0:06:55.370 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "gcr.io_google_containers_pause-amd64_3.1.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_filename": "gcr.io_google_containers_pause-amd64_3.1.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:54 +0900 (0:00:00.171) 0:06:55.542 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google_containers_pause-amd64_3.1.tar", "image_path_final": "/tmp/releases/images/gcr.io_google_containers_pause-amd64_3.1.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google_containers_pause-amd64_3.1.tar", "image_path_final": "/tmp/releases/images/gcr.io_google_containers_pause-amd64_3.1.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:54 +0900 (0:00:00.186) 0:06:55.728 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:54 +0900 (0:00:00.134) 0:06:55.862 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:54 +0900 (0:00:00.172) 0:06:56.035 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:54 +0900 (0:00:00.197) 0:06:56.233 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:55 +0900 (0:00:00.216) 0:06:56.449 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker, kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:55:55 +0900 (0:00:00.348) 0:06:56.798 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:55:55 +0900 (0:00:00.501) 0:06:57.299 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:55:56 +0900 (0:00:00.183) 0:06:57.483 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:56 +0900 (0:00:00.147) 0:06:57.631 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: True"
}
ok: [kubemaster] => {
"msg": "XXX Pull required is: True"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:56 +0900 (0:00:00.224) 0:06:57.856 ********
changed: [kubemaster -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "gcr.io/google_containers/pause-amd64:3.1"], "delta": "0:00:02.337913", "end": "2019-10-13 01:55:59.091165", "rc": 0, "start": "2019-10-13 01:55:56.753252", "stderr": "", "stderr_lines": [], "stdout": "3.1: Pulling from google_containers/pause-amd64\nDigest: sha256:59eec8837a4d942cc19a52b8c09ea75121acc38114a2c68b98983ce9356b8610\nStatus: Image is up to date for gcr.io/google_containers/pause-amd64:3.1\ngcr.io/google_containers/pause-amd64:3.1", "stdout_lines": ["3.1: Pulling from google_containers/pause-amd64", "Digest: sha256:59eec8837a4d942cc19a52b8c09ea75121acc38114a2c68b98983ce9356b8610", "Status: Image is up to date for gcr.io/google_containers/pause-amd64:3.1", "gcr.io/google_containers/pause-amd64:3.1"]}
changed: [kubeworker -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "gcr.io/google_containers/pause-amd64:3.1"], "delta": "0:00:02.894385", "end": "2019-10-13 01:55:59.512223", "rc": 0, "start": "2019-10-13 01:55:56.617838", "stderr": "", "stderr_lines": [], "stdout": "3.1: Pulling from google_containers/pause-amd64\nDigest: sha256:59eec8837a4d942cc19a52b8c09ea75121acc38114a2c68b98983ce9356b8610\nStatus: Image is up to date for gcr.io/google_containers/pause-amd64:3.1\ngcr.io/google_containers/pause-amd64:3.1", "stdout_lines": ["3.1: Pulling from google_containers/pause-amd64", "Digest: sha256:59eec8837a4d942cc19a52b8c09ea75121acc38114a2c68b98983ce9356b8610", "Status: Image is up to date for gcr.io/google_containers/pause-amd64:3.1", "gcr.io/google_containers/pause-amd64:3.1"]}
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:59 +0900 (0:00:03.161) 0:07:01.018 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:59 +0900 (0:00:00.125) 0:07:01.144 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:59 +0900 (0:00:00.097) 0:07:01.242 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:55:59 +0900 (0:00:00.135) 0:07:01.378 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google_containers_pause-amd64_3.1.tar", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google_containers_pause-amd64_3.1.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:56:00 +0900 (0:00:00.328) 0:07:01.706 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:00 +0900 (0:00:00.244) 0:07:01.950 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:00 +0900 (0:00:00.125) 0:07:02.076 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "docker.io/library/nginx:1.15"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:00 +0900 (0:00:00.152) 0:07:02.229 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:00 +0900 (0:00:00.085) 0:07:02.315 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:01 +0900 (0:00:00.358) 0:07:02.673 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:01 +0900 (0:00:00.149) 0:07:02.822 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:56:01 +0900 (0:00:00.237) 0:07:03.060 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:56:01 +0900 (0:00:00.108) 0:07:03.169 ********
ok: [kubeworker] => {
"msg": "docker.io/library/nginx"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:56:01 +0900 (0:00:00.134) 0:07:03.303 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:01 +0900 (0:00:00.117) 0:07:03.421 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "docker.io/library/nginx:1.15"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:02 +0900 (0:00:00.120) 0:07:03.541 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "docker.io_library_nginx_1.15.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:02 +0900 (0:00:00.120) 0:07:03.662 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/docker.io_library_nginx_1.15.tar", "image_path_final": "/tmp/releases/images/docker.io_library_nginx_1.15.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:02 +0900 (0:00:00.120) 0:07:03.783 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:02 +0900 (0:00:00.083) 0:07:03.866 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:02 +0900 (0:00:00.072) 0:07:03.939 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:02 +0900 (0:00:00.141) 0:07:04.080 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:02 +0900 (0:00:00.101) 0:07:04.182 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:56:02 +0900 (0:00:00.227) 0:07:04.409 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:56:03 +0900 (0:00:00.367) 0:07:04.777 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:56:03 +0900 (0:00:00.125) 0:07:04.903 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:03 +0900 (0:00:00.105) 0:07:05.008 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: True"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:03 +0900 (0:00:00.179) 0:07:05.188 ********
changed: [kubeworker -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "docker.io/library/nginx:1.15"], "delta": "0:00:13.544237", "end": "2019-10-13 01:56:17.515002", "rc": 0, "start": "2019-10-13 01:56:03.970765", "stderr": "", "stderr_lines": [], "stdout": "1.15: Pulling from library/nginx\n743f2d6c1f65: Pulling fs layer\n6bfc4ec4420a: Pulling fs layer\n688a776db95f: Pulling fs layer\n688a776db95f: Verifying Checksum\n688a776db95f: Download complete\n743f2d6c1f65: Verifying Checksum\n743f2d6c1f65: Download complete\n6bfc4ec4420a: Verifying Checksum\n6bfc4ec4420a: Download complete\n743f2d6c1f65: Pull complete\n6bfc4ec4420a: Pull complete\n688a776db95f: Pull complete\nDigest: sha256:23b4dcdf0d34d4a129755fc6f52e1c6e23bb34ea011b315d87e193033bcd1b68\nStatus: Downloaded newer image for nginx:1.15\ndocker.io/library/nginx:1.15", "stdout_lines": ["1.15: Pulling from library/nginx", "743f2d6c1f65: Pulling fs layer", "6bfc4ec4420a: Pulling fs layer", "688a776db95f: Pulling fs layer", "688a776db95f: Verifying Checksum", "688a776db95f: Download complete", "743f2d6c1f65: Verifying Checksum", "743f2d6c1f65: Download complete", "6bfc4ec4420a: Verifying Checksum", "6bfc4ec4420a: Download complete", "743f2d6c1f65: Pull complete", "6bfc4ec4420a: Pull complete", "688a776db95f: Pull complete", "Digest: sha256:23b4dcdf0d34d4a129755fc6f52e1c6e23bb34ea011b315d87e193033bcd1b68", "Status: Downloaded newer image for nginx:1.15", "docker.io/library/nginx:1.15"]}
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:17 +0900 (0:00:13.844) 0:07:19.032 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:17 +0900 (0:00:00.147) 0:07:19.180 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:17 +0900 (0:00:00.104) 0:07:19.284 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:17 +0900 (0:00:00.091) 0:07:19.376 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/docker.io_library_nginx_1.15.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:56:18 +0900 (0:00:00.255) 0:07:19.632 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker, kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:18 +0900 (0:00:00.308) 0:07:19.941 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:18 +0900 (0:00:00.210) 0:07:20.151 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/k8s-dns-node-cache:1.15.5"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/k8s-dns-node-cache:1.15.5"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:18 +0900 (0:00:00.183) 0:07:20.335 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:19 +0900 (0:00:00.133) 0:07:20.469 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:19 +0900 (0:00:00.438) 0:07:20.907 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:19 +0900 (0:00:00.205) 0:07:21.112 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:56:19 +0900 (0:00:00.224) 0:07:21.337 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:56:20 +0900 (0:00:00.161) 0:07:21.498 ********
ok: [kubeworker] => {
"msg": "gcr.io/google-containers/k8s-dns-node-cache"
}
ok: [kubemaster] => {
"msg": "gcr.io/google-containers/k8s-dns-node-cache"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:56:20 +0900 (0:00:00.173) 0:07:21.672 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:20 +0900 (0:00:00.174) 0:07:21.846 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/k8s-dns-node-cache:1.15.5"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/k8s-dns-node-cache:1.15.5"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:20 +0900 (0:00:00.212) 0:07:22.058 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_k8s-dns-node-cache_1.15.5.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_k8s-dns-node-cache_1.15.5.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:20 +0900 (0:00:00.206) 0:07:22.265 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_k8s-dns-node-cache_1.15.5.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_k8s-dns-node-cache_1.15.5.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_k8s-dns-node-cache_1.15.5.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_k8s-dns-node-cache_1.15.5.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:21 +0900 (0:00:00.190) 0:07:22.455 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:21 +0900 (0:00:00.167) 0:07:22.622 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:21 +0900 (0:00:00.128) 0:07:22.750 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:21 +0900 (0:00:00.213) 0:07:22.964 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:21 +0900 (0:00:00.208) 0:07:23.172 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker, kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:56:22 +0900 (0:00:00.403) 0:07:23.576 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:56:22 +0900 (0:00:00.479) 0:07:24.055 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:56:22 +0900 (0:00:00.197) 0:07:24.252 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:22 +0900 (0:00:00.152) 0:07:24.404 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: False"
}
ok: [kubemaster] => {
"msg": "XXX Pull required is: False"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:23 +0900 (0:00:00.220) 0:07:24.625 ********
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:23 +0900 (0:00:00.190) 0:07:24.816 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:23 +0900 (0:00:00.143) 0:07:24.959 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:23 +0900 (0:00:00.097) 0:07:25.057 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:23 +0900 (0:00:00.129) 0:07:25.187 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_k8s-dns-node-cache_1.15.5.tar", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_k8s-dns-node-cache_1.15.5.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:56:24 +0900 (0:00:00.354) 0:07:25.541 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:24 +0900 (0:00:00.170) 0:07:25.712 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:24 +0900 (0:00:00.121) 0:07:25.834 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "quay.io/external_storage/local-volume-provisioner:v2.3.2"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:24 +0900 (0:00:00.133) 0:07:25.967 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:24 +0900 (0:00:00.090) 0:07:26.057 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:24 +0900 (0:00:00.360) 0:07:26.417 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:25 +0900 (0:00:00.130) 0:07:26.548 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:56:25 +0900 (0:00:00.222) 0:07:26.770 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:56:25 +0900 (0:00:00.093) 0:07:26.864 ********
ok: [kubeworker] => {
"msg": "quay.io/external_storage/local-volume-provisioner"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:56:25 +0900 (0:00:00.120) 0:07:26.984 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:25 +0900 (0:00:00.124) 0:07:27.109 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "quay.io/external_storage/local-volume-provisioner:v2.3.2"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:25 +0900 (0:00:00.121) 0:07:27.231 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "quay.io_external_storage_local-volume-provisioner_v2.3.2.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:25 +0900 (0:00:00.120) 0:07:27.351 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/quay.io_external_storage_local-volume-provisioner_v2.3.2.tar", "image_path_final": "/tmp/releases/images/quay.io_external_storage_local-volume-provisioner_v2.3.2.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:26 +0900 (0:00:00.145) 0:07:27.496 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:26 +0900 (0:00:00.073) 0:07:27.569 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:26 +0900 (0:00:00.093) 0:07:27.663 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:26 +0900 (0:00:00.130) 0:07:27.793 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:26 +0900 (0:00:00.095) 0:07:27.889 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:56:26 +0900 (0:00:00.232) 0:07:28.121 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:56:27 +0900 (0:00:00.388) 0:07:28.510 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:56:27 +0900 (0:00:00.122) 0:07:28.632 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:27 +0900 (0:00:00.087) 0:07:28.720 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: True"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:27 +0900 (0:00:00.176) 0:07:28.897 ********
changed: [kubeworker -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "quay.io/external_storage/local-volume-provisioner:v2.3.2"], "delta": "0:00:14.640489", "end": "2019-10-13 01:56:42.303144", "rc": 0, "start": "2019-10-13 01:56:27.662655", "stderr": "", "stderr_lines": [], "stdout": "v2.3.2: Pulling from external_storage/local-volume-provisioner\n73e3e9d78c61: Pulling fs layer\n0c671217a10b: Pulling fs layer\n283a9d4059a8: Pulling fs layer\n676ee80290ea: Pulling fs layer\n676ee80290ea: Waiting\n283a9d4059a8: Verifying Checksum\n283a9d4059a8: Download complete\n0c671217a10b: Verifying Checksum\n0c671217a10b: Download complete\n73e3e9d78c61: Verifying Checksum\n73e3e9d78c61: Download complete\n73e3e9d78c61: Pull complete\n676ee80290ea: Verifying Checksum\n676ee80290ea: Download complete\n0c671217a10b: Pull complete\n283a9d4059a8: Pull complete\n676ee80290ea: Pull complete\nDigest: sha256:fcd21fbf858f75e202dbdf46cc6ade48c6479fd13548a65b91f68cbe22de6262\nStatus: Downloaded newer image for quay.io/external_storage/local-volume-provisioner:v2.3.2\nquay.io/external_storage/local-volume-provisioner:v2.3.2", "stdout_lines": ["v2.3.2: Pulling from external_storage/local-volume-provisioner", "73e3e9d78c61: Pulling fs layer", "0c671217a10b: Pulling fs layer", "283a9d4059a8: Pulling fs layer", "676ee80290ea: Pulling fs layer", "676ee80290ea: Waiting", "283a9d4059a8: Verifying Checksum", "283a9d4059a8: Download complete", "0c671217a10b: Verifying Checksum", "0c671217a10b: Download complete", "73e3e9d78c61: Verifying Checksum", "73e3e9d78c61: Download complete", "73e3e9d78c61: Pull complete", "676ee80290ea: Verifying Checksum", "676ee80290ea: Download complete", "0c671217a10b: Pull complete", "283a9d4059a8: Pull complete", "676ee80290ea: Pull complete", "Digest: sha256:fcd21fbf858f75e202dbdf46cc6ade48c6479fd13548a65b91f68cbe22de6262", "Status: Downloaded newer image for quay.io/external_storage/local-volume-provisioner:v2.3.2", "quay.io/external_storage/local-volume-provisioner:v2.3.2"]}
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:42 +0900 (0:00:14.918) 0:07:43.815 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:42 +0900 (0:00:00.145) 0:07:43.960 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:42 +0900 (0:00:00.081) 0:07:44.042 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:42 +0900 (0:00:00.098) 0:07:44.140 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/quay.io_external_storage_local-volume-provisioner_v2.3.2.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:56:43 +0900 (0:00:00.309) 0:07:44.450 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:43 +0900 (0:00:00.174) 0:07:44.624 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:43 +0900 (0:00:00.114) 0:07:44.739 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "docker.io/rancher/local-path-provisioner:v0.0.2"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:43 +0900 (0:00:00.115) 0:07:44.854 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:43 +0900 (0:00:00.092) 0:07:44.947 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:43 +0900 (0:00:00.342) 0:07:45.289 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:44 +0900 (0:00:00.144) 0:07:45.434 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:56:44 +0900 (0:00:00.215) 0:07:45.650 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:56:44 +0900 (0:00:00.089) 0:07:45.739 ********
ok: [kubeworker] => {
"msg": "docker.io/rancher/local-path-provisioner"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:56:44 +0900 (0:00:00.145) 0:07:45.885 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:44 +0900 (0:00:00.121) 0:07:46.006 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "docker.io/rancher/local-path-provisioner:v0.0.2"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:44 +0900 (0:00:00.132) 0:07:46.138 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "docker.io_rancher_local-path-provisioner_v0.0.2.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:44 +0900 (0:00:00.126) 0:07:46.264 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/docker.io_rancher_local-path-provisioner_v0.0.2.tar", "image_path_final": "/tmp/releases/images/docker.io_rancher_local-path-provisioner_v0.0.2.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:44 +0900 (0:00:00.133) 0:07:46.398 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:45 +0900 (0:00:00.077) 0:07:46.475 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:45 +0900 (0:00:00.088) 0:07:46.564 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:45 +0900 (0:00:00.141) 0:07:46.706 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:45 +0900 (0:00:00.085) 0:07:46.791 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:56:45 +0900 (0:00:00.223) 0:07:47.014 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:56:45 +0900 (0:00:00.395) 0:07:47.409 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:56:46 +0900 (0:00:00.122) 0:07:47.532 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:46 +0900 (0:00:00.080) 0:07:47.612 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: True"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:56:46 +0900 (0:00:00.168) 0:07:47.781 ********
changed: [kubeworker -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "docker.io/rancher/local-path-provisioner:v0.0.2"], "delta": "0:00:28.460776", "end": "2019-10-13 01:57:15.002018", "rc": 0, "start": "2019-10-13 01:56:46.541242", "stderr": "", "stderr_lines": [], "stdout": "v0.0.2: Pulling from rancher/local-path-provisioner\n3b37166ec614: Pulling fs layer\n504facff238f: Pulling fs layer\nebbcacd28e10: Pulling fs layer\nc7fb3351ecad: Pulling fs layer\n2e3debadcbf7: Pulling fs layer\nafd7421d0f9e: Pulling fs layer\nc93afabc9d78: Pulling fs layer\n2e3debadcbf7: Waiting\nafd7421d0f9e: Waiting\nc7fb3351ecad: Waiting\nc93afabc9d78: Waiting\n504facff238f: Verifying Checksum\n504facff238f: Download complete\nebbcacd28e10: Verifying Checksum\nebbcacd28e10: Download complete\nc7fb3351ecad: Verifying Checksum\nc7fb3351ecad: Download complete\n2e3debadcbf7: Verifying Checksum\n2e3debadcbf7: Download complete\nc93afabc9d78: Verifying Checksum\nc93afabc9d78: Download complete\n3b37166ec614: Verifying Checksum\n3b37166ec614: Download complete\n3b37166ec614: Pull complete\n504facff238f: Pull complete\nebbcacd28e10: Pull complete\nc7fb3351ecad: Pull complete\n2e3debadcbf7: Pull complete\nafd7421d0f9e: Verifying Checksum\nafd7421d0f9e: Download complete\nafd7421d0f9e: Pull complete\nc93afabc9d78: Pull complete\nDigest: sha256:78261ceb644c520ee24268b9124dd62ef5dfbd6479ea9db8a0154a0f484a5da4\nStatus: Downloaded newer image for rancher/local-path-provisioner:v0.0.2\ndocker.io/rancher/local-path-provisioner:v0.0.2", "stdout_lines": ["v0.0.2: Pulling from rancher/local-path-provisioner", "3b37166ec614: Pulling fs layer", "504facff238f: Pulling fs layer", "ebbcacd28e10: Pulling fs layer", "c7fb3351ecad: Pulling fs layer", "2e3debadcbf7: Pulling fs layer", "afd7421d0f9e: Pulling fs layer", "c93afabc9d78: Pulling fs layer", "2e3debadcbf7: Waiting", "afd7421d0f9e: Waiting", "c7fb3351ecad: Waiting", "c93afabc9d78: Waiting", "504facff238f: Verifying Checksum", "504facff238f: Download complete", "ebbcacd28e10: Verifying Checksum", "ebbcacd28e10: Download complete", "c7fb3351ecad: Verifying Checksum", "c7fb3351ecad: Download complete", "2e3debadcbf7: Verifying Checksum", "2e3debadcbf7: Download complete", "c93afabc9d78: Verifying Checksum", "c93afabc9d78: Download complete", "3b37166ec614: Verifying Checksum", "3b37166ec614: Download complete", "3b37166ec614: Pull complete", "504facff238f: Pull complete", "ebbcacd28e10: Pull complete", "c7fb3351ecad: Pull complete", "2e3debadcbf7: Pull complete", "afd7421d0f9e: Verifying Checksum", "afd7421d0f9e: Download complete", "afd7421d0f9e: Pull complete", "c93afabc9d78: Pull complete", "Digest: sha256:78261ceb644c520ee24268b9124dd62ef5dfbd6479ea9db8a0154a0f484a5da4", "Status: Downloaded newer image for rancher/local-path-provisioner:v0.0.2", "docker.io/rancher/local-path-provisioner:v0.0.2"]}
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:15 +0900 (0:00:28.733) 0:08:16.515 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:15 +0900 (0:00:00.134) 0:08:16.649 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:15 +0900 (0:00:00.079) 0:08:16.728 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:15 +0900 (0:00:00.078) 0:08:16.807 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/docker.io_rancher_local-path-provisioner_v0.0.2.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:15 +0900 (0:00:00.253) 0:08:17.061 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker, kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:15 +0900 (0:00:00.296) 0:08:17.357 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:16 +0900 (0:00:00.191) 0:08:17.549 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/kube-apiserver:v1.16.0"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/kube-apiserver:v1.16.0"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:16 +0900 (0:00:00.177) 0:08:17.727 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:16 +0900 (0:00:00.137) 0:08:17.864 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:16 +0900 (0:00:00.440) 0:08:18.305 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:17 +0900 (0:00:00.186) 0:08:18.492 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:57:17 +0900 (0:00:00.241) 0:08:18.733 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:17 +0900 (0:00:00.143) 0:08:18.877 ********
ok: [kubeworker] => {
"msg": "gcr.io/google-containers/kube-apiserver"
}
ok: [kubemaster] => {
"msg": "gcr.io/google-containers/kube-apiserver"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:17 +0900 (0:00:00.186) 0:08:19.064 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:17 +0900 (0:00:00.177) 0:08:19.241 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/kube-apiserver:v1.16.0"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/kube-apiserver:v1.16.0"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:17 +0900 (0:00:00.182) 0:08:19.424 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_kube-apiserver_v1.16.0.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_kube-apiserver_v1.16.0.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:18 +0900 (0:00:00.177) 0:08:19.602 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_kube-apiserver_v1.16.0.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_kube-apiserver_v1.16.0.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_kube-apiserver_v1.16.0.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_kube-apiserver_v1.16.0.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:18 +0900 (0:00:00.179) 0:08:19.781 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:18 +0900 (0:00:00.148) 0:08:19.930 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:18 +0900 (0:00:00.129) 0:08:20.059 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:18 +0900 (0:00:00.191) 0:08:20.251 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:19 +0900 (0:00:00.207) 0:08:20.459 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker, kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:19 +0900 (0:00:00.348) 0:08:20.807 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:57:19 +0900 (0:00:00.478) 0:08:21.286 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:57:20 +0900 (0:00:00.186) 0:08:21.473 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:20 +0900 (0:00:00.153) 0:08:21.626 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: False"
}
ok: [kubemaster] => {
"msg": "XXX Pull required is: False"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:20 +0900 (0:00:00.245) 0:08:21.872 ********
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:20 +0900 (0:00:00.219) 0:08:22.092 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:20 +0900 (0:00:00.171) 0:08:22.264 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:20 +0900 (0:00:00.099) 0:08:22.364 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:21 +0900 (0:00:00.143) 0:08:22.507 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_kube-apiserver_v1.16.0.tar", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_kube-apiserver_v1.16.0.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:21 +0900 (0:00:00.339) 0:08:22.847 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker, kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:21 +0900 (0:00:00.308) 0:08:23.155 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:21 +0900 (0:00:00.184) 0:08:23.340 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/kube-controller-manager:v1.16.0"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/kube-controller-manager:v1.16.0"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:22 +0900 (0:00:00.174) 0:08:23.514 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:22 +0900 (0:00:00.141) 0:08:23.656 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:22 +0900 (0:00:00.493) 0:08:24.149 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:22 +0900 (0:00:00.186) 0:08:24.335 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:57:23 +0900 (0:00:00.231) 0:08:24.567 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:23 +0900 (0:00:00.149) 0:08:24.716 ********
ok: [kubeworker] => {
"msg": "gcr.io/google-containers/kube-controller-manager"
}
ok: [kubemaster] => {
"msg": "gcr.io/google-containers/kube-controller-manager"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:23 +0900 (0:00:00.190) 0:08:24.907 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:23 +0900 (0:00:00.185) 0:08:25.092 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/kube-controller-manager:v1.16.0"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/kube-controller-manager:v1.16.0"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:23 +0900 (0:00:00.195) 0:08:25.287 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_kube-controller-manager_v1.16.0.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_kube-controller-manager_v1.16.0.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:24 +0900 (0:00:00.184) 0:08:25.471 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_kube-controller-manager_v1.16.0.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_kube-controller-manager_v1.16.0.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_kube-controller-manager_v1.16.0.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_kube-controller-manager_v1.16.0.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:24 +0900 (0:00:00.206) 0:08:25.678 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:24 +0900 (0:00:00.150) 0:08:25.829 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:24 +0900 (0:00:00.138) 0:08:25.967 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:24 +0900 (0:00:00.203) 0:08:26.171 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:24 +0900 (0:00:00.200) 0:08:26.372 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker, kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:25 +0900 (0:00:00.355) 0:08:26.727 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:57:25 +0900 (0:00:00.496) 0:08:27.223 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:57:25 +0900 (0:00:00.190) 0:08:27.414 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:26 +0900 (0:00:00.139) 0:08:27.553 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: False"
}
ok: [kubemaster] => {
"msg": "XXX Pull required is: False"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:26 +0900 (0:00:00.242) 0:08:27.796 ********
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:26 +0900 (0:00:00.202) 0:08:27.998 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:26 +0900 (0:00:00.143) 0:08:28.142 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:26 +0900 (0:00:00.088) 0:08:28.230 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:26 +0900 (0:00:00.170) 0:08:28.401 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_kube-controller-manager_v1.16.0.tar", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_kube-controller-manager_v1.16.0.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:27 +0900 (0:00:00.340) 0:08:28.741 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker, kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:27 +0900 (0:00:00.302) 0:08:29.043 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:27 +0900 (0:00:00.172) 0:08:29.216 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/kube-scheduler:v1.16.0"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/kube-scheduler:v1.16.0"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:27 +0900 (0:00:00.170) 0:08:29.386 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:28 +0900 (0:00:00.135) 0:08:29.521 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:28 +0900 (0:00:00.432) 0:08:29.954 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:28 +0900 (0:00:00.193) 0:08:30.147 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:57:28 +0900 (0:00:00.238) 0:08:30.386 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:29 +0900 (0:00:00.155) 0:08:30.541 ********
ok: [kubeworker] => {
"msg": "gcr.io/google-containers/kube-scheduler"
}
ok: [kubemaster] => {
"msg": "gcr.io/google-containers/kube-scheduler"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:29 +0900 (0:00:00.170) 0:08:30.712 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:29 +0900 (0:00:00.196) 0:08:30.908 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/kube-scheduler:v1.16.0"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/kube-scheduler:v1.16.0"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:29 +0900 (0:00:00.180) 0:08:31.089 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_kube-scheduler_v1.16.0.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_kube-scheduler_v1.16.0.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:29 +0900 (0:00:00.178) 0:08:31.268 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_kube-scheduler_v1.16.0.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_kube-scheduler_v1.16.0.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_kube-scheduler_v1.16.0.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_kube-scheduler_v1.16.0.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:30 +0900 (0:00:00.195) 0:08:31.463 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:30 +0900 (0:00:00.159) 0:08:31.623 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:30 +0900 (0:00:00.145) 0:08:31.768 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:30 +0900 (0:00:00.201) 0:08:31.969 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:30 +0900 (0:00:00.203) 0:08:32.173 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker, kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:31 +0900 (0:00:00.368) 0:08:32.541 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:57:31 +0900 (0:00:00.492) 0:08:33.033 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:57:31 +0900 (0:00:00.182) 0:08:33.216 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:31 +0900 (0:00:00.143) 0:08:33.360 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: False"
}
ok: [kubemaster] => {
"msg": "XXX Pull required is: False"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:32 +0900 (0:00:00.247) 0:08:33.607 ********
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:32 +0900 (0:00:00.206) 0:08:33.813 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:32 +0900 (0:00:00.132) 0:08:33.946 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:32 +0900 (0:00:00.087) 0:08:34.033 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:32 +0900 (0:00:00.139) 0:08:34.173 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_kube-scheduler_v1.16.0.tar", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_kube-scheduler_v1.16.0.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:33 +0900 (0:00:00.343) 0:08:34.516 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker, kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:33 +0900 (0:00:00.328) 0:08:34.845 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:33 +0900 (0:00:00.187) 0:08:35.032 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/kube-proxy:v1.16.0"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/kube-proxy:v1.16.0"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:33 +0900 (0:00:00.206) 0:08:35.238 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:33 +0900 (0:00:00.141) 0:08:35.379 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:34 +0900 (0:00:00.486) 0:08:35.866 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:34 +0900 (0:00:00.188) 0:08:36.054 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:57:34 +0900 (0:00:00.235) 0:08:36.290 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:35 +0900 (0:00:00.149) 0:08:36.440 ********
ok: [kubeworker] => {
"msg": "gcr.io/google-containers/kube-proxy"
}
ok: [kubemaster] => {
"msg": "gcr.io/google-containers/kube-proxy"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:35 +0900 (0:00:00.179) 0:08:36.620 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:35 +0900 (0:00:00.181) 0:08:36.801 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/kube-proxy:v1.16.0"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/kube-proxy:v1.16.0"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:35 +0900 (0:00:00.186) 0:08:36.987 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_kube-proxy_v1.16.0.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_kube-proxy_v1.16.0.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:35 +0900 (0:00:00.202) 0:08:37.190 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_kube-proxy_v1.16.0.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_kube-proxy_v1.16.0.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_kube-proxy_v1.16.0.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_kube-proxy_v1.16.0.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:35 +0900 (0:00:00.209) 0:08:37.400 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:36 +0900 (0:00:00.169) 0:08:37.569 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:36 +0900 (0:00:00.139) 0:08:37.709 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:36 +0900 (0:00:00.202) 0:08:37.912 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:36 +0900 (0:00:00.226) 0:08:38.138 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker, kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:37 +0900 (0:00:00.375) 0:08:38.514 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:57:37 +0900 (0:00:00.572) 0:08:39.086 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:57:37 +0900 (0:00:00.183) 0:08:39.269 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:37 +0900 (0:00:00.154) 0:08:39.424 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: False"
}
ok: [kubemaster] => {
"msg": "XXX Pull required is: False"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:38 +0900 (0:00:00.244) 0:08:39.669 ********
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:38 +0900 (0:00:00.206) 0:08:39.876 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:38 +0900 (0:00:00.138) 0:08:40.014 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:38 +0900 (0:00:00.104) 0:08:40.119 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:38 +0900 (0:00:00.137) 0:08:40.256 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_kube-proxy_v1.16.0.tar", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_kube-proxy_v1.16.0.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:39 +0900 (0:00:00.324) 0:08:40.580 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker, kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:39 +0900 (0:00:00.282) 0:08:40.863 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:39 +0900 (0:00:00.187) 0:08:41.051 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/pause:3.1"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/pause:3.1"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:39 +0900 (0:00:00.181) 0:08:41.232 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:39 +0900 (0:00:00.137) 0:08:41.370 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:40 +0900 (0:00:00.445) 0:08:41.815 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:40 +0900 (0:00:00.186) 0:08:42.002 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:57:40 +0900 (0:00:00.217) 0:08:42.220 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:40 +0900 (0:00:00.146) 0:08:42.366 ********
ok: [kubeworker] => {
"msg": "gcr.io/google-containers/pause"
}
ok: [kubemaster] => {
"msg": "gcr.io/google-containers/pause"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:41 +0900 (0:00:00.183) 0:08:42.549 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:41 +0900 (0:00:00.182) 0:08:42.732 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/pause:3.1"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/pause:3.1"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:41 +0900 (0:00:00.178) 0:08:42.911 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_pause_3.1.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_pause_3.1.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:41 +0900 (0:00:00.175) 0:08:43.086 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_pause_3.1.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_pause_3.1.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_pause_3.1.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_pause_3.1.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:41 +0900 (0:00:00.183) 0:08:43.269 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:41 +0900 (0:00:00.142) 0:08:43.412 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:42 +0900 (0:00:00.142) 0:08:43.555 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:42 +0900 (0:00:00.221) 0:08:43.777 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:42 +0900 (0:00:00.202) 0:08:43.979 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker, kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:42 +0900 (0:00:00.359) 0:08:44.338 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:57:43 +0900 (0:00:00.523) 0:08:44.861 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:57:43 +0900 (0:00:00.177) 0:08:45.039 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:43 +0900 (0:00:00.147) 0:08:45.186 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: False"
}
ok: [kubemaster] => {
"msg": "XXX Pull required is: False"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:43 +0900 (0:00:00.237) 0:08:45.423 ********
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:44 +0900 (0:00:00.215) 0:08:45.639 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:44 +0900 (0:00:00.138) 0:08:45.778 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:44 +0900 (0:00:00.083) 0:08:45.862 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:44 +0900 (0:00:00.127) 0:08:45.989 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_pause_3.1.tar", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_pause_3.1.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:44 +0900 (0:00:00.339) 0:08:46.329 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubeworker, kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:45 +0900 (0:00:00.323) 0:08:46.653 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:45 +0900 (0:00:00.197) 0:08:46.850 ********
ok: [kubeworker] => {"ansible_facts": {"pull_args": "docker.io/coredns/coredns:1.6.0"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_args": "docker.io/coredns/coredns:1.6.0"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:45 +0900 (0:00:00.202) 0:08:47.053 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:45 +0900 (0:00:00.133) 0:08:47.186 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:46 +0900 (0:00:00.433) 0:08:47.619 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:46 +0900 (0:00:00.193) 0:08:47.813 ********
ok: [kubeworker] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:57:46 +0900 (0:00:00.228) 0:08:48.042 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:46 +0900 (0:00:00.164) 0:08:48.206 ********
ok: [kubeworker] => {
"msg": "docker.io/coredns/coredns"
}
ok: [kubemaster] => {
"msg": "docker.io/coredns/coredns"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:46 +0900 (0:00:00.170) 0:08:48.376 ********
ok: [kubeworker] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:47 +0900 (0:00:00.223) 0:08:48.600 ********
ok: [kubeworker] => {"ansible_facts": {"image_reponame": "docker.io/coredns/coredns:1.6.0"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "docker.io/coredns/coredns:1.6.0"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:47 +0900 (0:00:00.259) 0:08:48.860 ********
ok: [kubeworker] => {"ansible_facts": {"image_filename": "docker.io_coredns_coredns_1.6.0.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_filename": "docker.io_coredns_coredns_1.6.0.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:47 +0900 (0:00:00.211) 0:08:49.071 ********
ok: [kubeworker] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/docker.io_coredns_coredns_1.6.0.tar", "image_path_final": "/tmp/releases/images/docker.io_coredns_coredns_1.6.0.tar"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/docker.io_coredns_coredns_1.6.0.tar", "image_path_final": "/tmp/releases/images/docker.io_coredns_coredns_1.6.0.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:47 +0900 (0:00:00.189) 0:08:49.261 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:47 +0900 (0:00:00.149) 0:08:49.410 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:48 +0900 (0:00:00.138) 0:08:49.548 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:48 +0900 (0:00:00.204) 0:08:49.754 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:48 +0900 (0:00:00.211) 0:08:49.965 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubeworker, kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:48 +0900 (0:00:00.346) 0:08:50.311 ********
ok: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:57:49 +0900 (0:00:00.528) 0:08:50.840 ********
ok: [kubeworker] => {"ansible_facts": {"pull_required": true}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:57:49 +0900 (0:00:00.188) 0:08:51.028 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:49 +0900 (0:00:00.137) 0:08:51.166 ********
ok: [kubeworker] => {
"msg": "XXX Pull required is: True"
}
ok: [kubemaster] => {
"msg": "XXX Pull required is: True"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:49 +0900 (0:00:00.234) 0:08:51.401 ********
changed: [kubeworker -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "docker.io/coredns/coredns:1.6.0"], "delta": "0:00:04.722856", "end": "2019-10-13 01:57:54.910666", "rc": 0, "start": "2019-10-13 01:57:50.187810", "stderr": "", "stderr_lines": [], "stdout": "1.6.0: Pulling from coredns/coredns\nDigest: sha256:263d03f2b889a75a0b91e035c2a14d45d7c1559c53444c5f7abf3a76014b779d\nStatus: Image is up to date for coredns/coredns:1.6.0\ndocker.io/coredns/coredns:1.6.0", "stdout_lines": ["1.6.0: Pulling from coredns/coredns", "Digest: sha256:263d03f2b889a75a0b91e035c2a14d45d7c1559c53444c5f7abf3a76014b779d", "Status: Image is up to date for coredns/coredns:1.6.0", "docker.io/coredns/coredns:1.6.0"]}
changed: [kubemaster -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "docker.io/coredns/coredns:1.6.0"], "delta": "0:00:04.835331", "end": "2019-10-13 01:57:55.135514", "rc": 0, "start": "2019-10-13 01:57:50.300183", "stderr": "", "stderr_lines": [], "stdout": "1.6.0: Pulling from coredns/coredns\nDigest: sha256:263d03f2b889a75a0b91e035c2a14d45d7c1559c53444c5f7abf3a76014b779d\nStatus: Image is up to date for coredns/coredns:1.6.0\ndocker.io/coredns/coredns:1.6.0", "stdout_lines": ["1.6.0: Pulling from coredns/coredns", "Digest: sha256:263d03f2b889a75a0b91e035c2a14d45d7c1559c53444c5f7abf3a76014b779d", "Status: Image is up to date for coredns/coredns:1.6.0", "docker.io/coredns/coredns:1.6.0"]}
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:55 +0900 (0:00:05.237) 0:08:56.639 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:55 +0900 (0:00:00.143) 0:08:56.782 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:55 +0900 (0:00:00.087) 0:08:56.869 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:55 +0900 (0:00:00.139) 0:08:57.009 ********
ok: [kubeworker] => {"changed": false, "path": "/tmp/releases/images/docker.io_coredns_coredns_1.6.0.tar", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/docker.io_coredns_coredns_1.6.0.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:55 +0900 (0:00:00.311) 0:08:57.321 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:56 +0900 (0:00:00.183) 0:08:57.504 ********
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:56 +0900 (0:00:00.125) 0:08:57.630 ********
ok: [kubemaster] => {"ansible_facts": {"pull_args": "quay.io/coreos/etcd:v3.3.10"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:56 +0900 (0:00:00.124) 0:08:57.755 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:56 +0900 (0:00:00.100) 0:08:57.855 ********
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:56 +0900 (0:00:00.402) 0:08:58.258 ********
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:56 +0900 (0:00:00.128) 0:08:58.386 ********
ok: [kubemaster] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:57:57 +0900 (0:00:00.249) 0:08:58.636 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:57 +0900 (0:00:00.095) 0:08:58.732 ********
ok: [kubemaster] => {
"msg": "quay.io/coreos/etcd"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:57:57 +0900 (0:00:00.112) 0:08:58.845 ********
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:57 +0900 (0:00:00.121) 0:08:58.966 ********
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "quay.io/coreos/etcd:v3.3.10"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:57 +0900 (0:00:00.124) 0:08:59.090 ********
ok: [kubemaster] => {"ansible_facts": {"image_filename": "quay.io_coreos_etcd_v3.3.10.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:57 +0900 (0:00:00.121) 0:08:59.212 ********
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/quay.io_coreos_etcd_v3.3.10.tar", "image_path_final": "/tmp/releases/images/quay.io_coreos_etcd_v3.3.10.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:57 +0900 (0:00:00.149) 0:08:59.361 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:58 +0900 (0:00:00.081) 0:08:59.443 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:58 +0900 (0:00:00.077) 0:08:59.520 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:58 +0900 (0:00:00.127) 0:08:59.647 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:58 +0900 (0:00:00.091) 0:08:59.739 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:58 +0900 (0:00:00.222) 0:08:59.962 ********
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:57:58 +0900 (0:00:00.377) 0:09:00.339 ********
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:57:59 +0900 (0:00:00.118) 0:09:00.458 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:59 +0900 (0:00:00.085) 0:09:00.544 ********
ok: [kubemaster] => {
"msg": "XXX Pull required is: False"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:59 +0900 (0:00:00.178) 0:09:00.722 ********
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:59 +0900 (0:00:00.084) 0:09:00.807 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:59 +0900 (0:00:00.132) 0:09:00.939 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:59 +0900 (0:00:00.085) 0:09:01.025 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:57:59 +0900 (0:00:00.073) 0:09:01.098 ********
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/quay.io_coreos_etcd_v3.3.10.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:57:59 +0900 (0:00:00.263) 0:09:01.362 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:00 +0900 (0:00:00.174) 0:09:01.537 ********
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:00 +0900 (0:00:00.136) 0:09:01.673 ********
ok: [kubemaster] => {"ansible_facts": {"pull_args": "docker.io/coredns/coredns:1.6.0"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:00 +0900 (0:00:00.158) 0:09:01.832 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:00 +0900 (0:00:00.087) 0:09:01.920 ********
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:00 +0900 (0:00:00.442) 0:09:02.362 ********
ok: [kubemaster] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:01 +0900 (0:00:00.135) 0:09:02.498 ********
ok: [kubemaster] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:58:01 +0900 (0:00:00.266) 0:09:02.765 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:58:01 +0900 (0:00:00.101) 0:09:02.866 ********
ok: [kubemaster] => {
"msg": "docker.io/coredns/coredns"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:58:01 +0900 (0:00:00.133) 0:09:02.999 ********
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:01 +0900 (0:00:00.156) 0:09:03.155 ********
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "docker.io/coredns/coredns:1.6.0"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:01 +0900 (0:00:00.118) 0:09:03.274 ********
ok: [kubemaster] => {"ansible_facts": {"image_filename": "docker.io_coredns_coredns_1.6.0.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:01 +0900 (0:00:00.135) 0:09:03.410 ********
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/docker.io_coredns_coredns_1.6.0.tar", "image_path_final": "/tmp/releases/images/docker.io_coredns_coredns_1.6.0.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:02 +0900 (0:00:00.148) 0:09:03.559 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:02 +0900 (0:00:00.078) 0:09:03.638 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:02 +0900 (0:00:00.078) 0:09:03.717 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:02 +0900 (0:00:00.133) 0:09:03.850 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:02 +0900 (0:00:00.096) 0:09:03.946 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:58:02 +0900 (0:00:00.224) 0:09:04.171 ********
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:58:03 +0900 (0:00:00.407) 0:09:04.578 ********
ok: [kubemaster] => {"ansible_facts": {"pull_required": true}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:58:03 +0900 (0:00:00.121) 0:09:04.699 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:03 +0900 (0:00:00.081) 0:09:04.781 ********
ok: [kubemaster] => {
"msg": "XXX Pull required is: True"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:03 +0900 (0:00:00.176) 0:09:04.957 ********
changed: [kubemaster -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/bin/docker", "pull", "docker.io/coredns/coredns:1.6.0"], "delta": "0:00:05.680901", "end": "2019-10-13 01:58:09.415495", "rc": 0, "start": "2019-10-13 01:58:03.734594", "stderr": "", "stderr_lines": [], "stdout": "1.6.0: Pulling from coredns/coredns\nDigest: sha256:263d03f2b889a75a0b91e035c2a14d45d7c1559c53444c5f7abf3a76014b779d\nStatus: Image is up to date for coredns/coredns:1.6.0\ndocker.io/coredns/coredns:1.6.0", "stdout_lines": ["1.6.0: Pulling from coredns/coredns", "Digest: sha256:263d03f2b889a75a0b91e035c2a14d45d7c1559c53444c5f7abf3a76014b779d", "Status: Image is up to date for coredns/coredns:1.6.0", "docker.io/coredns/coredns:1.6.0"]}
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:09 +0900 (0:00:05.963) 0:09:10.921 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:09 +0900 (0:00:00.146) 0:09:11.067 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:09 +0900 (0:00:00.095) 0:09:11.163 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:09 +0900 (0:00:00.085) 0:09:11.249 ********
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/docker.io_coredns_coredns_1.6.0.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:58:10 +0900 (0:00:00.274) 0:09:11.523 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:10 +0900 (0:00:00.174) 0:09:11.698 ********
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:10 +0900 (0:00:00.133) 0:09:11.831 ********
ok: [kubemaster] => {"ansible_facts": {"pull_args": "gcr.io/google-containers/cluster-proportional-autoscaler-amd64:1.6.0"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:10 +0900 (0:00:00.120) 0:09:11.952 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:10 +0900 (0:00:00.089) 0:09:12.042 ********
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:10 +0900 (0:00:00.361) 0:09:12.404 ********
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:11 +0900 (0:00:00.152) 0:09:12.556 ********
ok: [kubemaster] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:58:11 +0900 (0:00:00.243) 0:09:12.799 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:58:11 +0900 (0:00:00.091) 0:09:12.891 ********
ok: [kubemaster] => {
"msg": "gcr.io/google-containers/cluster-proportional-autoscaler-amd64"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:58:11 +0900 (0:00:00.124) 0:09:13.015 ********
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:11 +0900 (0:00:00.118) 0:09:13.134 ********
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "gcr.io/google-containers/cluster-proportional-autoscaler-amd64:1.6.0"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:11 +0900 (0:00:00.122) 0:09:13.256 ********
ok: [kubemaster] => {"ansible_facts": {"image_filename": "gcr.io_google-containers_cluster-proportional-autoscaler-amd64_1.6.0.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:11 +0900 (0:00:00.126) 0:09:13.382 ********
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google-containers_cluster-proportional-autoscaler-amd64_1.6.0.tar", "image_path_final": "/tmp/releases/images/gcr.io_google-containers_cluster-proportional-autoscaler-amd64_1.6.0.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:12 +0900 (0:00:00.119) 0:09:13.502 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:12 +0900 (0:00:00.074) 0:09:13.577 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:12 +0900 (0:00:00.081) 0:09:13.658 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:12 +0900 (0:00:00.130) 0:09:13.789 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:12 +0900 (0:00:00.080) 0:09:13.869 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:58:12 +0900 (0:00:00.241) 0:09:14.110 ********
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:58:13 +0900 (0:00:00.380) 0:09:14.491 ********
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:58:13 +0900 (0:00:00.131) 0:09:14.623 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:13 +0900 (0:00:00.097) 0:09:14.720 ********
ok: [kubemaster] => {
"msg": "XXX Pull required is: False"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:13 +0900 (0:00:00.173) 0:09:14.894 ********
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:13 +0900 (0:00:00.091) 0:09:14.985 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:13 +0900 (0:00:00.154) 0:09:15.139 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:13 +0900 (0:00:00.090) 0:09:15.230 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:13 +0900 (0:00:00.077) 0:09:15.308 ********
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google-containers_cluster-proportional-autoscaler-amd64_1.6.0.tar", "state": "absent"}
TASK [download : container_download | Make download decision if pull is required by tag or sha256] *****************************************************************************************************************************************
Sunday 13 October 2019 01:58:14 +0900 (0:00:00.285) 0:09:15.594 ********
included: /root/kubespray/kubespray/roles/download/tasks/set_docker_image_facts.yml for kubemaster
TASK [download : Set if containers should be pulled by digest] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:14 +0900 (0:00:00.186) 0:09:15.781 ********
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : Set pull_args] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:14 +0900 (0:00:00.122) 0:09:15.903 ********
ok: [kubemaster] => {"ansible_facts": {"pull_args": "gcr.io/google_containers/kubernetes-dashboard-amd64:v1.10.1"}, "changed": false}
TASK [download : Set image pull command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:14 +0900 (0:00:00.139) 0:09:16.042 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:14 +0900 (0:00:00.093) 0:09:16.136 ********
ok: [kubemaster] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : Set if pull is required per container] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:15 +0900 (0:00:00.395) 0:09:16.531 ********
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : Does any host require container pull?] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:15 +0900 (0:00:00.132) 0:09:16.664 ********
ok: [kubemaster] => {"ansible_facts": {"any_pull_required": true}, "changed": false}
TASK [download : Check the local digest sha256 corresponds to the given image tag] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:58:15 +0900 (0:00:00.235) 0:09:16.900 ********
TASK [download : set_container_facts | Display the name of the image being processed] ******************************************************************************************************************************************************
Sunday 13 October 2019 01:58:15 +0900 (0:00:00.108) 0:09:17.009 ********
ok: [kubemaster] => {
"msg": "gcr.io/google_containers/kubernetes-dashboard-amd64"
}
TASK [download : set_container_facts | Set if containers should be pulled by digest] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:58:15 +0900 (0:00:00.139) 0:09:17.148 ********
ok: [kubemaster] => {"ansible_facts": {"pull_by_digest": false}, "changed": false}
TASK [download : set_container_facts | Define by what name to pull the image] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:15 +0900 (0:00:00.146) 0:09:17.294 ********
ok: [kubemaster] => {"ansible_facts": {"image_reponame": "gcr.io/google_containers/kubernetes-dashboard-amd64:v1.10.1"}, "changed": false}
TASK [download : set_container_facts | Define file name of image] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:15 +0900 (0:00:00.130) 0:09:17.425 ********
ok: [kubemaster] => {"ansible_facts": {"image_filename": "gcr.io_google_containers_kubernetes-dashboard-amd64_v1.10.1.tar"}, "changed": false}
TASK [download : set_container_facts | Define path of image] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:16 +0900 (0:00:00.111) 0:09:17.537 ********
ok: [kubemaster] => {"ansible_facts": {"image_path_cached": "/tmp/kubespray_cache/images/gcr.io_google_containers_kubernetes-dashboard-amd64_v1.10.1.tar", "image_path_final": "/tmp/releases/images/gcr.io_google_containers_kubernetes-dashboard-amd64_v1.10.1.tar"}, "changed": false}
TASK [download : download_container | Determine if image is in cache] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:16 +0900 (0:00:00.122) 0:09:17.659 ********
TASK [download : download_container | Set fact indicating if image is in cache] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:16 +0900 (0:00:00.076) 0:09:17.736 ********
TASK [download : download_container | Upload image to node if it is cached] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:16 +0900 (0:00:00.074) 0:09:17.810 ********
TASK [download : download_container | Load image into docker] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:16 +0900 (0:00:00.131) 0:09:17.942 ********
TASK [download : download_container | Prepare container download] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:16 +0900 (0:00:00.080) 0:09:18.022 ********
included: /root/kubespray/kubespray/roles/download/tasks/check_pull_required.yml for kubemaster
TASK [download : check_pull_required | Generate a list of information about the images on a node] *****************************************************************************************************************************************
Sunday 13 October 2019 01:58:16 +0900 (0:00:00.236) 0:09:18.259 ********
ok: [kubemaster -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [download : check_pull_required | Set pull_required if the desired image is not yet loaded] *******************************************************************************************************************************************
Sunday 13 October 2019 01:58:17 +0900 (0:00:00.415) 0:09:18.674 ********
ok: [kubemaster] => {"ansible_facts": {"pull_required": false}, "changed": false}
TASK [download : check_pull_required | Check that the local digest sha256 corresponds to the given image tag] ******************************************************************************************************************************
Sunday 13 October 2019 01:58:17 +0900 (0:00:00.119) 0:09:18.794 ********
TASK [download : debug] ********************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:17 +0900 (0:00:00.081) 0:09:18.875 ********
ok: [kubemaster] => {
"msg": "XXX Pull required is: False"
}
TASK [download : download_container | Download image if required] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:17 +0900 (0:00:00.185) 0:09:19.061 ********
TASK [download : download_container | Check if image changed] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:17 +0900 (0:00:00.096) 0:09:19.158 ********
TASK [download : download_container | Save and compress image] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:17 +0900 (0:00:00.148) 0:09:19.306 ********
TASK [download : download_container | Copy image to ansible host cache] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:17 +0900 (0:00:00.083) 0:09:19.389 ********
TASK [download : download_container | Remove container image from cache] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:18 +0900 (0:00:00.077) 0:09:19.467 ********
ok: [kubemaster] => {"changed": false, "path": "/tmp/releases/images/gcr.io_google_containers_kubernetes-dashboard-amd64_v1.10.1.tar", "state": "absent"}
TASK [download : download | Sync files / images from ansible host to nodes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:18 +0900 (0:00:00.262) 0:09:19.729 ********
PLAY [etcd] ********************************************************************************************************************************************************************************************************************************
TASK [download : prep_download | Set a few facts] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:20 +0900 (0:00:02.378) 0:09:22.107 ********
TASK [download : Set image info command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:20 +0900 (0:00:00.080) 0:09:22.188 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:20 +0900 (0:00:00.073) 0:09:22.261 ********
TASK [download : prep_download | Create staging directory on remote node] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:20 +0900 (0:00:00.082) 0:09:22.344 ********
TASK [download : prep_download | Create local cache for files and images] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:20 +0900 (0:00:00.086) 0:09:22.430 ********
TASK [download : prep_download | On localhost, check if passwordless root is possible] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:58:21 +0900 (0:00:00.084) 0:09:22.515 ********
TASK [download : prep_download | On localhost, check if user has access to docker without using sudo] **************************************************************************************************************************************
Sunday 13 October 2019 01:58:21 +0900 (0:00:00.091) 0:09:22.607 ********
TASK [download : prep_download | Parse the outputs of the previous commands] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:21 +0900 (0:00:00.078) 0:09:22.685 ********
TASK [download : prep_download | Check that local user is in group or can become root] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:58:21 +0900 (0:00:00.080) 0:09:22.765 ********
TASK [download : include_tasks] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:21 +0900 (0:00:00.076) 0:09:22.842 ********
TASK [download : download | Get kubeadm binary and list of required images] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:21 +0900 (0:00:00.075) 0:09:22.917 ********
TASK [download : download | Download files / images] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:21 +0900 (0:00:00.087) 0:09:23.004 ********
TASK [download : download | Sync files / images from ansible host to nodes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:22 +0900 (0:00:01.399) 0:09:24.404 ********
TASK [kubespray-defaults : Configure defaults] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:24 +0900 (0:00:01.426) 0:09:25.830 ********
ok: [kubemaster] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
TASK [adduser : User | Create User Group] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:24 +0900 (0:00:00.083) 0:09:25.913 ********
changed: [kubemaster] => {"changed": true, "gid": 992, "name": "etcd", "state": "present", "system": true}
TASK [adduser : User | Create User] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:24 +0900 (0:00:00.270) 0:09:26.183 ********
changed: [kubemaster] => {"changed": true, "comment": "Etcd user", "create_home": true, "group": 992, "home": "/var/lib/etcd", "name": "etcd", "shell": "/bin/nologin", "state": "present", "stderr": "useradd: warning: the home directory already exists.\nNot copying any file from skel directory into it.\n", "stderr_lines": ["useradd: warning: the home directory already exists.", "Not copying any file from skel directory into it."], "system": true, "uid": 996}
TASK [adduser : User | Create User Group] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:25 +0900 (0:00:00.294) 0:09:26.478 ********
ok: [kubemaster] => {"changed": false, "gid": 994, "name": "kube-cert", "state": "present", "system": true}
TASK [adduser : User | Create User] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:25 +0900 (0:00:00.236) 0:09:26.714 ********
ok: [kubemaster] => {"append": false, "changed": false, "comment": "Kubernetes user", "group": 994, "home": "/home/kube", "move_home": false, "name": "kube", "shell": "/sbin/nologin", "state": "present", "uid": 997}
TASK [etcd : set architecture_groups] ******************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:25 +0900 (0:00:00.293) 0:09:27.008 ********
ok: [kubemaster] => {"ansible_facts": {"architecture_groups": {"aarch64": "arm64", "x86_64": "amd64"}}, "changed": false}
TASK [etcd : ansible_architecture_rename] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:25 +0900 (0:00:00.099) 0:09:27.108 ********
ok: [kubemaster] => {"ansible_facts": {"host_architecture": "amd64"}, "changed": false}
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:25 +0900 (0:00:00.102) 0:09:27.210 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/check_certs.yml for kubemaster
TASK [etcd : Check_certs | check if all certs have already been generated on first master] *************************************************************************************************************************************************
Sunday 13 October 2019 01:58:26 +0900 (0:00:00.233) 0:09:27.444 ********
ok: [kubemaster -> None] => {"changed": false, "examined": 0, "files": [], "matched": 0, "msg": "/etc/ssl/etcd/ssl was skipped as it does not seem to be a valid directory or it cannot be accessed\n"}
TASK [etcd : Check_certs | Set default value for 'sync_certs', 'gen_certs' and 'etcd_secret_changed' to false] *****************************************************************************************************************************
Sunday 13 October 2019 01:58:26 +0900 (0:00:00.577) 0:09:28.021 ********
ok: [kubemaster] => {"ansible_facts": {"etcd_secret_changed": false, "gen_certs": false, "sync_certs": false}, "changed": false}
TASK [etcd : Check certs | check if a cert already exists on node] *************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:26 +0900 (0:00:00.100) 0:09:28.121 ********
ok: [kubemaster] => (item=ca.pem) => {"ansible_loop_var": "item", "changed": false, "item": "ca.pem", "stat": {"exists": false}}
ok: [kubemaster] => (item=node-kubemaster-key.pem) => {"ansible_loop_var": "item", "changed": false, "item": "node-kubemaster-key.pem", "stat": {"exists": false}}
TASK [etcd : Check_certs | Set 'gen_certs' to true] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:27 +0900 (0:00:00.441) 0:09:28.563 ********
ok: [kubemaster] => (item=/etc/ssl/etcd/ssl/ca.pem) => {"ansible_facts": {"gen_certs": true}, "ansible_loop_var": "item", "changed": false, "item": "/etc/ssl/etcd/ssl/ca.pem"}
ok: [kubemaster] => (item=/etc/ssl/etcd/ssl/node-kubemaster-key.pem) => {"ansible_facts": {"gen_certs": true}, "ansible_loop_var": "item", "changed": false, "item": "/etc/ssl/etcd/ssl/node-kubemaster-key.pem"}
ok: [kubemaster] => (item=/etc/ssl/etcd/ssl/node-kubeworker-key.pem) => {"ansible_facts": {"gen_certs": true}, "ansible_loop_var": "item", "changed": false, "item": "/etc/ssl/etcd/ssl/node-kubeworker-key.pem"}
TASK [etcd : Check_certs | Set 'gen_node_certs' to true] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:27 +0900 (0:00:00.194) 0:09:28.758 ********
ok: [kubemaster] => {"ansible_facts": {"gen_node_certs": {"kubemaster": true, "kubeworker": true}}, "changed": false}
TASK [etcd : Check_certs | Set 'sync_certs' to true] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:27 +0900 (0:00:00.111) 0:09:28.869 ********
ok: [kubemaster] => {"ansible_facts": {"sync_certs": true}, "changed": false}
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:27 +0900 (0:00:00.112) 0:09:28.982 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/gen_certs_script.yml for kubemaster
TASK [etcd : Gen_certs | create etcd cert dir] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:27 +0900 (0:00:00.252) 0:09:29.234 ********
changed: [kubemaster] => {"changed": true, "gid": 0, "group": "root", "mode": "0700", "owner": "kube", "path": "/etc/ssl/etcd/ssl", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 6, "state": "directory", "uid": 997}
TASK [etcd : Gen_certs | create etcd script dir (on kubemaster)] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:28 +0900 (0:00:00.309) 0:09:29.544 ********
changed: [kubemaster -> None] => {"changed": true, "gid": 0, "group": "root", "mode": "0700", "owner": "root", "path": "/usr/local/bin/etcd-scripts", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 6, "state": "directory", "uid": 0}
TASK [etcd : Gen_certs | create etcd cert dir (on kubemaster)] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:28 +0900 (0:00:00.313) 0:09:29.857 ********
ok: [kubemaster -> None] => {"changed": false, "gid": 0, "group": "root", "mode": "0700", "owner": "kube", "path": "/etc/ssl/etcd/ssl", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 6, "state": "directory", "uid": 997}
TASK [etcd : Gen_certs | write openssl config] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:28 +0900 (0:00:00.338) 0:09:30.196 ********
changed: [kubemaster -> None] => {"changed": true, "checksum": "0c5fe39fdb1d8fb491d1e9e93eb7b6fa77351e95", "dest": "/etc/ssl/etcd/openssl.conf", "gid": 0, "group": "root", "md5sum": "cf80e53366111d07a35f0cb0f507e260", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:cert_t:s0", "size": 812, "src": "/root/.ansible/tmp/ansible-tmp-1570899508.8153648-65122759833590/source", "state": "file", "uid": 0}
TASK [etcd : Gen_certs | copy certs generation script] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:29 +0900 (0:00:00.883) 0:09:31.079 ********
changed: [kubemaster -> None] => {"changed": true, "checksum": "266083b165c0bcb22f1eba7e88d4a6e3720e1ce1", "dest": "/usr/local/bin/etcd-scripts/make-ssl-etcd.sh", "gid": 0, "group": "root", "md5sum": "1fc99690a2ed343216be5d01050c7129", "mode": "0700", "owner": "root", "secontext": "system_u:object_r:bin_t:s0", "size": 3411, "src": "/root/.ansible/tmp/ansible-tmp-1570899509.7189422-250290597824615/source", "state": "file", "uid": 0}
TASK [etcd : Gen_certs | run cert generation script] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:30 +0900 (0:00:00.576) 0:09:31.656 ********
changed: [kubemaster -> None] => {"changed": true, "cmd": ["bash", "-x", "/usr/local/bin/etcd-scripts/make-ssl-etcd.sh", "-f", "/etc/ssl/etcd/openssl.conf", "-d", "/etc/ssl/etcd/ssl"], "delta": "0:00:00.589577", "end": "2019-10-13 01:58:30.994745", "rc": 0, "start": "2019-10-13 01:58:30.405168", "stderr": "+ set -o errexit\n+ set -o pipefail\n+ (( 4 ))\n+ case \"$1\" in\n+ CONFIG=/etc/ssl/etcd/openssl.conf\n+ shift 2\n+ (( 2 ))\n+ case \"$1\" in\n+ SSLDIR=/etc/ssl/etcd/ssl\n+ shift 2\n+ (( 0 ))\n+ '[' -z /etc/ssl/etcd/openssl.conf ']'\n+ '[' -z /etc/ssl/etcd/ssl ']'\n++ mktemp -d /tmp/etcd_cacert.XXXXXX\n+ tmpdir=/tmp/etcd_cacert.vbBJ48\n+ trap 'rm -rf \"${tmpdir}\"' EXIT\n+ cd /tmp/etcd_cacert.vbBJ48\n+ mkdir -p /etc/ssl/etcd/ssl\n+ '[' -e /etc/ssl/etcd/ssl/ca-key.pem ']'\n+ openssl genrsa -out ca-key.pem 2048\n+ openssl req -x509 -new -nodes -key ca-key.pem -days 36500 -out ca.pem -subj /CN=etcd-ca\n+ '[' -n ' kubemaster ' ']'\n+ for host in '$MASTERS'\n+ cn=kubemaster\n+ openssl genrsa -out member-kubemaster-key.pem 2048\n+ openssl req -new -key member-kubemaster-key.pem -out member-kubemaster.csr -subj /CN=etcd-member-kubemaster -config /etc/ssl/etcd/openssl.conf\n+ openssl x509 -req -in member-kubemaster.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out member-kubemaster.pem -days 36500 -extensions ssl_client -extfile /etc/ssl/etcd/openssl.conf\n+ openssl genrsa -out admin-kubemaster-key.pem 2048\n+ openssl req -new -key admin-kubemaster-key.pem -out admin-kubemaster.csr -subj /CN=etcd-admin-kubemaster\n+ openssl x509 -req -in admin-kubemaster.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out admin-kubemaster.pem -days 36500 -extensions ssl_client -extfile /etc/ssl/etcd/openssl.conf\n+ '[' -n ' kubeworker kubemaster ' ']'\n+ for host in '$HOSTS'\n+ cn=kubeworker\n+ openssl genrsa -out node-kubeworker-key.pem 2048\n+ openssl req -new -key node-kubeworker-key.pem -out node-kubeworker.csr -subj /CN=etcd-node-kubeworker\n+ openssl x509 -req -in node-kubeworker.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out node-kubeworker.pem -days 36500 -extensions ssl_client -extfile /etc/ssl/etcd/openssl.conf\n+ for host in '$HOSTS'\n+ cn=kubemaster\n+ openssl genrsa -out node-kubemaster-key.pem 2048\n+ openssl req -new -key node-kubemaster-key.pem -out node-kubemaster.csr -subj /CN=etcd-node-kubemaster\n+ openssl x509 -req -in node-kubemaster.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out node-kubemaster.pem -days 36500 -extensions ssl_client -extfile /etc/ssl/etcd/openssl.conf\n+ '[' -e /etc/ssl/etcd/ssl/ca-key.pem ']'\n+ mv admin-kubemaster-key.pem admin-kubemaster.pem ca-key.pem ca.pem member-kubemaster-key.pem member-kubemaster.pem node-kubemaster-key.pem node-kubemaster.pem node-kubeworker-key.pem node-kubeworker.pem /etc/ssl/etcd/ssl/\n+ rm -rf /tmp/etcd_cacert.vbBJ48", "stderr_lines": ["+ set -o errexit", "+ set -o pipefail", "+ (( 4 ))", "+ case \"$1\" in", "+ CONFIG=/etc/ssl/etcd/openssl.conf", "+ shift 2", "+ (( 2 ))", "+ case \"$1\" in", "+ SSLDIR=/etc/ssl/etcd/ssl", "+ shift 2", "+ (( 0 ))", "+ '[' -z /etc/ssl/etcd/openssl.conf ']'", "+ '[' -z /etc/ssl/etcd/ssl ']'", "++ mktemp -d /tmp/etcd_cacert.XXXXXX", "+ tmpdir=/tmp/etcd_cacert.vbBJ48", "+ trap 'rm -rf \"${tmpdir}\"' EXIT", "+ cd /tmp/etcd_cacert.vbBJ48", "+ mkdir -p /etc/ssl/etcd/ssl", "+ '[' -e /etc/ssl/etcd/ssl/ca-key.pem ']'", "+ openssl genrsa -out ca-key.pem 2048", "+ openssl req -x509 -new -nodes -key ca-key.pem -days 36500 -out ca.pem -subj /CN=etcd-ca", "+ '[' -n ' kubemaster ' ']'", "+ for host in '$MASTERS'", "+ cn=kubemaster", "+ openssl genrsa -out member-kubemaster-key.pem 2048", "+ openssl req -new -key member-kubemaster-key.pem -out member-kubemaster.csr -subj /CN=etcd-member-kubemaster -config /etc/ssl/etcd/openssl.conf", "+ openssl x509 -req -in member-kubemaster.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out member-kubemaster.pem -days 36500 -extensions ssl_client -extfile /etc/ssl/etcd/openssl.conf", "+ openssl genrsa -out admin-kubemaster-key.pem 2048", "+ openssl req -new -key admin-kubemaster-key.pem -out admin-kubemaster.csr -subj /CN=etcd-admin-kubemaster", "+ openssl x509 -req -in admin-kubemaster.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out admin-kubemaster.pem -days 36500 -extensions ssl_client -extfile /etc/ssl/etcd/openssl.conf", "+ '[' -n ' kubeworker kubemaster ' ']'", "+ for host in '$HOSTS'", "+ cn=kubeworker", "+ openssl genrsa -out node-kubeworker-key.pem 2048", "+ openssl req -new -key node-kubeworker-key.pem -out node-kubeworker.csr -subj /CN=etcd-node-kubeworker", "+ openssl x509 -req -in node-kubeworker.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out node-kubeworker.pem -days 36500 -extensions ssl_client -extfile /etc/ssl/etcd/openssl.conf", "+ for host in '$HOSTS'", "+ cn=kubemaster", "+ openssl genrsa -out node-kubemaster-key.pem 2048", "+ openssl req -new -key node-kubemaster-key.pem -out node-kubemaster.csr -subj /CN=etcd-node-kubemaster", "+ openssl x509 -req -in node-kubemaster.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out node-kubemaster.pem -days 36500 -extensions ssl_client -extfile /etc/ssl/etcd/openssl.conf", "+ '[' -e /etc/ssl/etcd/ssl/ca-key.pem ']'", "+ mv admin-kubemaster-key.pem admin-kubemaster.pem ca-key.pem ca.pem member-kubemaster-key.pem member-kubemaster.pem node-kubemaster-key.pem node-kubemaster.pem node-kubeworker-key.pem node-kubeworker.pem /etc/ssl/etcd/ssl/", "+ rm -rf /tmp/etcd_cacert.vbBJ48"], "stdout": "", "stdout_lines": []}
TASK [etcd : Gen_certs | Gather etcd master certs] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:31 +0900 (0:00:01.015) 0:09:32.671 ********
TASK [etcd : Gen_certs | Write etcd master certs] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:31 +0900 (0:00:00.376) 0:09:33.048 ********
TASK [etcd : Gen_certs | Set cert names per node] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:31 +0900 (0:00:00.382) 0:09:33.431 ********
ok: [kubemaster] => {"ansible_facts": {"my_etcd_node_certs": ["ca.pem", "node-kubemaster.pem", "node-kubemaster-key.pem"]}, "changed": false}
TASK [etcd : Check_certs | Set 'sync_certs' to true on nodes] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:32 +0900 (0:00:00.090) 0:09:33.521 ********
TASK [etcd : Gen_certs | Gather node certs] ************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:32 +0900 (0:00:00.187) 0:09:33.709 ********
TASK [etcd : Gen_certs | Copy certs on nodes] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:32 +0900 (0:00:00.087) 0:09:33.796 ********
TASK [etcd : Gen_certs | check certificate permissions] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:32 +0900 (0:00:00.091) 0:09:33.887 ********
changed: [kubemaster] => {"changed": true, "gid": 0, "group": "root", "mode": "0640", "owner": "kube", "path": "/etc/ssl/etcd/ssl", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 276, "state": "directory", "uid": 997}
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:32 +0900 (0:00:00.263) 0:09:34.151 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/upd_ca_trust.yml for kubemaster
TASK [etcd : Gen_certs | target ca-certificate store file] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:32 +0900 (0:00:00.193) 0:09:34.345 ********
ok: [kubemaster] => {"ansible_facts": {"ca_cert_path": "/etc/pki/ca-trust/source/anchors/etcd-ca.crt"}, "changed": false}
TASK [etcd : Gen_certs | add CA to trusted CA dir] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:33 +0900 (0:00:00.108) 0:09:34.453 ********
changed: [kubemaster] => {"changed": true, "checksum": "bb6b6492177fb2c569cb7768f29c845699006d99", "dest": "/etc/pki/ca-trust/source/anchors/etcd-ca.crt", "gid": 0, "group": "root", "md5sum": "14306045384cbd3726f723740ede3048", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:cert_t:s0", "size": 1090, "src": "/etc/ssl/etcd/ssl/ca.pem", "state": "file", "uid": 0}
TASK [etcd : Gen_certs | update ca-certificates (Debian/Ubuntu/SUSE/Container Linux by CoreOS)] ********************************************************************************************************************************************
Sunday 13 October 2019 01:58:33 +0900 (0:00:00.311) 0:09:34.765 ********
TASK [etcd : Gen_certs | update ca-certificates (RedHat)] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:33 +0900 (0:00:00.091) 0:09:34.857 ********
changed: [kubemaster] => {"changed": true, "cmd": ["update-ca-trust", "extract"], "delta": "0:00:00.632417", "end": "2019-10-13 01:58:34.227851", "rc": 0, "start": "2019-10-13 01:58:33.595434", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
TASK [etcd : Gen_certs | update ca-certificates (ClearLinux)] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:34 +0900 (0:00:00.873) 0:09:35.730 ********
TASK [etcd : Gen_certs | Get etcd certificate serials] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:34 +0900 (0:00:00.085) 0:09:35.815 ********
ok: [kubemaster] => {"changed": false, "cmd": ["openssl", "x509", "-in", "/etc/ssl/etcd/ssl/node-kubemaster.pem", "-noout", "-serial"], "delta": "0:00:00.019445", "end": "2019-10-13 01:58:34.605715", "rc": 0, "start": "2019-10-13 01:58:34.586270", "stderr": "", "stderr_lines": [], "stdout": "serial=CBD5DC0AA51F6DFA", "stdout_lines": ["serial=CBD5DC0AA51F6DFA"]}
TASK [etcd : Set etcd_client_cert_serial] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:34 +0900 (0:00:00.309) 0:09:36.124 ********
ok: [kubemaster] => {"ansible_facts": {"etcd_client_cert_serial": "CBD5DC0AA51F6DFA"}, "changed": false}
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:34 +0900 (0:00:00.122) 0:09:36.247 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/install_docker.yml for kubemaster
TASK [etcd : Install | Copy etcdctl binary from docker container] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:35 +0900 (0:00:00.194) 0:09:36.442 ********
ok: [kubemaster] => {"attempts": 1, "changed": false, "cmd": ["sh", "-c", "/usr/bin/docker rm -f etcdctl-binarycopy; /usr/bin/docker create --name etcdctl-binarycopy quay.io/coreos/etcd:v3.3.10 && /usr/bin/docker cp etcdctl-binarycopy:/usr/local/bin/etcdctl /usr/local/bin/etcdctl && /usr/bin/docker rm -f etcdctl-binarycopy"], "delta": "0:00:00.318883", "end": "2019-10-13 01:58:35.503099", "rc": 0, "start": "2019-10-13 01:58:35.184216", "stderr": "Error: No such container: etcdctl-binarycopy", "stderr_lines": ["Error: No such container: etcdctl-binarycopy"], "stdout": "2f53f3f07b621a92c66c16e6b0357805aa09aa410e2ec1600ff89253e530730a\netcdctl-binarycopy", "stdout_lines": ["2f53f3f07b621a92c66c16e6b0357805aa09aa410e2ec1600ff89253e530730a", "etcdctl-binarycopy"]}
TASK [etcd : Install etcd launch script] ***************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:35 +0900 (0:00:00.584) 0:09:37.026 ********
changed: [kubemaster] => {"changed": true, "checksum": "1f1f360e30553a5279712cc60a5f423620c2bf02", "dest": "/usr/local/bin/etcd", "gid": 0, "group": "root", "md5sum": "99530b97b500b5cd98c26018d9bbd79a", "mode": "0750", "owner": "root", "secontext": "system_u:object_r:bin_t:s0", "size": 351, "src": "/root/.ansible/tmp/ansible-tmp-1570899515.6385882-182993662190973/source", "state": "file", "uid": 0}
TASK [etcd : Install etcd-events launch script] ********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:36 +0900 (0:00:00.618) 0:09:37.644 ********
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:36 +0900 (0:00:00.084) 0:09:37.729 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/configure.yml for kubemaster
TASK [etcd : Configure | Check if etcd cluster is healthy] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:36 +0900 (0:00:00.299) 0:09:38.028 ********
fatal: [kubemaster]: FAILED! => {"changed": false, "cmd": "/usr/local/bin/etcdctl --endpoints=https://192.168.56.107:2379 cluster-health | grep -q 'cluster is healthy'", "delta": "0:00:00.011891", "end": "2019-10-13 01:58:37.067527", "msg": "non-zero return code", "rc": 1, "start": "2019-10-13 01:58:37.055636", "stderr": "Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.56.107:2379: connect: connection refused\n\nerror #0: dial tcp 192.168.56.107:2379: connect: connection refused", "stderr_lines": ["Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.56.107:2379: connect: connection refused", "", "error #0: dial tcp 192.168.56.107:2379: connect: connection refused"], "stdout": "", "stdout_lines": []}
...ignoring
TASK [etcd : Configure | Check if etcd-events cluster is healthy] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:37 +0900 (0:00:00.546) 0:09:38.575 ********
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:37 +0900 (0:00:00.093) 0:09:38.668 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/refresh_config.yml for kubemaster
TASK [etcd : Refresh config | Create etcd config file] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:37 +0900 (0:00:00.167) 0:09:38.835 ********
changed: [kubemaster] => {"changed": true, "checksum": "132981dbe2824abce332a3f66ce7af5730371305", "dest": "/etc/etcd.env", "gid": 0, "group": "root", "md5sum": "8dbab0d86963611646a5754bac6b2650", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 1002, "src": "/root/.ansible/tmp/ansible-tmp-1570899517.4450831-4724783222670/source", "state": "file", "uid": 0}
TASK [etcd : Refresh config | Create etcd-events config file] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:38 +0900 (0:00:01.085) 0:09:39.921 ********
TASK [etcd : Configure | Copy etcd.service systemd file] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:38 +0900 (0:00:00.080) 0:09:40.002 ********
changed: [kubemaster] => {"changed": true, "checksum": "643a64e6855d3b34a04abbf561fd843945934936", "dest": "/etc/systemd/system/etcd.service", "gid": 0, "group": "root", "md5sum": "6c8c57186e4e724d74edfed4489ea5be", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:systemd_unit_file_t:s0", "size": 354, "src": "/root/.ansible/tmp/ansible-tmp-1570899518.6152716-247968012018985/source", "state": "file", "uid": 0}
TASK [etcd : Configure | Copy etcd-events.service systemd file] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:39 +0900 (0:00:00.558) 0:09:40.561 ********
TASK [etcd : Configure | reload systemd] ***************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:39 +0900 (0:00:00.094) 0:09:40.655 ********
ok: [kubemaster] => {"changed": false, "name": null, "status": {}}
TASK [etcd : Configure | Ensure etcd is running] *******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:39 +0900 (0:00:00.442) 0:09:41.097 ********
changed: [kubemaster] => {"changed": true, "enabled": true, "name": "etcd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "systemd-journald.socket docker.service system.slice basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "etcd docker wrapper", "DevicePolicy": "auto", "EnvironmentFile": "/etc/etcd.env (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/local/bin/etcd ; argv[]=/usr/local/bin/etcd ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPre": "{ path=/usr/bin/docker ; argv[]=/usr/bin/docker rm -f etcd1 ; ignore_errors=yes ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/bin/docker ; argv[]=/usr/bin/docker stop etcd1 ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/etc/systemd/system/etcd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "etcd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "7259", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7259", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "etcd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "yes", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "system.slice basic.target", "Restart": "always", "RestartUSec": "15s", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "disabled", "User": "root", "Wants": "docker.socket", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}}
TASK [etcd : Configure | Ensure etcd-events is running] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:40 +0900 (0:00:00.482) 0:09:41.579 ********
TASK [etcd : Configure | Check if etcd cluster is healthy] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:40 +0900 (0:00:00.140) 0:09:41.720 ********
FAILED - RETRYING: Configure | Check if etcd cluster is healthy (4 retries left).
ok: [kubemaster -> None] => {"attempts": 2, "changed": false, "cmd": "/usr/local/bin/etcdctl --no-sync --endpoints=https://192.168.56.107:2379 cluster-health | grep -q 'cluster is healthy'", "delta": "0:00:00.028710", "end": "2019-10-13 01:58:46.846539", "rc": 0, "start": "2019-10-13 01:58:46.817829", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
TASK [etcd : Configure | Check if etcd-events cluster is healthy] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:46 +0900 (0:00:06.684) 0:09:48.404 ********
TASK [etcd : Configure | Check if member is in etcd cluster] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:47 +0900 (0:00:00.081) 0:09:48.485 ********
ok: [kubemaster] => {"changed": false, "cmd": "/usr/local/bin/etcdctl --no-sync --endpoints=https://192.168.56.107:2379 member list | grep -q 192.168.56.107", "delta": "0:00:00.019718", "end": "2019-10-13 01:58:47.499833", "rc": 0, "start": "2019-10-13 01:58:47.480115", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
TASK [etcd : Configure | Check if member is in etcd-events cluster] ************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:47 +0900 (0:00:00.516) 0:09:49.002 ********
TASK [etcd : Configure | Join member(s) to etcd cluster one at a time] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:47 +0900 (0:00:00.085) 0:09:49.088 ********
TASK [etcd : Configure | Join member(s) to etcd-events cluster one at a time] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:47 +0900 (0:00:00.096) 0:09:49.185 ********
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:47 +0900 (0:00:00.113) 0:09:49.298 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/refresh_config.yml for kubemaster
TASK [etcd : Refresh config | Create etcd config file] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:48 +0900 (0:00:00.176) 0:09:49.475 ********
changed: [kubemaster] => {"changed": true, "checksum": "60fe53219bd34b6d8fd282becc9bd4b89aafc07e", "dest": "/etc/etcd.env", "gid": 0, "group": "root", "md5sum": "53084b148c2368b3db2f590db69e24ff", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 1007, "src": "/root/.ansible/tmp/ansible-tmp-1570899528.092452-233493757799207/source", "state": "file", "uid": 0}
TASK [etcd : Refresh config | Create etcd-events config file] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:49 +0900 (0:00:00.992) 0:09:50.467 ********
TASK [etcd : Restart etcd if certs changed] ************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:49 +0900 (0:00:00.091) 0:09:50.559 ********
TASK [etcd : Restart etcd-events if certs changed] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:49 +0900 (0:00:00.091) 0:09:50.650 ********
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:49 +0900 (0:00:00.095) 0:09:50.746 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/refresh_config.yml for kubemaster
TASK [etcd : Refresh config | Create etcd config file] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:49 +0900 (0:00:00.178) 0:09:50.925 ********
ok: [kubemaster] => {"changed": false, "checksum": "60fe53219bd34b6d8fd282becc9bd4b89aafc07e", "dest": "/etc/etcd.env", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/etcd.env", "secontext": "system_u:object_r:etc_t:s0", "size": 1007, "state": "file", "uid": 0}
TASK [etcd : Refresh config | Create etcd-events config file] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:50 +0900 (0:00:00.886) 0:09:51.812 ********
TASK [etcd : Install etcdctl binary from etcd role] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:50 +0900 (0:00:00.085) 0:09:51.897 ********
RUNNING HANDLER [etcd : restart etcd] ******************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:50 +0900 (0:00:00.026) 0:09:51.924 ********
changed: [kubemaster] => {"changed": true, "cmd": ["/bin/true"], "delta": "0:00:00.002470", "end": "2019-10-13 01:58:50.726849", "rc": 0, "start": "2019-10-13 01:58:50.724379", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
RUNNING HANDLER [etcd : Backup etcd data] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:50 +0900 (0:00:00.446) 0:09:52.371 ********
changed: [kubemaster] => {"changed": true, "cmd": ["/bin/true"], "delta": "0:00:00.001893", "end": "2019-10-13 01:58:51.151439", "rc": 0, "start": "2019-10-13 01:58:51.149546", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
RUNNING HANDLER [etcd : Refresh Time Fact] *************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:51 +0900 (0:00:00.489) 0:09:52.860 ********
ok: [kubemaster]
RUNNING HANDLER [etcd : Set Backup Directory] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:52 +0900 (0:00:00.581) 0:09:53.441 ********
ok: [kubemaster] => {"ansible_facts": {"etcd_backup_directory": "/var/backups/etcd-2019-10-13_01:58:51"}, "changed": false}
RUNNING HANDLER [etcd : Create Backup Directory] *******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:52 +0900 (0:00:00.096) 0:09:53.538 ********
changed: [kubemaster] => {"changed": true, "gid": 0, "group": "root", "mode": "0600", "owner": "root", "path": "/var/backups/etcd-2019-10-13_01:58:51", "secontext": "unconfined_u:object_r:var_t:s0", "size": 6, "state": "directory", "uid": 0}
RUNNING HANDLER [etcd : Stat etcd v2 data directory] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:52 +0900 (0:00:00.231) 0:09:53.770 ********
ok: [kubemaster] => {"changed": false, "stat": {"atime": 1570899520.442, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 0, "charset": "binary", "ctime": 1570899520.463, "dev": 64768, "device_type": 0, "executable": true, "exists": true, "gid": 0, "gr_name": "root", "inode": 5495478, "isblk": false, "ischr": false, "isdir": true, "isfifo": false, "isgid": false, "islnk": false, "isreg": false, "issock": false, "isuid": false, "mimetype": "inode/directory", "mode": "0700", "mtime": 1570899520.463, "nlink": 4, "path": "/var/lib/etcd/member", "pw_name": "root", "readable": true, "rgrp": false, "roth": false, "rusr": true, "size": 29, "uid": 0, "version": "18446744071960427374", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": true}}
RUNNING HANDLER [etcd : Backup etcd v2 data] ***********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:52 +0900 (0:00:00.247) 0:09:54.017 ********
changed: [kubemaster] => {"attempts": 1, "changed": true, "cmd": ["/usr/local/bin/etcdctl", "backup", "--data-dir", "/var/lib/etcd", "--backup-dir", "/var/backups/etcd-2019-10-13_01:58:51"], "delta": "0:00:00.032914", "end": "2019-10-13 01:58:52.836380", "rc": 0, "start": "2019-10-13 01:58:52.803466", "stderr": "2019-10-13 01:58:52.816513 I | ignoring EntryConfChange raft entry\n2019-10-13 01:58:52.816606 I | ignoring member attribute update on /0/members/1367faf593bb0be7/attributes", "stderr_lines": ["2019-10-13 01:58:52.816513 I | ignoring EntryConfChange raft entry", "2019-10-13 01:58:52.816606 I | ignoring member attribute update on /0/members/1367faf593bb0be7/attributes"], "stdout": "", "stdout_lines": []}
RUNNING HANDLER [etcd : Backup etcd v3 data] ***********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:52 +0900 (0:00:00.273) 0:09:54.291 ********
changed: [kubemaster] => {"attempts": 1, "changed": true, "cmd": ["/usr/local/bin/etcdctl", "--endpoints=https://192.168.56.107:2379", "snapshot", "save", "/var/backups/etcd-2019-10-13_01:58:51/snapshot.db"], "delta": "0:00:00.034771", "end": "2019-10-13 01:58:53.359039", "rc": 0, "start": "2019-10-13 01:58:53.324268", "stderr": "", "stderr_lines": [], "stdout": "Snapshot saved at /var/backups/etcd-2019-10-13_01:58:51/snapshot.db", "stdout_lines": ["Snapshot saved at /var/backups/etcd-2019-10-13_01:58:51/snapshot.db"]}
RUNNING HANDLER [etcd : etcd | reload systemd] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:53 +0900 (0:00:00.524) 0:09:54.815 ********
ok: [kubemaster] => {"changed": false, "name": null, "status": {}}
RUNNING HANDLER [etcd : reload etcd] *******************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:53 +0900 (0:00:00.398) 0:09:55.214 ********
changed: [kubemaster] => {"changed": true, "name": "etcd", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-10-13 01:58:40 KST", "ActiveEnterTimestampMonotonic": "21861001882", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target system.slice docker.service systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-10-13 01:58:39 KST", "AssertTimestampMonotonic": "21860940182", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-10-13 01:58:39 KST", "ConditionTimestampMonotonic": "21860940182", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/etcd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "etcd docker wrapper", "DevicePolicy": "auto", "EnvironmentFile": "/etc/etcd.env (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "30633", "ExecMainStartTimestamp": "Sun 2019-10-13 01:58:40 KST", "ExecMainStartTimestampMonotonic": "21861001492", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/local/bin/etcd ; argv[]=/usr/local/bin/etcd ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPre": "{ path=/usr/bin/docker ; argv[]=/usr/bin/docker rm -f etcd1 ; ignore_errors=yes ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/bin/docker ; argv[]=/usr/bin/docker stop etcd1 ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/etc/systemd/system/etcd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "etcd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2019-10-13 01:58:39 KST", "InactiveExitTimestampMonotonic": "21860946710", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "7259", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7259", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "30633", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "etcd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "yes", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target system.slice", "Restart": "always", "RestartUSec": "15s", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "root", "WantedBy": "multi-user.target", "Wants": "docker.socket", "WatchdogTimestamp": "Sun 2019-10-13 01:58:40 KST", "WatchdogTimestampMonotonic": "21861001866", "WatchdogUSec": "0"}}
RUNNING HANDLER [etcd : wait for etcd up] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:58:55 +0900 (0:00:01.592) 0:09:56.807 ********
ok: [kubemaster] => {"attempts": 1, "changed": false, "connection": "close", "content_length": "17", "content_type": "text/plain; charset=utf-8", "cookies": {}, "cookies_string": "", "date": "Sat, 12 Oct 2019 16:59:00 GMT", "elapsed": 4, "json": {"health": "true"}, "msg": "OK (17 bytes)", "redirected": false, "status": 200, "url": "https://192.168.56.107:2379/health"}
RUNNING HANDLER [etcd : set etcd_secret_changed] *******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:00 +0900 (0:00:05.283) 0:10:02.090 ********
ok: [kubemaster] => {"ansible_facts": {"etcd_secret_changed": true}, "changed": false}
PLAY [k8s-cluster] *************************************************************************************************************************************************************************************************************************
TASK [download : prep_download | Set a few facts] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:00 +0900 (0:00:00.184) 0:10:02.275 ********
TASK [download : Set image info command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:00 +0900 (0:00:00.135) 0:10:02.411 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:01 +0900 (0:00:00.154) 0:10:02.565 ********
TASK [download : prep_download | Create staging directory on remote node] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:01 +0900 (0:00:00.132) 0:10:02.697 ********
TASK [download : prep_download | Create local cache for files and images] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:01 +0900 (0:00:00.164) 0:10:02.862 ********
TASK [download : prep_download | On localhost, check if passwordless root is possible] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:59:01 +0900 (0:00:00.091) 0:10:02.954 ********
TASK [download : prep_download | On localhost, check if user has access to docker without using sudo] **************************************************************************************************************************************
Sunday 13 October 2019 01:59:01 +0900 (0:00:00.084) 0:10:03.039 ********
TASK [download : prep_download | Parse the outputs of the previous commands] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:01 +0900 (0:00:00.075) 0:10:03.114 ********
TASK [download : prep_download | Check that local user is in group or can become root] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:59:01 +0900 (0:00:00.145) 0:10:03.260 ********
TASK [download : include_tasks] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:01 +0900 (0:00:00.136) 0:10:03.396 ********
TASK [download : download | Get kubeadm binary and list of required images] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:02 +0900 (0:00:00.141) 0:10:03.537 ********
TASK [download : download | Download files / images] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:02 +0900 (0:00:00.128) 0:10:03.665 ********
TASK [download : download | Sync files / images from ansible host to nodes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:03 +0900 (0:00:01.627) 0:10:05.293 ********
TASK [kubespray-defaults : Configure defaults] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:05 +0900 (0:00:01.575) 0:10:06.869 ********
ok: [kubeworker] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
ok: [kubemaster] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
TASK [adduser : User | Create User Group] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:05 +0900 (0:00:00.141) 0:10:07.010 ********
changed: [kubeworker] => {"changed": true, "gid": 992, "name": "etcd", "state": "present", "system": true}
ok: [kubemaster] => {"changed": false, "gid": 992, "name": "etcd", "state": "present", "system": true}
TASK [adduser : User | Create User] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:05 +0900 (0:00:00.315) 0:10:07.326 ********
changed: [kubeworker] => {"changed": true, "comment": "Etcd user", "create_home": true, "group": 992, "home": "/var/lib/etcd", "name": "etcd", "shell": "/bin/nologin", "state": "present", "system": true, "uid": 996}
ok: [kubemaster] => {"append": false, "changed": false, "comment": "Etcd user", "group": 992, "home": "/var/lib/etcd", "move_home": false, "name": "etcd", "shell": "/bin/nologin", "state": "present", "uid": 996}
TASK [adduser : User | Create User Group] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:06 +0900 (0:00:00.362) 0:10:07.688 ********
ok: [kubeworker] => {"changed": false, "gid": 994, "name": "kube-cert", "state": "present", "system": true}
ok: [kubemaster] => {"changed": false, "gid": 994, "name": "kube-cert", "state": "present", "system": true}
TASK [adduser : User | Create User] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:06 +0900 (0:00:00.301) 0:10:07.990 ********
ok: [kubeworker] => {"append": false, "changed": false, "comment": "Kubernetes user", "group": 994, "home": "/home/kube", "move_home": false, "name": "kube", "shell": "/sbin/nologin", "state": "present", "uid": 997}
ok: [kubemaster] => {"append": false, "changed": false, "comment": "Kubernetes user", "group": 994, "home": "/home/kube", "move_home": false, "name": "kube", "shell": "/sbin/nologin", "state": "present", "uid": 997}
TASK [etcd : set architecture_groups] ******************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:06 +0900 (0:00:00.381) 0:10:08.371 ********
ok: [kubeworker] => {"ansible_facts": {"architecture_groups": {"aarch64": "arm64", "x86_64": "amd64"}}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"architecture_groups": {"aarch64": "arm64", "x86_64": "amd64"}}, "changed": false}
TASK [etcd : ansible_architecture_rename] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:07 +0900 (0:00:00.177) 0:10:08.549 ********
ok: [kubeworker] => {"ansible_facts": {"host_architecture": "amd64"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"host_architecture": "amd64"}, "changed": false}
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:07 +0900 (0:00:00.146) 0:10:08.696 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/check_certs.yml for kubeworker, kubemaster
TASK [etcd : Check_certs | check if all certs have already been generated on first master] *************************************************************************************************************************************************
Sunday 13 October 2019 01:59:07 +0900 (0:00:00.298) 0:10:08.994 ********
ok: [kubeworker -> None] => {"changed": false, "examined": 10, "files": [{"atime": 1570899513.25, "checksum": "bb6b6492177fb2c569cb7768f29c845699006d99", "ctime": 1570899512.647, "dev": 64768, "gid": 0, "gr_name": "root", "inode": 5495461, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0640", "mtime": 1570899510.494, "nlink": 1, "path": "/etc/ssl/etcd/ssl/ca.pem", "pw_name": "kube", "rgrp": true, "roth": false, "rusr": true, "size": 1090, "uid": 997, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}, {"atime": 1570899510.97, "checksum": "12f077696a6e6c20ac3754cc9f6069c472fb8d1a", "ctime": 1570899512.647, "dev": 64768, "gid": 0, "gr_name": "root", "inode": 5495473, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0640", "mtime": 1570899510.966, "nlink": 1, "path": "/etc/ssl/etcd/ssl/node-kubemaster-key.pem", "pw_name": "kube", "rgrp": true, "roth": false, "rusr": true, "size": 1675, "uid": 997, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}, {"atime": 1570899514.603, "checksum": "593da22fe01d121ab1685b543a9b0f9f49c24767", "ctime": 1570899512.647, "dev": 64768, "gid": 0, "gr_name": "root", "inode": 5495475, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0640", "mtime": 1570899510.987, "nlink": 1, "path": "/etc/ssl/etcd/ssl/node-kubemaster.pem", "pw_name": "kube", "rgrp": true, "roth": false, "rusr": true, "size": 1371, "uid": 997, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}, {"atime": 1570899510.915, "checksum": "4f78f3550ecfa8a5af742237624966d39d76a268", "ctime": 1570899512.649, "dev": 64768, "gid": 0, "gr_name": "root", "inode": 5495470, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0640", "mtime": 1570899510.911, "nlink": 1, "path": "/etc/ssl/etcd/ssl/node-kubeworker-key.pem", "pw_name": "kube", "rgrp": true, "roth": false, "rusr": true, "size": 1675, "uid": 997, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}, {"atime": 1570899510.928, "checksum": "cbe8025c618ed37275faf6783fc62e1cb9b30ca2", "ctime": 1570899512.649, "dev": 64768, "gid": 0, "gr_name": "root", "inode": 5495472, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0640", "mtime": 1570899510.929, "nlink": 1, "path": "/etc/ssl/etcd/ssl/node-kubeworker.pem", "pw_name": "kube", "rgrp": true, "roth": false, "rusr": true, "size": 1371, "uid": 997, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}], "matched": 5, "msg": ""}
TASK [etcd : Check_certs | Set default value for 'sync_certs', 'gen_certs' and 'etcd_secret_changed' to false] *****************************************************************************************************************************
Sunday 13 October 2019 01:59:07 +0900 (0:00:00.306) 0:10:09.301 ********
ok: [kubeworker] => {"ansible_facts": {"etcd_secret_changed": false, "gen_certs": false, "sync_certs": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"etcd_secret_changed": false, "gen_certs": false, "sync_certs": false}, "changed": false}
TASK [etcd : Check certs | check if a cert already exists on node] *************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:08 +0900 (0:00:00.187) 0:10:09.488 ********
ok: [kubeworker] => (item=ca.pem) => {"ansible_loop_var": "item", "changed": false, "item": "ca.pem", "stat": {"exists": false}}
ok: [kubemaster] => (item=ca.pem) => {"ansible_loop_var": "item", "changed": false, "item": "ca.pem", "stat": {"atime": 1570899513.25, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "bb6b6492177fb2c569cb7768f29c845699006d99", "ctime": 1570899512.647, "dev": 64768, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 5495461, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0640", "mtime": 1570899510.494, "nlink": 1, "path": "/etc/ssl/etcd/ssl/ca.pem", "pw_name": "kube", "readable": true, "rgrp": true, "roth": false, "rusr": true, "size": 1090, "uid": 997, "version": "18446744071564676591", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}}
ok: [kubeworker] => (item=node-kubeworker-key.pem) => {"ansible_loop_var": "item", "changed": false, "item": "node-kubeworker-key.pem", "stat": {"exists": false}}
ok: [kubemaster] => (item=node-kubemaster-key.pem) => {"ansible_loop_var": "item", "changed": false, "item": "node-kubemaster-key.pem", "stat": {"atime": 1570899547.755, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "12f077696a6e6c20ac3754cc9f6069c472fb8d1a", "ctime": 1570899512.647, "dev": 64768, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 5495473, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0640", "mtime": 1570899510.966, "nlink": 1, "path": "/etc/ssl/etcd/ssl/node-kubemaster-key.pem", "pw_name": "kube", "readable": true, "rgrp": true, "roth": false, "rusr": true, "size": 1675, "uid": 997, "version": "725876255", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}}
TASK [etcd : Check_certs | Set 'gen_certs' to true] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:08 +0900 (0:00:00.496) 0:10:09.985 ********
TASK [etcd : Check_certs | Set 'gen_node_certs' to true] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:08 +0900 (0:00:00.139) 0:10:10.124 ********
ok: [kubeworker] => {"ansible_facts": {"gen_node_certs": {"kubemaster": false, "kubeworker": false}}, "changed": false}
TASK [etcd : Check_certs | Set 'sync_certs' to true] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:08 +0900 (0:00:00.114) 0:10:10.238 ********
ok: [kubeworker] => {"ansible_facts": {"sync_certs": true}, "changed": false}
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:08 +0900 (0:00:00.160) 0:10:10.398 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/gen_certs_script.yml for kubeworker, kubemaster
TASK [etcd : Gen_certs | create etcd cert dir] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:09 +0900 (0:00:00.302) 0:10:10.701 ********
changed: [kubeworker] => {"changed": true, "gid": 0, "group": "root", "mode": "0700", "owner": "kube", "path": "/etc/ssl/etcd/ssl", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 6, "state": "directory", "uid": 997}
changed: [kubemaster] => {"changed": true, "gid": 0, "group": "root", "mode": "0700", "owner": "kube", "path": "/etc/ssl/etcd/ssl", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 276, "state": "directory", "uid": 997}
TASK [etcd : Gen_certs | create etcd script dir (on kubemaster)] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:09 +0900 (0:00:00.380) 0:10:11.081 ********
TASK [etcd : Gen_certs | create etcd cert dir (on kubemaster)] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:09 +0900 (0:00:00.141) 0:10:11.223 ********
TASK [etcd : Gen_certs | write openssl config] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:09 +0900 (0:00:00.130) 0:10:11.354 ********
TASK [etcd : Gen_certs | copy certs generation script] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:10 +0900 (0:00:00.148) 0:10:11.502 ********
TASK [etcd : Gen_certs | run cert generation script] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:10 +0900 (0:00:00.133) 0:10:11.635 ********
TASK [etcd : Gen_certs | Gather etcd master certs] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:10 +0900 (0:00:00.171) 0:10:11.807 ********
TASK [etcd : Gen_certs | Write etcd master certs] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:10 +0900 (0:00:00.489) 0:10:12.297 ********
TASK [etcd : Gen_certs | Set cert names per node] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:11 +0900 (0:00:00.373) 0:10:12.670 ********
ok: [kubeworker] => {"ansible_facts": {"my_etcd_node_certs": ["ca.pem", "node-kubeworker.pem", "node-kubeworker-key.pem"]}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"my_etcd_node_certs": ["ca.pem", "node-kubemaster.pem", "node-kubemaster-key.pem"]}, "changed": false}
TASK [etcd : Check_certs | Set 'sync_certs' to true on nodes] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:11 +0900 (0:00:00.146) 0:10:12.817 ********
ok: [kubeworker] => (item=ca.pem) => {"ansible_facts": {"sync_certs": true}, "ansible_loop_var": "item", "changed": false, "item": "ca.pem"}
ok: [kubeworker] => (item=node-kubeworker.pem) => {"ansible_facts": {"sync_certs": true}, "ansible_loop_var": "item", "changed": false, "item": "node-kubeworker.pem"}
ok: [kubeworker] => (item=node-kubeworker-key.pem) => {"ansible_facts": {"sync_certs": true}, "ansible_loop_var": "item", "changed": false, "item": "node-kubeworker-key.pem"}
TASK [etcd : Gen_certs | Gather node certs] ************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:11 +0900 (0:00:00.268) 0:10:13.086 ********
changed: [kubeworker -> None] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true}
TASK [etcd : Gen_certs | Copy certs on nodes] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:11 +0900 (0:00:00.267) 0:10:13.353 ********
ok: [kubeworker] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
TASK [etcd : Gen_certs | check certificate permissions] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:12 +0900 (0:00:00.264) 0:10:13.618 ********
changed: [kubeworker] => {"changed": true, "gid": 0, "group": "root", "mode": "0640", "owner": "kube", "path": "/etc/ssl/etcd/ssl", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 78, "state": "directory", "uid": 997}
changed: [kubemaster] => {"changed": true, "gid": 0, "group": "root", "mode": "0640", "owner": "kube", "path": "/etc/ssl/etcd/ssl", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 276, "state": "directory", "uid": 997}
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:12 +0900 (0:00:00.332) 0:10:13.950 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/upd_ca_trust.yml for kubeworker, kubemaster
TASK [etcd : Gen_certs | target ca-certificate store file] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:12 +0900 (0:00:00.295) 0:10:14.246 ********
ok: [kubeworker] => {"ansible_facts": {"ca_cert_path": "/etc/pki/ca-trust/source/anchors/etcd-ca.crt"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"ca_cert_path": "/etc/pki/ca-trust/source/anchors/etcd-ca.crt"}, "changed": false}
TASK [etcd : Gen_certs | add CA to trusted CA dir] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:12 +0900 (0:00:00.165) 0:10:14.411 ********
changed: [kubeworker] => {"changed": true, "checksum": "bb6b6492177fb2c569cb7768f29c845699006d99", "dest": "/etc/pki/ca-trust/source/anchors/etcd-ca.crt", "gid": 0, "group": "root", "md5sum": "14306045384cbd3726f723740ede3048", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:cert_t:s0", "size": 1090, "src": "/etc/ssl/etcd/ssl/ca.pem", "state": "file", "uid": 0}
ok: [kubemaster] => {"changed": false, "checksum": "bb6b6492177fb2c569cb7768f29c845699006d99", "dest": "/etc/pki/ca-trust/source/anchors/etcd-ca.crt", "gid": 0, "group": "root", "md5sum": "14306045384cbd3726f723740ede3048", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:cert_t:s0", "size": 1090, "src": "/etc/ssl/etcd/ssl/ca.pem", "state": "file", "uid": 0}
TASK [etcd : Gen_certs | update ca-certificates (Debian/Ubuntu/SUSE/Container Linux by CoreOS)] ********************************************************************************************************************************************
Sunday 13 October 2019 01:59:13 +0900 (0:00:00.331) 0:10:14.742 ********
TASK [etcd : Gen_certs | update ca-certificates (RedHat)] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:13 +0900 (0:00:00.133) 0:10:14.876 ********
changed: [kubeworker] => {"changed": true, "cmd": ["update-ca-trust", "extract"], "delta": "0:00:00.643722", "end": "2019-10-13 01:59:13.971391", "rc": 0, "start": "2019-10-13 01:59:13.327669", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
TASK [etcd : Gen_certs | update ca-certificates (ClearLinux)] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:14 +0900 (0:00:00.884) 0:10:15.761 ********
TASK [etcd : Gen_certs | Get etcd certificate serials] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:14 +0900 (0:00:00.145) 0:10:15.906 ********
ok: [kubeworker] => {"changed": false, "cmd": ["openssl", "x509", "-in", "/etc/ssl/etcd/ssl/node-kubeworker.pem", "-noout", "-serial"], "delta": "0:00:00.015465", "end": "2019-10-13 01:59:14.390095", "rc": 0, "start": "2019-10-13 01:59:14.374630", "stderr": "", "stderr_lines": [], "stdout": "serial=CBD5DC0AA51F6DF9", "stdout_lines": ["serial=CBD5DC0AA51F6DF9"]}
ok: [kubemaster] => {"changed": false, "cmd": ["openssl", "x509", "-in", "/etc/ssl/etcd/ssl/node-kubemaster.pem", "-noout", "-serial"], "delta": "0:00:00.013226", "end": "2019-10-13 01:59:14.752337", "rc": 0, "start": "2019-10-13 01:59:14.739111", "stderr": "", "stderr_lines": [], "stdout": "serial=CBD5DC0AA51F6DFA", "stdout_lines": ["serial=CBD5DC0AA51F6DFA"]}
TASK [etcd : Set etcd_client_cert_serial] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:14 +0900 (0:00:00.346) 0:10:16.253 ********
ok: [kubeworker] => {"ansible_facts": {"etcd_client_cert_serial": "CBD5DC0AA51F6DF9"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"etcd_client_cert_serial": "CBD5DC0AA51F6DFA"}, "changed": false}
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:14 +0900 (0:00:00.167) 0:10:16.421 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/install_docker.yml for kubemaster
TASK [etcd : Install | Copy etcdctl binary from docker container] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:15 +0900 (0:00:00.265) 0:10:16.687 ********
TASK [etcd : Install etcd launch script] ***************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:15 +0900 (0:00:00.089) 0:10:16.777 ********
TASK [etcd : Install etcd-events launch script] ********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:15 +0900 (0:00:00.077) 0:10:16.855 ********
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:15 +0900 (0:00:00.075) 0:10:16.930 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/configure.yml for kubemaster
TASK [etcd : Configure | Check if etcd cluster is healthy] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:15 +0900 (0:00:00.301) 0:10:17.232 ********
TASK [etcd : Configure | Check if etcd-events cluster is healthy] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:15 +0900 (0:00:00.080) 0:10:17.312 ********
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:15 +0900 (0:00:00.082) 0:10:17.395 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/refresh_config.yml for kubemaster
TASK [etcd : Refresh config | Create etcd config file] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:16 +0900 (0:00:00.157) 0:10:17.552 ********
TASK [etcd : Refresh config | Create etcd-events config file] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:16 +0900 (0:00:00.096) 0:10:17.649 ********
TASK [etcd : Configure | Copy etcd.service systemd file] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:16 +0900 (0:00:00.083) 0:10:17.732 ********
TASK [etcd : Configure | Copy etcd-events.service systemd file] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:16 +0900 (0:00:00.096) 0:10:17.829 ********
TASK [etcd : Configure | reload systemd] ***************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:16 +0900 (0:00:00.078) 0:10:17.908 ********
ok: [kubemaster] => {"changed": false, "name": null, "status": {}}
TASK [etcd : Configure | Ensure etcd is running] *******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:16 +0900 (0:00:00.380) 0:10:18.289 ********
TASK [etcd : Configure | Ensure etcd-events is running] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:16 +0900 (0:00:00.086) 0:10:18.375 ********
TASK [etcd : Configure | Check if etcd cluster is healthy] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:17 +0900 (0:00:00.133) 0:10:18.508 ********
TASK [etcd : Configure | Check if etcd-events cluster is healthy] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:17 +0900 (0:00:00.130) 0:10:18.639 ********
TASK [etcd : Configure | Check if member is in etcd cluster] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:17 +0900 (0:00:00.080) 0:10:18.719 ********
TASK [etcd : Configure | Check if member is in etcd-events cluster] ************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:17 +0900 (0:00:00.080) 0:10:18.800 ********
TASK [etcd : Configure | Join member(s) to etcd cluster one at a time] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:17 +0900 (0:00:00.100) 0:10:18.901 ********
TASK [etcd : Configure | Join member(s) to etcd-events cluster one at a time] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:17 +0900 (0:00:00.106) 0:10:19.007 ********
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:17 +0900 (0:00:00.100) 0:10:19.107 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/refresh_config.yml for kubemaster
TASK [etcd : Refresh config | Create etcd config file] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:17 +0900 (0:00:00.283) 0:10:19.390 ********
TASK [etcd : Refresh config | Create etcd-events config file] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:18 +0900 (0:00:00.089) 0:10:19.480 ********
TASK [etcd : Restart etcd if certs changed] ************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:18 +0900 (0:00:00.094) 0:10:19.575 ********
TASK [etcd : Restart etcd-events if certs changed] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:18 +0900 (0:00:00.136) 0:10:19.711 ********
TASK [etcd : include_tasks] ****************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:18 +0900 (0:00:00.139) 0:10:19.850 ********
included: /root/kubespray/kubespray/roles/etcd/tasks/refresh_config.yml for kubemaster
TASK [etcd : Refresh config | Create etcd config file] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:18 +0900 (0:00:00.276) 0:10:20.127 ********
TASK [etcd : Refresh config | Create etcd-events config file] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:18 +0900 (0:00:00.088) 0:10:20.216 ********
TASK [etcd : Install etcdctl binary from etcd role] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:18 +0900 (0:00:00.079) 0:10:20.295 ********
PLAY [k8s-cluster] *************************************************************************************************************************************************************************************************************************
TASK [download : prep_download | Set a few facts] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:19 +0900 (0:00:00.209) 0:10:20.505 ********
TASK [download : Set image info command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:19 +0900 (0:00:00.128) 0:10:20.634 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:19 +0900 (0:00:00.162) 0:10:20.796 ********
TASK [download : prep_download | Create staging directory on remote node] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:19 +0900 (0:00:00.135) 0:10:20.932 ********
TASK [download : prep_download | Create local cache for files and images] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:19 +0900 (0:00:00.153) 0:10:21.085 ********
TASK [download : prep_download | On localhost, check if passwordless root is possible] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:59:19 +0900 (0:00:00.093) 0:10:21.179 ********
TASK [download : prep_download | On localhost, check if user has access to docker without using sudo] **************************************************************************************************************************************
Sunday 13 October 2019 01:59:19 +0900 (0:00:00.080) 0:10:21.259 ********
TASK [download : prep_download | Parse the outputs of the previous commands] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:19 +0900 (0:00:00.081) 0:10:21.340 ********
TASK [download : prep_download | Check that local user is in group or can become root] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:59:20 +0900 (0:00:00.139) 0:10:21.479 ********
TASK [download : include_tasks] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:20 +0900 (0:00:00.139) 0:10:21.619 ********
TASK [download : download | Get kubeadm binary and list of required images] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:20 +0900 (0:00:00.129) 0:10:21.748 ********
TASK [download : download | Download files / images] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:20 +0900 (0:00:00.134) 0:10:21.883 ********
TASK [download : download | Sync files / images from ansible host to nodes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:22 +0900 (0:00:01.706) 0:10:23.589 ********
TASK [kubespray-defaults : Configure defaults] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:23 +0900 (0:00:01.636) 0:10:25.226 ********
ok: [kubeworker] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
ok: [kubemaster] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
TASK [kubernetes/node : look up docker cgroup driver] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:23 +0900 (0:00:00.153) 0:10:25.380 ********
ok: [kubeworker] => {"changed": false, "cmd": "docker info | grep 'Cgroup Driver' | awk -F': ' '{ print $2; }'", "delta": "0:00:00.077381", "end": "2019-10-13 01:59:23.925940", "rc": 0, "start": "2019-10-13 01:59:23.848559", "stderr": "", "stderr_lines": [], "stdout": "cgroupfs", "stdout_lines": ["cgroupfs"]}
ok: [kubemaster] => {"changed": false, "cmd": "docker info | grep 'Cgroup Driver' | awk -F': ' '{ print $2; }'", "delta": "0:00:00.129496", "end": "2019-10-13 01:59:24.317685", "rc": 0, "start": "2019-10-13 01:59:24.188189", "stderr": "WARNING: bridge-nf-call-iptables is disabled\nWARNING: bridge-nf-call-ip6tables is disabled", "stderr_lines": ["WARNING: bridge-nf-call-iptables is disabled", "WARNING: bridge-nf-call-ip6tables is disabled"], "stdout": "cgroupfs", "stdout_lines": ["cgroupfs"]}
TASK [kubernetes/node : set standalone_kubelet fact] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:24 +0900 (0:00:00.453) 0:10:25.833 ********
ok: [kubeworker] => {"ansible_facts": {"standalone_kubelet": false}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"standalone_kubelet": true}, "changed": false}
TASK [kubernetes/node : set kubelet_cgroup_driver_detected fact for containerd] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:24 +0900 (0:00:00.180) 0:10:26.013 ********
TASK [kubernetes/node : set kubelet_cgroup_driver_detected fact for other engines] *********************************************************************************************************************************************************
Sunday 13 October 2019 01:59:24 +0900 (0:00:00.137) 0:10:26.151 ********
ok: [kubeworker] => {"ansible_facts": {"kubelet_cgroup_driver_detected": "cgroupfs"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"kubelet_cgroup_driver_detected": "cgroupfs"}, "changed": false}
TASK [kubernetes/node : os specific vars] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:24 +0900 (0:00:00.255) 0:10:26.407 ********
TASK [kubernetes/node : Pre-upgrade | check if kubelet container exists] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:25 +0900 (0:00:00.234) 0:10:26.641 ********
ok: [kubeworker] => {"changed": false, "cmd": " docker ps -af name=kubelet | grep kubelet ", "delta": "0:00:00.040337", "end": "2019-10-13 01:59:25.149702", "failed_when_result": false, "msg": "non-zero return code", "rc": 1, "start": "2019-10-13 01:59:25.109365", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
ok: [kubemaster] => {"changed": false, "cmd": " docker ps -af name=kubelet | grep kubelet ", "delta": "0:00:00.038430", "end": "2019-10-13 01:59:25.507806", "failed_when_result": false, "msg": "non-zero return code", "rc": 1, "start": "2019-10-13 01:59:25.469376", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
TASK [kubernetes/node : Pre-upgrade | copy /var/lib/cni from kubelet] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:25 +0900 (0:00:00.369) 0:10:27.011 ********
TASK [kubernetes/node : Pre-upgrade | ensure kubelet container service is stopped if using host deployment] ********************************************************************************************************************************
Sunday 13 October 2019 01:59:25 +0900 (0:00:00.147) 0:10:27.159 ********
TASK [kubernetes/node : Pre-upgrade | ensure kubelet container is removed if using host deployment] ****************************************************************************************************************************************
Sunday 13 October 2019 01:59:25 +0900 (0:00:00.129) 0:10:27.288 ********
TASK [kubernetes/node : Ensure /var/lib/cni exists] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:25 +0900 (0:00:00.128) 0:10:27.417 ********
changed: [kubeworker] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/cni", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 6, "state": "directory", "uid": 0}
changed: [kubemaster] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/cni", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 6, "state": "directory", "uid": 0}
TASK [kubernetes/node : install | Copy kubeadm binary from download dir] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:26 +0900 (0:00:00.370) 0:10:27.787 ********
changed: [kubeworker -> None] => {"changed": true, "cmd": "/usr/bin/rsync --delay-updates -F --archive --no-owner --no-group --out-format=<<CHANGED>>%i %n%L /tmp/releases/kubeadm-v1.16.0-amd64 /usr/local/bin/kubeadm", "msg": ">f+++++++++ kubeadm-v1.16.0-amd64\n", "rc": 0, "stdout_lines": [">f+++++++++ kubeadm-v1.16.0-amd64"]}
TASK [kubernetes/node : install | Set kubeadm binary permissions] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:26 +0900 (0:00:00.508) 0:10:28.296 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/local/bin/kubeadm", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 44244800, "state": "file", "uid": 0}
TASK [kubernetes/node : install | Copy kubelet binary from download dir] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:27 +0900 (0:00:00.346) 0:10:28.642 ********
changed: [kubeworker -> None] => {"changed": true, "cmd": "/usr/bin/rsync --delay-updates -F --archive --no-owner --no-group --out-format=<<CHANGED>>%i %n%L /tmp/releases/hyperkube-v1.16.0-amd64 /usr/local/bin/kubelet", "msg": ">f+++++++++ hyperkube-v1.16.0-amd64\n", "rc": 0, "stdout_lines": [">f+++++++++ hyperkube-v1.16.0-amd64"]}
changed: [kubemaster -> None] => {"changed": true, "cmd": "/usr/bin/rsync --delay-updates -F --archive --no-owner --no-group --out-format=<<CHANGED>>%i %n%L /tmp/releases/hyperkube-v1.16.0-amd64 /usr/local/bin/kubelet", "msg": ">f+++++++++ hyperkube-v1.16.0-amd64\n", "rc": 0, "stdout_lines": [">f+++++++++ hyperkube-v1.16.0-amd64"]}
TASK [kubernetes/node : install | Set kubelet binary permissions] **************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:28 +0900 (0:00:01.389) 0:10:30.032 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/local/bin/kubelet", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 212345584, "state": "file", "uid": 0}
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/local/bin/kubelet", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 212345584, "state": "file", "uid": 0}
TASK [kubernetes/node : install | Copy hyperkube binary from download dir] *****************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:28 +0900 (0:00:00.364) 0:10:30.397 ********
changed: [kubeworker -> None] => {"changed": true, "cmd": "/usr/bin/rsync --delay-updates -F --archive --no-owner --no-group --out-format=<<CHANGED>>%i %n%L /tmp/releases/hyperkube-v1.16.0-amd64 /usr/local/bin/hyperkube", "msg": ">f+++++++++ hyperkube-v1.16.0-amd64\n", "rc": 0, "stdout_lines": [">f+++++++++ hyperkube-v1.16.0-amd64"]}
changed: [kubemaster -> None] => {"changed": true, "cmd": "/usr/bin/rsync --delay-updates -F --archive --no-owner --no-group --out-format=<<CHANGED>>%i %n%L /tmp/releases/hyperkube-v1.16.0-amd64 /usr/local/bin/hyperkube", "msg": ">f+++++++++ hyperkube-v1.16.0-amd64\n", "rc": 0, "stdout_lines": [">f+++++++++ hyperkube-v1.16.0-amd64"]}
TASK [kubernetes/node : install | Set hyperkube binary permissions] ************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:30 +0900 (0:00:01.275) 0:10:31.673 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/local/bin/hyperkube", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 212345584, "state": "file", "uid": 0}
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/local/bin/hyperkube", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 212345584, "state": "file", "uid": 0}
TASK [kubernetes/node : install | Copy socat wrapper for Container Linux] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:30 +0900 (0:00:00.312) 0:10:31.986 ********
TASK [kubernetes/node : haproxy | Cleanup potentially deployed haproxy] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:30 +0900 (0:00:00.149) 0:10:32.135 ********
ok: [kubeworker] => {"changed": false, "path": "/etc/kubernetes/manifests/haproxy.yml", "state": "absent"}
TASK [kubernetes/node : nginx-proxy | Make nginx directory] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:30 +0900 (0:00:00.279) 0:10:32.414 ********
changed: [kubeworker] => {"changed": true, "gid": 0, "group": "root", "mode": "0700", "owner": "root", "path": "/etc/nginx", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 6, "state": "directory", "uid": 0}
TASK [kubernetes/node : nginx-proxy | Write nginx-proxy configuration] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:31 +0900 (0:00:00.279) 0:10:32.694 ********
changed: [kubeworker] => {"changed": true, "checksum": "4f6e3a247d6f190ccf5036addb76371c3a0e673e", "dest": "/etc/nginx/nginx.conf", "gid": 0, "group": "root", "md5sum": "f5eef75d4bb9411b103de763c24f14bd", "mode": "0755", "owner": "root", "secontext": "system_u:object_r:httpd_config_t:s0", "size": 607, "src": "/root/.ansible/tmp/ansible-tmp-1570899571.3277953-23530468663635/source", "state": "file", "uid": 0}
TASK [kubernetes/node : nginx-proxy | Get checksum from config] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:32 +0900 (0:00:00.809) 0:10:33.503 ********
ok: [kubeworker] => {"changed": false, "stat": {"atime": 1570899571.688, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "4f6e3a247d6f190ccf5036addb76371c3a0e673e", "ctime": 1570899571.691, "dev": 64768, "device_type": 0, "executable": true, "exists": true, "gid": 0, "gr_name": "root", "inode": 357877, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0755", "mtime": 1570899571.538, "nlink": 1, "path": "/etc/nginx/nginx.conf", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 607, "uid": 0, "version": "1940343408", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": true, "xoth": true, "xusr": true}}
TASK [kubernetes/node : nginx-proxy | Write static pod] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:32 +0900 (0:00:00.266) 0:10:33.769 ********
changed: [kubeworker] => {"changed": true, "checksum": "319c3ba43c4693eb557d0fdb0707c0a906a9dade", "dest": "/etc/kubernetes/manifests/nginx-proxy.yml", "gid": 0, "group": "root", "md5sum": "0875ff2c20350a5156e9650726a68cbf", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 776, "src": "/root/.ansible/tmp/ansible-tmp-1570899572.4133518-28391448041728/source", "state": "file", "uid": 0}
TASK [kubernetes/node : haproxy | Cleanup potentially deployed nginx-proxy] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:32 +0900 (0:00:00.550) 0:10:34.320 ********
TASK [kubernetes/node : haproxy | Make haproxy directory] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:33 +0900 (0:00:00.139) 0:10:34.460 ********
TASK [kubernetes/node : haproxy | Write haproxy configuration] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:33 +0900 (0:00:00.132) 0:10:34.593 ********
TASK [kubernetes/node : haproxy | Get checksum from config] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:33 +0900 (0:00:00.130) 0:10:34.723 ********
TASK [kubernetes/node : haproxy | Write static pod] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:33 +0900 (0:00:00.151) 0:10:34.874 ********
TASK [kubernetes/node : Ensure nodePort range is reserved] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:33 +0900 (0:00:00.147) 0:10:35.022 ********
changed: [kubeworker] => {"changed": true}
changed: [kubemaster] => {"changed": true}
TASK [kubernetes/node : Verify if br_netfilter module exists] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:33 +0900 (0:00:00.339) 0:10:35.362 ********
ok: [kubeworker] => {"changed": false, "cmd": "modinfo br_netfilter", "delta": "0:00:00.006887", "end": "2019-10-13 01:59:33.834129", "failed_when_result": false, "rc": 0, "start": "2019-10-13 01:59:33.827242", "stderr": "", "stderr_lines": [], "stdout": "filename: /lib/modules/3.10.0-957.el7.x86_64/kernel/net/bridge/br_netfilter.ko.xz\ndescription: Linux ethernet netfilter firewall bridge\nauthor: Bart De Schuymer <bdschuym@pandora.be>\nauthor: Lennert Buytenhek <buytenh@gnu.org>\nlicense: GPL\nretpoline: Y\nrhelversion: 7.6\nsrcversion: C4DE536495D55C12BA6A8A8\ndepends: bridge\nintree: Y\nvermagic: 3.10.0-957.el7.x86_64 SMP mod_unload modversions \nsigner: CentOS Linux kernel signing key\nsig_key: B7:0D:CF:0D:F2:D9:B7:F2:91:59:24:82:49:FD:6F:E8:7B:78:14:27\nsig_hashalgo: sha256", "stdout_lines": ["filename: /lib/modules/3.10.0-957.el7.x86_64/kernel/net/bridge/br_netfilter.ko.xz", "description: Linux ethernet netfilter firewall bridge", "author: Bart De Schuymer <bdschuym@pandora.be>", "author: Lennert Buytenhek <buytenh@gnu.org>", "license: GPL", "retpoline: Y", "rhelversion: 7.6", "srcversion: C4DE536495D55C12BA6A8A8", "depends: bridge", "intree: Y", "vermagic: 3.10.0-957.el7.x86_64 SMP mod_unload modversions ", "signer: CentOS Linux kernel signing key", "sig_key: B7:0D:CF:0D:F2:D9:B7:F2:91:59:24:82:49:FD:6F:E8:7B:78:14:27", "sig_hashalgo: sha256"]}
ok: [kubemaster] => {"changed": false, "cmd": "modinfo br_netfilter", "delta": "0:00:00.008032", "end": "2019-10-13 01:59:34.172951", "failed_when_result": false, "rc": 0, "start": "2019-10-13 01:59:34.164919", "stderr": "", "stderr_lines": [], "stdout": "filename: /lib/modules/3.10.0-1062.1.1.el7.x86_64/kernel/net/bridge/br_netfilter.ko.xz\ndescription: Linux ethernet netfilter firewall bridge\nauthor: Bart De Schuymer <bdschuym@pandora.be>\nauthor: Lennert Buytenhek <buytenh@gnu.org>\nlicense: GPL\nretpoline: Y\nrhelversion: 7.7\nsrcversion: C4DE536495D55C12BA6A8A8\ndepends: bridge\nintree: Y\nvermagic: 3.10.0-1062.1.1.el7.x86_64 SMP mod_unload modversions \nsigner: CentOS Linux kernel signing key\nsig_key: 34:1A:1E:7B:06:D6:87:15:3E:3A:E9:8D:3E:B5:6E:0E:CD:30:DB:79\nsig_hashalgo: sha256", "stdout_lines": ["filename: /lib/modules/3.10.0-1062.1.1.el7.x86_64/kernel/net/bridge/br_netfilter.ko.xz", "description: Linux ethernet netfilter firewall bridge", "author: Bart De Schuymer <bdschuym@pandora.be>", "author: Lennert Buytenhek <buytenh@gnu.org>", "license: GPL", "retpoline: Y", "rhelversion: 7.7", "srcversion: C4DE536495D55C12BA6A8A8", "depends: bridge", "intree: Y", "vermagic: 3.10.0-1062.1.1.el7.x86_64 SMP mod_unload modversions ", "signer: CentOS Linux kernel signing key", "sig_key: 34:1A:1E:7B:06:D6:87:15:3E:3A:E9:8D:3E:B5:6E:0E:CD:30:DB:79", "sig_hashalgo: sha256"]}
TASK [kubernetes/node : Verify br_netfilter module path exists] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:34 +0900 (0:00:00.316) 0:10:35.678 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/modules-load.d", "secontext": "system_u:object_r:etc_t:s0", "size": 6, "state": "directory", "uid": 0}
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/modules-load.d", "secontext": "system_u:object_r:etc_t:s0", "size": 6, "state": "directory", "uid": 0}
TASK [kubernetes/node : Enable br_netfilter module] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:34 +0900 (0:00:00.348) 0:10:36.027 ********
ok: [kubemaster] => {"changed": false, "name": "br_netfilter", "params": "", "state": "present"}
ok: [kubeworker] => {"changed": false, "name": "br_netfilter", "params": "", "state": "present"}
TASK [kubernetes/node : Persist br_netfilter module] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:35 +0900 (0:00:00.570) 0:10:36.597 ********
changed: [kubeworker] => {"changed": true, "checksum": "c3394aff3e3a3e3920e68205b8e839b3d725648e", "dest": "/etc/modules-load.d/kubespray-br_netfilter.conf", "gid": 0, "group": "root", "md5sum": "6f034a1d190cd9ba928c0c4df2a4aad4", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 12, "src": "/root/.ansible/tmp/ansible-tmp-1570899575.220113-106567386802835/source", "state": "file", "uid": 0}
changed: [kubemaster] => {"changed": true, "checksum": "c3394aff3e3a3e3920e68205b8e839b3d725648e", "dest": "/etc/modules-load.d/kubespray-br_netfilter.conf", "gid": 0, "group": "root", "md5sum": "6f034a1d190cd9ba928c0c4df2a4aad4", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 12, "src": "/root/.ansible/tmp/ansible-tmp-1570899575.2792025-197089597155624/source", "state": "file", "uid": 0}
TASK [kubernetes/node : Check if bridge-nf-call-iptables key exists] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:35 +0900 (0:00:00.558) 0:10:37.155 ********
ok: [kubeworker] => {"changed": false, "cmd": ["sysctl", "net.bridge.bridge-nf-call-iptables"], "delta": "0:00:00.005098", "end": "2019-10-13 01:59:35.630988", "failed_when_result": false, "rc": 0, "start": "2019-10-13 01:59:35.625890", "stderr": "", "stderr_lines": [], "stdout": "net.bridge.bridge-nf-call-iptables = 1", "stdout_lines": ["net.bridge.bridge-nf-call-iptables = 1"]}
ok: [kubemaster] => {"changed": false, "cmd": ["sysctl", "net.bridge.bridge-nf-call-iptables"], "delta": "0:00:00.002951", "end": "2019-10-13 01:59:35.972346", "failed_when_result": false, "rc": 0, "start": "2019-10-13 01:59:35.969395", "stderr": "", "stderr_lines": [], "stdout": "net.bridge.bridge-nf-call-iptables = 0", "stdout_lines": ["net.bridge.bridge-nf-call-iptables = 0"]}
TASK [kubernetes/node : Enable bridge-nf-call tables] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:36 +0900 (0:00:00.329) 0:10:37.484 ********
changed: [kubeworker] => (item=net.bridge.bridge-nf-call-iptables) => {"ansible_loop_var": "item", "changed": true, "item": "net.bridge.bridge-nf-call-iptables"}
changed: [kubemaster] => (item=net.bridge.bridge-nf-call-iptables) => {"ansible_loop_var": "item", "changed": true, "item": "net.bridge.bridge-nf-call-iptables"}
changed: [kubeworker] => (item=net.bridge.bridge-nf-call-arptables) => {"ansible_loop_var": "item", "changed": true, "item": "net.bridge.bridge-nf-call-arptables"}
changed: [kubemaster] => (item=net.bridge.bridge-nf-call-arptables) => {"ansible_loop_var": "item", "changed": true, "item": "net.bridge.bridge-nf-call-arptables"}
changed: [kubeworker] => (item=net.bridge.bridge-nf-call-ip6tables) => {"ansible_loop_var": "item", "changed": true, "item": "net.bridge.bridge-nf-call-ip6tables"}
changed: [kubemaster] => (item=net.bridge.bridge-nf-call-ip6tables) => {"ansible_loop_var": "item", "changed": true, "item": "net.bridge.bridge-nf-call-ip6tables"}
TASK [kubernetes/node : Modprode Kernel Module for IPVS] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:36 +0900 (0:00:00.730) 0:10:38.215 ********
changed: [kubeworker] => (item=ip_vs) => {"ansible_loop_var": "item", "changed": true, "item": "ip_vs", "name": "ip_vs", "params": "", "state": "present"}
changed: [kubemaster] => (item=ip_vs) => {"ansible_loop_var": "item", "changed": true, "item": "ip_vs", "name": "ip_vs", "params": "", "state": "present"}
changed: [kubeworker] => (item=ip_vs_rr) => {"ansible_loop_var": "item", "changed": true, "item": "ip_vs_rr", "name": "ip_vs_rr", "params": "", "state": "present"}
changed: [kubemaster] => (item=ip_vs_rr) => {"ansible_loop_var": "item", "changed": true, "item": "ip_vs_rr", "name": "ip_vs_rr", "params": "", "state": "present"}
changed: [kubeworker] => (item=ip_vs_wrr) => {"ansible_loop_var": "item", "changed": true, "item": "ip_vs_wrr", "name": "ip_vs_wrr", "params": "", "state": "present"}
changed: [kubemaster] => (item=ip_vs_wrr) => {"ansible_loop_var": "item", "changed": true, "item": "ip_vs_wrr", "name": "ip_vs_wrr", "params": "", "state": "present"}
changed: [kubeworker] => (item=ip_vs_sh) => {"ansible_loop_var": "item", "changed": true, "item": "ip_vs_sh", "name": "ip_vs_sh", "params": "", "state": "present"}
changed: [kubemaster] => (item=ip_vs_sh) => {"ansible_loop_var": "item", "changed": true, "item": "ip_vs_sh", "name": "ip_vs_sh", "params": "", "state": "present"}
TASK [kubernetes/node : Modprobe nf_conntrack_ipv4 for kernels < 4.19] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:37 +0900 (0:00:00.939) 0:10:39.155 ********
ok: [kubeworker] => {"changed": false, "name": "nf_conntrack_ipv4", "params": "", "state": "present"}
ok: [kubemaster] => {"changed": false, "name": "nf_conntrack_ipv4", "params": "", "state": "present"}
TASK [kubernetes/node : Modprobe nf_conntrack for kernels >= 4.19] *************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:38 +0900 (0:00:00.302) 0:10:39.457 ********
TASK [kubernetes/node : Persist ip_vs modules] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:38 +0900 (0:00:00.136) 0:10:39.594 ********
changed: [kubeworker] => {"changed": true, "checksum": "fb666e350805b04493805c3b11187436eb0a0e31", "dest": "/etc/modules-load.d/kube_proxy-ipvs.conf", "gid": 0, "group": "root", "md5sum": "d9cc4491b5121715fca6283f5790775c", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 52, "src": "/root/.ansible/tmp/ansible-tmp-1570899578.2161918-208577840563651/source", "state": "file", "uid": 0}
changed: [kubemaster] => {"changed": true, "checksum": "fb666e350805b04493805c3b11187436eb0a0e31", "dest": "/etc/modules-load.d/kube_proxy-ipvs.conf", "gid": 0, "group": "root", "md5sum": "d9cc4491b5121715fca6283f5790775c", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 52, "src": "/root/.ansible/tmp/ansible-tmp-1570899578.2732987-262312986861976/source", "state": "file", "uid": 0}
TASK [kubernetes/node : Purge proxy manifest for kubeadm or if proxy services being provided by other means, e.g. network_plugin] **********************************************************************************************************
Sunday 13 October 2019 01:59:38 +0900 (0:00:00.560) 0:10:40.154 ********
TASK [kubernetes/node : Cleanup kube-proxy leftovers from node] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:38 +0900 (0:00:00.122) 0:10:40.277 ********
TASK [kubernetes/node : include_tasks] *****************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:38 +0900 (0:00:00.126) 0:10:40.403 ********
TASK [kubernetes/node : Test if openstack_cacert is a base64 string] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:39 +0900 (0:00:00.128) 0:10:40.532 ********
TASK [kubernetes/node : Write cacert file] *************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:39 +0900 (0:00:00.134) 0:10:40.667 ********
TASK [kubernetes/node : Write cloud-config] ************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:39 +0900 (0:00:00.129) 0:10:40.796 ********
TASK [kubernetes/node : Make sure dynamic kubelet configuration directory is writeable] ****************************************************************************************************************************************************
Sunday 13 October 2019 01:59:39 +0900 (0:00:00.146) 0:10:40.942 ********
TASK [kubernetes/node : gets the kubeadm version] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:39 +0900 (0:00:00.131) 0:10:41.073 ********
changed: [kubeworker] => {"changed": true, "cmd": ["/usr/local/bin/kubeadm", "version", "-o", "short"], "delta": "0:00:00.062174", "end": "2019-10-13 01:59:39.603584", "rc": 0, "start": "2019-10-13 01:59:39.541410", "stderr": "", "stderr_lines": [], "stdout": "v1.16.0", "stdout_lines": ["v1.16.0"]}
changed: [kubemaster] => {"changed": true, "cmd": ["/usr/local/bin/kubeadm", "version", "-o", "short"], "delta": "0:00:00.076891", "end": "2019-10-13 01:59:40.002505", "rc": 0, "start": "2019-10-13 01:59:39.925614", "stderr": "", "stderr_lines": [], "stdout": "v1.16.0", "stdout_lines": ["v1.16.0"]}
TASK [kubernetes/node : sets kubeadm api version to v1beta1] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:40 +0900 (0:00:00.427) 0:10:41.501 ********
ok: [kubeworker] => {"ansible_facts": {"kubeadmConfig_api_version": "v1beta1"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"kubeadmConfig_api_version": "v1beta1"}, "changed": false}
TASK [kubernetes/node : Write kubelet environment config file (kubeadm)] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:40 +0900 (0:00:00.165) 0:10:41.666 ********
changed: [kubeworker] => {"changed": true, "checksum": "b7e0df3b2b39e034722fca1c062816afdd01f2b6", "dest": "/etc/kubernetes/kubelet.env", "gid": 0, "group": "root", "md5sum": "1e3ed38112db1700d191adeed7bb8c05", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 670, "src": "/root/.ansible/tmp/ansible-tmp-1570899580.289667-188844347739026/source", "state": "file", "uid": 0}
changed: [kubemaster] => {"changed": true, "checksum": "0d5cbc7c2184f3a9e686aee4fedb15898846f038", "dest": "/etc/kubernetes/kubelet.env", "gid": 0, "group": "root", "md5sum": "faae80cfdb85b298b1a0c70a6831cae5", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 670, "src": "/root/.ansible/tmp/ansible-tmp-1570899580.3606606-75741046089957/source", "state": "file", "uid": 0}
TASK [kubernetes/node : Write kubelet config file] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:41 +0900 (0:00:00.847) 0:10:42.514 ********
changed: [kubeworker] => {"changed": true, "checksum": "088fb68cbba64010a0235f7ee29fcdaaa19d8c7c", "dest": "/etc/kubernetes/kubelet-config.yaml", "gid": 0, "group": "root", "md5sum": "63238d9db6e5ce89b91b315cfdbb65f2", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 564, "src": "/root/.ansible/tmp/ansible-tmp-1570899581.1553838-58505212647099/source", "state": "file", "uid": 0}
changed: [kubemaster] => {"changed": true, "checksum": "f7f01f9d7d23d904fad9b2aec7310a1a8690660a", "dest": "/etc/kubernetes/kubelet-config.yaml", "gid": 0, "group": "root", "md5sum": "6e52be6a5f0ead643737ad10c9b3743f", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 564, "src": "/root/.ansible/tmp/ansible-tmp-1570899581.2060573-161346699080329/source", "state": "file", "uid": 0}
TASK [kubernetes/node : Write kubelet systemd init file] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:41 +0900 (0:00:00.635) 0:10:43.149 ********
changed: [kubeworker] => {"changed": true, "checksum": "761e8e18b8328b49b5bbdaa6b69a2a48add4a96c", "dest": "/etc/systemd/system/kubelet.service", "gid": 0, "group": "root", "md5sum": "db0895130fa041b6d7f49e95d5cd7125", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:systemd_unit_file_t:s0", "size": 563, "src": "/root/.ansible/tmp/ansible-tmp-1570899581.776359-154599456891720/source", "state": "file", "uid": 0}
changed: [kubemaster] => {"changed": true, "checksum": "761e8e18b8328b49b5bbdaa6b69a2a48add4a96c", "dest": "/etc/systemd/system/kubelet.service", "gid": 0, "group": "root", "md5sum": "db0895130fa041b6d7f49e95d5cd7125", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:systemd_unit_file_t:s0", "size": 563, "src": "/root/.ansible/tmp/ansible-tmp-1570899581.8449552-269431874338328/source", "state": "file", "uid": 0}
RUNNING HANDLER [kubernetes/node : restart kubelet] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:42 +0900 (0:00:00.546) 0:10:43.696 ********
changed: [kubeworker] => {"changed": true, "cmd": ["/bin/true"], "delta": "0:00:00.002257", "end": "2019-10-13 01:59:42.209705", "rc": 0, "start": "2019-10-13 01:59:42.207448", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
changed: [kubemaster] => {"changed": true, "cmd": ["/bin/true"], "delta": "0:00:00.003041", "end": "2019-10-13 01:59:42.550910", "rc": 0, "start": "2019-10-13 01:59:42.547869", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
RUNNING HANDLER [kubernetes/node : Kubelet | reload systemd] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:42 +0900 (0:00:00.319) 0:10:44.015 ********
ok: [kubeworker] => {"changed": false, "name": null, "status": {}}
ok: [kubemaster] => {"changed": false, "name": null, "status": {}}
RUNNING HANDLER [kubernetes/node : Kubelet | reload kubelet] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:43 +0900 (0:00:00.484) 0:10:44.499 ********
changed: [kubeworker] => {"changed": true, "name": "kubelet", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2019-10-12 19:55:49 KST", "ActiveEnterTimestampMonotonic": "86985374", "ActiveExitTimestamp": "Sat 2019-10-12 19:55:49 KST", "ActiveExitTimestampMonotonic": "87080195", "ActiveState": "inactive", "After": "docker.service basic.target system.slice systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sat 2019-10-12 19:55:49 KST", "AssertTimestampMonotonic": "86984479", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2019-10-12 19:55:49 KST", "ConditionTimestampMonotonic": "86984479", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Kubernetes Kubelet Server", "DevicePolicy": "auto", "Documentation": "https://github.com/GoogleCloudPlatform/kubernetes", "DropInPaths": "/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf", "Environment": "KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf\\x20--kubeconfig=/etc/kubernetes/kubelet.conf KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml", "EnvironmentFile": "/etc/sysconfig/kubelet (ignore_errors=yes)", "ExecMainCode": "1", "ExecMainExitTimestamp": "Sat 2019-10-12 19:55:49 KST", "ExecMainExitTimestampMonotonic": "87079950", "ExecMainPID": "13612", "ExecMainStartTimestamp": "Sat 2019-10-12 19:55:49 KST", "ExecMainStartTimestampMonotonic": "86985322", "ExecMainStatus": "255", "ExecStart": "{ path=/usr/bin/kubelet ; argv[]=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/etc/systemd/system/kubelet.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "kubelet.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sat 2019-10-12 19:55:52 KST", "InactiveEnterTimestampMonotonic": "90890824", "InactiveExitTimestamp": "Sat 2019-10-12 19:55:49 KST", "InactiveExitTimestampMonotonic": "87080319", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "7257", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7257", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "kubelet.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "always", "RestartUSec": "10s", "Result": "exit-code", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "root", "WantedBy": "multi-user.target", "Wants": "docker.socket system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}}
changed: [kubemaster] => {"changed": true, "name": "kubelet", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-10-13 01:59:37 KST", "ActiveEnterTimestampMonotonic": "21918018105", "ActiveExitTimestamp": "Sun 2019-10-13 01:59:37 KST", "ActiveExitTimestampMonotonic": "21918122251", "ActiveState": "activating", "After": "docker.service system.slice basic.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-10-13 01:59:37 KST", "AssertTimestampMonotonic": "21918012075", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-10-13 01:59:37 KST", "ConditionTimestampMonotonic": "21918012075", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Kubernetes Kubelet Server", "DevicePolicy": "auto", "Documentation": "https://github.com/GoogleCloudPlatform/kubernetes", "DropInPaths": "/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf", "Environment": "KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml", "EnvironmentFile": "/etc/sysconfig/kubelet (ignore_errors=yes)", "ExecMainCode": "1", "ExecMainExitTimestamp": "Sun 2019-10-13 01:59:37 KST", "ExecMainExitTimestampMonotonic": "21918121973", "ExecMainPID": "32065", "ExecMainStartTimestamp": "Sun 2019-10-13 01:59:37 KST", "ExecMainStartTimestampMonotonic": "21918018057", "ExecMainStatus": "255", "ExecStart": "{ path=/usr/bin/kubelet ; argv[]=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/etc/systemd/system/kubelet.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "kubelet.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2019-10-13 01:59:37 KST", "InactiveEnterTimestampMonotonic": "21918122251", "InactiveExitTimestamp": "Sun 2019-10-13 01:59:37 KST", "InactiveExitTimestampMonotonic": "21918142006", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "7259", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7259", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "kubelet.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "system.slice basic.target", "Restart": "always", "RestartUSec": "10s", "Result": "exit-code", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "auto-restart", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "root", "WantedBy": "multi-user.target", "Wants": "docker.socket", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}}
TASK [kubernetes/node : Enable kubelet] ****************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:43 +0900 (0:00:00.521) 0:10:45.021 ********
ok: [kubeworker] => {"changed": false, "enabled": true, "name": "kubelet", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-10-13 01:59:43 KST", "ActiveEnterTimestampMonotonic": "21921052314", "ActiveExitTimestamp": "Sun 2019-10-13 01:59:43 KST", "ActiveExitTimestampMonotonic": "21921183067", "ActiveState": "activating", "After": "docker.service basic.target system.slice systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-10-13 01:59:43 KST", "AssertTimestampMonotonic": "21921046965", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-10-13 01:59:43 KST", "ConditionTimestampMonotonic": "21921046965", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Kubernetes Kubelet Server", "DevicePolicy": "auto", "Documentation": "https://github.com/GoogleCloudPlatform/kubernetes", "DropInPaths": "/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf", "Environment": "KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf\\x20--kubeconfig=/etc/kubernetes/kubelet.conf KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml", "EnvironmentFile": "/etc/sysconfig/kubelet (ignore_errors=yes)", "ExecMainCode": "1", "ExecMainExitTimestamp": "Sun 2019-10-13 01:59:43 KST", "ExecMainExitTimestampMonotonic": "21921182721", "ExecMainPID": "18325", "ExecMainStartTimestamp": "Sun 2019-10-13 01:59:43 KST", "ExecMainStartTimestampMonotonic": "21921052271", "ExecMainStatus": "255", "ExecStart": "{ path=/usr/bin/kubelet ; argv[]=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS ; ignore_errors=no ; start_time=[Sun 2019-10-13 01:59:43 KST] ; stop_time=[Sun 2019-10-13 01:59:43 KST] ; pid=18325 ; code=exited ; status=255 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/etc/systemd/system/kubelet.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "kubelet.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2019-10-13 01:59:43 KST", "InactiveEnterTimestampMonotonic": "21921183067", "InactiveExitTimestamp": "Sun 2019-10-13 01:59:43 KST", "InactiveExitTimestampMonotonic": "21921190388", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "7257", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7257", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "kubelet.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "always", "RestartUSec": "10s", "Result": "exit-code", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "auto-restart", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "root", "WantedBy": "multi-user.target", "Wants": "docker.socket system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}}
ok: [kubemaster] => {"changed": false, "enabled": true, "name": "kubelet", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-10-13 01:59:43 KST", "ActiveEnterTimestampMonotonic": "21924437365", "ActiveExitTimestamp": "Sun 2019-10-13 01:59:43 KST", "ActiveExitTimestampMonotonic": "21924507698", "ActiveState": "activating", "After": "docker.service system.slice basic.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-10-13 01:59:43 KST", "AssertTimestampMonotonic": "21924430548", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-10-13 01:59:43 KST", "ConditionTimestampMonotonic": "21924430547", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Kubernetes Kubelet Server", "DevicePolicy": "auto", "Documentation": "https://github.com/GoogleCloudPlatform/kubernetes", "DropInPaths": "/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf", "Environment": "KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml", "EnvironmentFile": "/etc/sysconfig/kubelet (ignore_errors=yes)", "ExecMainCode": "1", "ExecMainExitTimestamp": "Sun 2019-10-13 01:59:43 KST", "ExecMainExitTimestampMonotonic": "21924507416", "ExecMainPID": "32438", "ExecMainStartTimestamp": "Sun 2019-10-13 01:59:43 KST", "ExecMainStartTimestampMonotonic": "21924437286", "ExecMainStatus": "255", "ExecStart": "{ path=/usr/bin/kubelet ; argv[]=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS ; ignore_errors=no ; start_time=[Sun 2019-10-13 01:59:43 KST] ; stop_time=[Sun 2019-10-13 01:59:43 KST] ; pid=32438 ; code=exited ; status=255 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/etc/systemd/system/kubelet.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "kubelet.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2019-10-13 01:59:43 KST", "InactiveEnterTimestampMonotonic": "21924507698", "InactiveExitTimestamp": "Sun 2019-10-13 01:59:43 KST", "InactiveExitTimestampMonotonic": "21924520854", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "7259", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7259", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "kubelet.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "system.slice basic.target", "Restart": "always", "RestartUSec": "10s", "Result": "exit-code", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "auto-restart", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "root", "WantedBy": "multi-user.target", "Wants": "docker.socket", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}}
PLAY [kube-master] *************************************************************************************************************************************************************************************************************************
TASK [download : prep_download | Set a few facts] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:44 +0900 (0:00:00.550) 0:10:45.572 ********
TASK [download : Set image info command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:44 +0900 (0:00:00.078) 0:10:45.650 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:44 +0900 (0:00:00.075) 0:10:45.726 ********
TASK [download : prep_download | Create staging directory on remote node] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:44 +0900 (0:00:00.088) 0:10:45.814 ********
TASK [download : prep_download | Create local cache for files and images] ******************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:44 +0900 (0:00:00.107) 0:10:45.922 ********
TASK [download : prep_download | On localhost, check if passwordless root is possible] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:59:44 +0900 (0:00:00.119) 0:10:46.042 ********
TASK [download : prep_download | On localhost, check if user has access to docker without using sudo] **************************************************************************************************************************************
Sunday 13 October 2019 01:59:44 +0900 (0:00:00.105) 0:10:46.147 ********
TASK [download : prep_download | Parse the outputs of the previous commands] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:44 +0900 (0:00:00.075) 0:10:46.223 ********
TASK [download : prep_download | Check that local user is in group or can become root] *****************************************************************************************************************************************************
Sunday 13 October 2019 01:59:44 +0900 (0:00:00.083) 0:10:46.306 ********
TASK [download : include_tasks] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:44 +0900 (0:00:00.080) 0:10:46.387 ********
TASK [download : download | Get kubeadm binary and list of required images] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:45 +0900 (0:00:00.075) 0:10:46.462 ********
TASK [download : download | Download files / images] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:45 +0900 (0:00:00.080) 0:10:46.543 ********
TASK [download : download | Sync files / images from ansible host to nodes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:46 +0900 (0:00:01.384) 0:10:47.927 ********
TASK [kubespray-defaults : Configure defaults] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:47 +0900 (0:00:01.377) 0:10:49.305 ********
ok: [kubemaster] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
TASK [kubernetes/tokens : Check_tokens | check if the tokens have already been generated on first master] **********************************************************************************************************************************
Sunday 13 October 2019 01:59:48 +0900 (0:00:00.171) 0:10:49.476 ********
TASK [kubernetes/tokens : Check_tokens | Set default value for 'sync_tokens' and 'gen_tokens' to false] ************************************************************************************************************************************
Sunday 13 October 2019 01:59:48 +0900 (0:00:00.078) 0:10:49.554 ********
TASK [kubernetes/tokens : Check_tokens | Set 'sync_tokens' and 'gen_tokens' to true] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:59:48 +0900 (0:00:00.069) 0:10:49.624 ********
TASK [kubernetes/tokens : Check tokens | check if a cert already exists] *******************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:48 +0900 (0:00:00.069) 0:10:49.694 ********
TASK [kubernetes/tokens : Check_tokens | Set 'sync_tokens' to true] ************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:48 +0900 (0:00:00.069) 0:10:49.764 ********
TASK [kubernetes/tokens : Make sure the tokens directory exits] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:48 +0900 (0:00:00.074) 0:10:49.839 ********
TASK [kubernetes/tokens : Gen_tokens | copy tokens generation script] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:48 +0900 (0:00:00.121) 0:10:49.960 ********
TASK [kubernetes/tokens : Gen_tokens | generate tokens for master components] **************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:48 +0900 (0:00:00.151) 0:10:50.111 ********
TASK [kubernetes/tokens : Gen_tokens | generate tokens for node components] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:48 +0900 (0:00:00.149) 0:10:50.260 ********
TASK [kubernetes/tokens : Gen_tokens | Get list of tokens from first master] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:48 +0900 (0:00:00.156) 0:10:50.417 ********
TASK [kubernetes/tokens : Gen_tokens | Gather tokens] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:49 +0900 (0:00:00.138) 0:10:50.556 ********
TASK [kubernetes/tokens : Gen_tokens | Copy tokens on masters] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:49 +0900 (0:00:00.072) 0:10:50.629 ********
TASK [kubernetes/master : Pre-upgrade | Delete master manifests if etcd secrets changed] ***************************************************************************************************************************************************
Sunday 13 October 2019 01:59:49 +0900 (0:00:00.069) 0:10:50.698 ********
TASK [kubernetes/master : Pre-upgrade | Delete master containers forcefully] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:49 +0900 (0:00:00.118) 0:10:50.816 ********
TASK [kubernetes/master : Make sure the users directory exits] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:49 +0900 (0:00:00.132) 0:10:50.948 ********
TASK [kubernetes/master : Populate users for basic auth in API] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:49 +0900 (0:00:00.083) 0:10:51.032 ********
TASK [kubernetes/master : Create webhook token auth config] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:49 +0900 (0:00:00.073) 0:10:51.106 ********
TASK [kubernetes/master : Check if secret for encrypting data at rest already exist] *******************************************************************************************************************************************************
Sunday 13 October 2019 01:59:49 +0900 (0:00:00.082) 0:10:51.188 ********
TASK [kubernetes/master : Slurp secrets_encryption file if it exists] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:49 +0900 (0:00:00.069) 0:10:51.258 ********
TASK [kubernetes/master : Base 64 Decode slurped secrets_encryption.yaml file] *************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:49 +0900 (0:00:00.083) 0:10:51.342 ********
TASK [kubernetes/master : Extract secret value from secrets_encryption.yaml] ***************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:50 +0900 (0:00:00.102) 0:10:51.445 ********
TASK [kubernetes/master : Set kube_encrypt_token across master nodes] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:50 +0900 (0:00:00.143) 0:10:51.589 ********
TASK [kubernetes/master : Write secrets for encrypting secret data at rest] ****************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:50 +0900 (0:00:00.078) 0:10:51.667 ********
TASK [kubernetes/master : Install | Copy kubectl binary from download dir] *****************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:50 +0900 (0:00:00.151) 0:10:51.818 ********
ok: [kubemaster -> None] => {"changed": false, "cmd": "/usr/bin/rsync --delay-updates -F --archive --no-owner --no-group --out-format=<<CHANGED>>%i %n%L /tmp/releases/hyperkube-v1.16.0-amd64 /usr/local/bin/kubectl", "msg": ">f+++++++++ hyperkube-v1.16.0-amd64\n", "rc": 0, "stdout_lines": [">f+++++++++ hyperkube-v1.16.0-amd64"]}
TASK [kubernetes/master : install | Set kubectl binary permissions] ************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:51 +0900 (0:00:01.171) 0:10:52.990 ********
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/local/bin/kubectl", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 212345584, "state": "file", "uid": 0}
TASK [kubernetes/master : Install kubectl bash completion] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:51 +0900 (0:00:00.231) 0:10:53.221 ********
changed: [kubemaster] => {"changed": true, "cmd": "/usr/local/bin/kubectl completion bash >/etc/bash_completion.d/kubectl.sh", "delta": "0:00:00.141439", "end": "2019-10-13 01:59:52.093955", "rc": 0, "start": "2019-10-13 01:59:51.952516", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
TASK [kubernetes/master : Set kubectl bash completion file permissions] ********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:52 +0900 (0:00:00.374) 0:10:53.595 ********
changed: [kubemaster] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/bash_completion.d/kubectl.sh", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 660403, "state": "file", "uid": 0}
TASK [kubernetes/master : Disable SecurityContextDeny admission-controller and enable PodSecurityPolicy] ***********************************************************************************************************************************
Sunday 13 October 2019 01:59:52 +0900 (0:00:00.228) 0:10:53.824 ********
TASK [kubernetes/master : kubeadm | Check if old apiserver cert exists on host] ************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:52 +0900 (0:00:00.124) 0:10:53.948 ********
ok: [kubemaster -> None] => {"changed": false, "stat": {"exists": false}}
TASK [kubernetes/master : Copy old certs to the kubeadm expected path] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:52 +0900 (0:00:00.230) 0:10:54.179 ********
TASK [kubernetes/master : Install OIDC certificate] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:53 +0900 (0:00:00.262) 0:10:54.441 ********
TASK [kubernetes/master : kubeadm | Check serviceaccount key] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:53 +0900 (0:00:00.072) 0:10:54.514 ********
ok: [kubemaster] => {"changed": false, "stat": {"exists": false}}
TASK [kubernetes/master : kubeadm | Check if kubeadm has already run] **********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:53 +0900 (0:00:00.206) 0:10:54.720 ********
ok: [kubemaster] => {"changed": false, "stat": {"exists": false}}
TASK [kubernetes/master : kubeadm | Delete old admin.conf] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:53 +0900 (0:00:00.218) 0:10:54.939 ********
ok: [kubemaster] => {"changed": false, "path": "/etc/kubernetes/admin.conf", "state": "absent"}
TASK [kubernetes/master : kubeadm | Delete old static pods] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:53 +0900 (0:00:00.228) 0:10:55.167 ********
TASK [kubernetes/master : kubeadm | Forcefully delete old static pods] *********************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:53 +0900 (0:00:00.151) 0:10:55.319 ********
TASK [kubernetes/master : kubeadm | aggregate all SANs] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:53 +0900 (0:00:00.113) 0:10:55.432 ********
ok: [kubemaster] => {"ansible_facts": {"apiserver_sans": ["kubernetes", "kubernetes.default", "kubernetes.default.svc", "kubernetes.default.svc.cluster.local", "10.233.0.1", "localhost", "127.0.0.1", "kubemaster", "lb-apiserver.kubernetes.local", "192.168.56.107", "10.0.3.15"]}, "changed": false}
TASK [kubernetes/master : Create audit-policy directory] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:54 +0900 (0:00:00.364) 0:10:55.796 ********
TASK [kubernetes/master : Write api audit policy yaml] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:54 +0900 (0:00:00.081) 0:10:55.878 ********
TASK [kubernetes/master : set kubeadm_config_api_fqdn define] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:54 +0900 (0:00:00.082) 0:10:55.961 ********
TASK [kubernetes/master : gets the kubeadm version] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:54 +0900 (0:00:00.108) 0:10:56.069 ********
changed: [kubemaster] => {"changed": true, "cmd": ["/usr/local/bin/kubeadm", "version", "-o", "short"], "delta": "0:00:00.050482", "end": "2019-10-13 01:59:54.846525", "rc": 0, "start": "2019-10-13 01:59:54.796043", "stderr": "", "stderr_lines": [], "stdout": "v1.16.0", "stdout_lines": ["v1.16.0"]}
TASK [kubernetes/master : sets kubeadm api version to v1beta2] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:54 +0900 (0:00:00.283) 0:10:56.353 ********
ok: [kubemaster] => {"ansible_facts": {"kubeadmConfig_api_version": "v1beta2"}, "changed": false}
TASK [kubernetes/master : kubeadm | Create kubeadm config] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:55 +0900 (0:00:00.088) 0:10:56.441 ********
changed: [kubemaster] => {"changed": true, "checksum": "c45d3188b7275826c5fc2f2889b791fd25fa2fe2", "dest": "/etc/kubernetes/kubeadm-config.yaml", "gid": 0, "group": "root", "md5sum": "5cc89709364aba2c9e9827513069813b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 3045, "src": "/root/.ansible/tmp/ansible-tmp-1570899595.0373583-274501564407996/source", "state": "file", "uid": 0}
TASK [kubernetes/master : Backup old certs and keys] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:56 +0900 (0:00:01.023) 0:10:57.464 ********
TASK [kubernetes/master : kubeadm | Initialize first master] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 01:59:56 +0900 (0:00:00.183) 0:10:57.648 ********
changed: [kubemaster] => {"attempts": 1, "changed": true, "cmd": ["timeout", "-k", "300s", "300s", "/usr/local/bin/kubeadm", "init", "--config=/etc/kubernetes/kubeadm-config.yaml", "--ignore-preflight-errors=all", "--skip-phases=addon/coredns", "--upload-certs"], "delta": "0:00:22.574260", "end": "2019-10-13 02:00:18.948842", "failed_when_result": false, "rc": 0, "start": "2019-10-13 01:59:56.374582", "stderr": "\t[WARNING IsDockerSystemdCheck]: detected \"cgroupfs\" as the Docker cgroup driver. The recommended driver is \"systemd\". Please follow the guide at https://kubernetes.io/docs/setup/cri/", "stderr_lines": ["\t[WARNING IsDockerSystemdCheck]: detected \"cgroupfs\" as the Docker cgroup driver. The recommended driver is \"systemd\". Please follow the guide at https://kubernetes.io/docs/setup/cri/"], "stdout": "[init] Using Kubernetes version: v1.16.0\n[preflight] Running pre-flight checks\n[preflight] Pulling images required for setting up a Kubernetes cluster\n[preflight] This might take a minute or two, depending on the speed of your internet connection\n[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'\n[kubelet-start] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"\n[kubelet-start] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"\n[kubelet-start] Activating the kubelet service\n[certs] Using certificateDir folder \"/etc/kubernetes/ssl\"\n[certs] Generating \"ca\" certificate and key\n[certs] Generating \"apiserver\" certificate and key\n[certs] apiserver serving cert is signed for DNS names [kubemaster kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost kubemaster lb-apiserver.kubernetes.local] and IPs [10.233.0.1 192.168.56.107 192.168.56.107 10.233.0.1 127.0.0.1 192.168.56.107 10.0.3.15]\n[certs] Generating \"apiserver-kubelet-client\" certificate and key\n[certs] Generating \"front-proxy-ca\" certificate and key\n[certs] Generating \"front-proxy-client\" certificate and key\n[certs] External etcd mode: Skipping etcd/ca certificate authority generation\n[certs] External etcd mode: Skipping etcd/server certificate generation\n[certs] External etcd mode: Skipping etcd/peer certificate generation\n[certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation\n[certs] External etcd mode: Skipping apiserver-etcd-client certificate generation\n[certs] Generating \"sa\" key and public key\n[kubeconfig] Using kubeconfig folder \"/etc/kubernetes\"\n[kubeconfig] Writing \"admin.conf\" kubeconfig file\n[kubeconfig] Writing \"kubelet.conf\" kubeconfig file\n[kubeconfig] Writing \"controller-manager.conf\" kubeconfig file\n[kubeconfig] Writing \"scheduler.conf\" kubeconfig file\n[control-plane] Using manifest folder \"/etc/kubernetes/manifests\"\n[control-plane] Creating static Pod manifest for \"kube-apiserver\"\n[controlplane] Adding extra host path mount \"etc-pki-tls\" to \"kube-apiserver\"\n[controlplane] Adding extra host path mount \"etc-pki-ca-trust\" to \"kube-apiserver\"\n[control-plane] Creating static Pod manifest for \"kube-controller-manager\"\n[controlplane] Adding extra host path mount \"etc-pki-tls\" to \"kube-apiserver\"\n[controlplane] Adding extra host path mount \"etc-pki-ca-trust\" to \"kube-apiserver\"\n[control-plane] Creating static Pod manifest for \"kube-scheduler\"\n[controlplane] Adding extra host path mount \"etc-pki-tls\" to \"kube-apiserver\"\n[controlplane] Adding extra host path mount \"etc-pki-ca-trust\" to \"kube-apiserver\"\n[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory \"/etc/kubernetes/manifests\". This can take up to 5m0s\n[apiclient] All control plane components are healthy after 18.002531 seconds\n[upload-config] Storing the configuration used in ConfigMap \"kubeadm-config\" in the \"kube-system\" Namespace\n[kubelet] Creating a ConfigMap \"kubelet-config-1.16\" in namespace kube-system with the configuration for the kubelets in the cluster\n[upload-certs] Storing the certificates in Secret \"kubeadm-certs\" in the \"kube-system\" Namespace\n[upload-certs] Using certificate key:\n3647c577d355e585beee351cebdf827f836039a0f6c6d3a94d7bdabfd76bd698\n[mark-control-plane] Marking the node kubemaster as control-plane by adding the label \"node-role.kubernetes.io/master=''\"\n[mark-control-plane] Marking the node kubemaster as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]\n[bootstrap-token] Using token: 27ilft.02qafa3746z2p8ue\n[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles\n[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials\n[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token\n[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster\n[bootstrap-token] Creating the \"cluster-info\" ConfigMap in the \"kube-public\" namespace\n[addons] Applied essential addon: kube-proxy\n\nYour Kubernetes control-plane has initialized successfully!\n\nTo start using your cluster, you need to run the following as a regular user:\n\n mkdir -p $HOME/.kube\n sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config\n sudo chown $(id -u):$(id -g) $HOME/.kube/config\n\nYou should now deploy a pod network to the cluster.\nRun \"kubectl apply -f [podnetwork].yaml\" with one of the options listed at:\n https://kubernetes.io/docs/concepts/cluster-administration/addons/\n\nYou can now join any number of the control-plane node running the following command on each as root:\n\n kubeadm join 192.168.56.107:6443 --token 27ilft.02qafa3746z2p8ue \\\n --discovery-token-ca-cert-hash sha256:27b4e9454913a040325c2c88deea4d3e41bb5047f7b0a522cc2ac3d6b67102be \\\n --control-plane --certificate-key 3647c577d355e585beee351cebdf827f836039a0f6c6d3a94d7bdabfd76bd698\n\nPlease note that the certificate-key gives access to cluster sensitive data, keep it secret!\nAs a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use \n\"kubeadm init phase upload-certs --upload-certs\" to reload certs afterward.\n\nThen you can join any number of worker nodes by running the following on each as root:\n\nkubeadm join 192.168.56.107:6443 --token 27ilft.02qafa3746z2p8ue \\\n --discovery-token-ca-cert-hash sha256:27b4e9454913a040325c2c88deea4d3e41bb5047f7b0a522cc2ac3d6b67102be ", "stdout_lines": ["[init] Using Kubernetes version: v1.16.0", "[preflight] Running pre-flight checks", "[preflight] Pulling images required for setting up a Kubernetes cluster", "[preflight] This might take a minute or two, depending on the speed of your internet connection", "[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'", "[kubelet-start] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"", "[kubelet-start] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"", "[kubelet-start] Activating the kubelet service", "[certs] Using certificateDir folder \"/etc/kubernetes/ssl\"", "[certs] Generating \"ca\" certificate and key", "[certs] Generating \"apiserver\" certificate and key", "[certs] apiserver serving cert is signed for DNS names [kubemaster kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost kubemaster lb-apiserver.kubernetes.local] and IPs [10.233.0.1 192.168.56.107 192.168.56.107 10.233.0.1 127.0.0.1 192.168.56.107 10.0.3.15]", "[certs] Generating \"apiserver-kubelet-client\" certificate and key", "[certs] Generating \"front-proxy-ca\" certificate and key", "[certs] Generating \"front-proxy-client\" certificate and key", "[certs] External etcd mode: Skipping etcd/ca certificate authority generation", "[certs] External etcd mode: Skipping etcd/server certificate generation", "[certs] External etcd mode: Skipping etcd/peer certificate generation", "[certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation", "[certs] External etcd mode: Skipping apiserver-etcd-client certificate generation", "[certs] Generating \"sa\" key and public key", "[kubeconfig] Using kubeconfig folder \"/etc/kubernetes\"", "[kubeconfig] Writing \"admin.conf\" kubeconfig file", "[kubeconfig] Writing \"kubelet.conf\" kubeconfig file", "[kubeconfig] Writing \"controller-manager.conf\" kubeconfig file", "[kubeconfig] Writing \"scheduler.conf\" kubeconfig file", "[control-plane] Using manifest folder \"/etc/kubernetes/manifests\"", "[control-plane] Creating static Pod manifest for \"kube-apiserver\"", "[controlplane] Adding extra host path mount \"etc-pki-tls\" to \"kube-apiserver\"", "[controlplane] Adding extra host path mount \"etc-pki-ca-trust\" to \"kube-apiserver\"", "[control-plane] Creating static Pod manifest for \"kube-controller-manager\"", "[controlplane] Adding extra host path mount \"etc-pki-tls\" to \"kube-apiserver\"", "[controlplane] Adding extra host path mount \"etc-pki-ca-trust\" to \"kube-apiserver\"", "[control-plane] Creating static Pod manifest for \"kube-scheduler\"", "[controlplane] Adding extra host path mount \"etc-pki-tls\" to \"kube-apiserver\"", "[controlplane] Adding extra host path mount \"etc-pki-ca-trust\" to \"kube-apiserver\"", "[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory \"/etc/kubernetes/manifests\". This can take up to 5m0s", "[apiclient] All control plane components are healthy after 18.002531 seconds", "[upload-config] Storing the configuration used in ConfigMap \"kubeadm-config\" in the \"kube-system\" Namespace", "[kubelet] Creating a ConfigMap \"kubelet-config-1.16\" in namespace kube-system with the configuration for the kubelets in the cluster", "[upload-certs] Storing the certificates in Secret \"kubeadm-certs\" in the \"kube-system\" Namespace", "[upload-certs] Using certificate key:", "3647c577d355e585beee351cebdf827f836039a0f6c6d3a94d7bdabfd76bd698", "[mark-control-plane] Marking the node kubemaster as control-plane by adding the label \"node-role.kubernetes.io/master=''\"", "[mark-control-plane] Marking the node kubemaster as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]", "[bootstrap-token] Using token: 27ilft.02qafa3746z2p8ue", "[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles", "[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials", "[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token", "[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster", "[bootstrap-token] Creating the \"cluster-info\" ConfigMap in the \"kube-public\" namespace", "[addons] Applied essential addon: kube-proxy", "", "Your Kubernetes control-plane has initialized successfully!", "", "To start using your cluster, you need to run the following as a regular user:", "", " mkdir -p $HOME/.kube", " sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config", " sudo chown $(id -u):$(id -g) $HOME/.kube/config", "", "You should now deploy a pod network to the cluster.", "Run \"kubectl apply -f [podnetwork].yaml\" with one of the options listed at:", " https://kubernetes.io/docs/concepts/cluster-administration/addons/", "", "You can now join any number of the control-plane node running the following command on each as root:", "", " kubeadm join 192.168.56.107:6443 --token 27ilft.02qafa3746z2p8ue \\", " --discovery-token-ca-cert-hash sha256:27b4e9454913a040325c2c88deea4d3e41bb5047f7b0a522cc2ac3d6b67102be \\", " --control-plane --certificate-key 3647c577d355e585beee351cebdf827f836039a0f6c6d3a94d7bdabfd76bd698", "", "Please note that the certificate-key gives access to cluster sensitive data, keep it secret!", "As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use ", "\"kubeadm init phase upload-certs --upload-certs\" to reload certs afterward.", "", "Then you can join any number of worker nodes by running the following on each as root:", "", "kubeadm join 192.168.56.107:6443 --token 27ilft.02qafa3746z2p8ue \\", " --discovery-token-ca-cert-hash sha256:27b4e9454913a040325c2c88deea4d3e41bb5047f7b0a522cc2ac3d6b67102be "]}
TASK [kubernetes/master : set kubeadm certificate key] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:19 +0900 (0:00:22.906) 0:11:20.555 ********
ok: [kubemaster] => (item= --control-plane --certificate-key 3647c577d355e585beee351cebdf827f836039a0f6c6d3a94d7bdabfd76bd698) => {"ansible_facts": {"kubeadm_certificate_key": "3647c577d355e585beee351cebdf827f836039a0f6c6d3a94d7bdabfd76bd698"}, "ansible_loop_var": "item", "changed": false, "item": " --control-plane --certificate-key 3647c577d355e585beee351cebdf827f836039a0f6c6d3a94d7bdabfd76bd698"}
TASK [kubernetes/master : Create hardcoded kubeadm token for joining nodes with 24h expiration (if defined)] *******************************************************************************************************************************
Sunday 13 October 2019 02:00:20 +0900 (0:00:01.627) 0:11:22.182 ********
TASK [kubernetes/master : Create kubeadm token for joining nodes with 24h expiration (default)] ********************************************************************************************************************************************
Sunday 13 October 2019 02:00:20 +0900 (0:00:00.159) 0:11:22.341 ********
changed: [kubemaster -> None] => {"attempts": 1, "changed": true, "cmd": ["/usr/local/bin/kubeadm", "--kubeconfig", "/etc/kubernetes/admin.conf", "token", "create"], "delta": "0:00:00.076661", "end": "2019-10-13 02:00:21.223139", "rc": 0, "start": "2019-10-13 02:00:21.146478", "stderr": "", "stderr_lines": [], "stdout": "kixx0y.dkdcx6glpg4h1dib", "stdout_lines": ["kixx0y.dkdcx6glpg4h1dib"]}
TASK [kubernetes/master : Set kubeadm_token] ***********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:21 +0900 (0:00:00.389) 0:11:22.730 ********
ok: [kubemaster] => {"ansible_facts": {"kubeadm_token": "kixx0y.dkdcx6glpg4h1dib"}, "changed": false}
TASK [kubernetes/master : Set kubeadm_discovery_address] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:21 +0900 (0:00:00.099) 0:11:22.829 ********
TASK [kubernetes/master : Upload certificates so they are fresh and not expired] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:00:21 +0900 (0:00:00.070) 0:11:22.900 ********
TASK [kubernetes/master : Parse certificate key if not set] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:21 +0900 (0:00:00.082) 0:11:22.982 ********
TASK [kubernetes/master : Create kubeadm ControlPlane config] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:21 +0900 (0:00:00.072) 0:11:23.055 ********
TASK [kubernetes/master : Wait for k8s apiserver] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:21 +0900 (0:00:00.079) 0:11:23.135 ********
TASK [kubernetes/master : check already run] ***********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:21 +0900 (0:00:00.084) 0:11:23.220 ********
TASK [kubernetes/master : Joining control plane node to the cluster.] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:21 +0900 (0:00:00.086) 0:11:23.306 ********
TASK [kubernetes/master : Set secret_changed to false to avoid extra token rotation] *******************************************************************************************************************************************************
Sunday 13 October 2019 02:00:21 +0900 (0:00:00.074) 0:11:23.381 ********
TASK [kubernetes/master : slurp kubeadm certs] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:22 +0900 (0:00:00.160) 0:11:23.541 ********
ok: [kubemaster -> None] => (item=/etc/kubernetes/ssl/apiserver.crt) => {"ansible_loop_var": "item", "changed": false, "content": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVFekNDQXZ1Z0F3SUJBZ0lITjErWEZMamF5ekFOQmdrcWhraUc5dzBCQVFzRkFEQVZNUk13RVFZRFZRUUQKRXdwcmRXSmxjbTVsZEdWek1CNFhEVEU1TVRBeE1qRTJOVGsxTjFvWERUSXdNVEF4TVRFMk5UazFPRm93R1RFWApNQlVHQTFVRUF4TU9hM1ZpWlMxaGNHbHpaWEoyWlhJd2dnRWlNQTBHQ1NxR1NJYjNEUUVCQVFVQUE0SUJEd0F3CmdnRUtBb0lCQVFES2pyQkZnbWlVdDBHdUd3KzRsVjFsbzVwelZvcldEeC9XNzBQbysxWThhME54bDlFN0o0ZWkKQVNXN1JDTW9tMFJRdlVIZklRbFdKc0dxaC9xaGw2R3V0eDBBc1ZyOWJrR2hnUE5kTlkzeTg4RTRxbnBOUjl4bwoxRytYTWxtQTRURjZCU1o3aVRxOFVsbHlrd3pYOXQ1d2thWm14UU5SdHExQUdnOUJYNWZFeFB5NWJaa25KQ21zCk1BY3RwcGxLMmF4WDBxTk9BNDRLUzdaVjVQaWNEK1hMMEE5bllUaituL09DcE41WVJpcGFxN0NjWjlFMncvbkEKeS8zekt6TUZQdEd6cUZaOExoS2hFVGU3QWQ2MHBtdmgxdnp1RUNlRk52WU1PeEphMHBHa1dNUjhNa1U4aTk4TApXNHNqUk1VWSt5UXl4bWNuS2RaN0w1b0t2TGo1cmNHRkFnTUJBQUdqZ2dGaU1JSUJYakFPQmdOVkhROEJBZjhFCkJBTUNCYUF3RXdZRFZSMGxCQXd3Q2dZSUt3WUJCUVVIQXdFd2dnRTFCZ05WSFJFRWdnRXNNSUlCS0lJS2EzVmkKWlcxaGMzUmxjb0lLYTNWaVpYSnVaWFJsYzRJU2EzVmlaWEp1WlhSbGN5NWtaV1poZFd4MGdoWnJkV0psY201bApkR1Z6TG1SbFptRjFiSFF1YzNaamdpUnJkV0psY201bGRHVnpMbVJsWm1GMWJIUXVjM1pqTG1Oc2RYTjBaWEl1CmJHOWpZV3lDQ210MVltVnlibVYwWlhPQ0VtdDFZbVZ5Ym1WMFpYTXVaR1ZtWVhWc2RJSVdhM1ZpWlhKdVpYUmwKY3k1a1pXWmhkV3gwTG5OMlk0SWthM1ZpWlhKdVpYUmxjeTVrWldaaGRXeDBMbk4yWXk1amJIVnpkR1Z5TG14dgpZMkZzZ2dsc2IyTmhiR2h2YzNTQ0NtdDFZbVZ0WVhOMFpYS0NIV3hpTFdGd2FYTmxjblpsY2k1cmRXSmxjbTVsCmRHVnpMbXh2WTJGc2h3UUs2UUFCaHdUQXFEaHJod1RBcURocmh3UUs2UUFCaHdSL0FBQUJod1RBcURocmh3UUsKQUFNUE1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQXlCV0VjOGhpZjY3MVRsZnU3RDNXY2ZnbUdnaDF5Z2IyRwpCd2dlNi9iMjFQRnFZa01XQkNEUFdiR1AwRDVlTTlNVWduR1RBNFh5STJrcTVjRmVBWExIZHp3VmVWMWNQUEtmClhmRHJ4d1JLNC9oWnJMSlNoa0psVVYySFFQTHhaeXk5aWZnVllRSDl1c2IvKzNRUmh5KzVWaHQ3eE1XNENDM0YKRjZodkNOL2JiOG5rYVRGOXE3UHR5bUhjaGhaelVVbmdpb2xORFU3SHBValdaTmFFZy9VNnBBTHhnR3lPVDlJdgpVTVRmbzRucTFzb3BYNE5ObjlWN3pJUlU2TUhkVy9UdWhhNHhuNlh5ZmRBbjRMam5KZFNMak90VVRlREI2YWpECjZjaWQwakdycFNuM1ZmV1NXaVp4VTBwejNBbm93RkZ6ODNsMjZQNFlIb2FUemNDL2xMazcKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=", "encoding": "base64", "item": "/etc/kubernetes/ssl/apiserver.crt", "source": "/etc/kubernetes/ssl/apiserver.crt"}
ok: [kubemaster -> None] => (item=/etc/kubernetes/ssl/apiserver.key) => {"ansible_loop_var": "item", "changed": false, "content": "LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb2dJQkFBS0NBUUVBeW82d1JZSm9sTGRCcmhzUHVKVmRaYU9hYzFhSzFnOGYxdTlENlB0V1BHdERjWmZSCk95ZUhvZ0VsdTBRaktKdEVVTDFCM3lFSlZpYkJxb2Y2b1plaHJyY2RBTEZhL1c1Qm9ZRHpYVFdOOHZQQk9LcDYKVFVmY2FOUnZsekpaZ09FeGVnVW1lNGs2dkZKWmNwTU0xL2JlY0pHbVpzVURVYmF0UUJvUFFWK1h4TVQ4dVcyWgpKeVFwckRBSExhYVpTdG1zVjlLalRnT09Da3UyVmVUNG5BL2x5OUFQWjJFNC9wL3pncVRlV0VZcVdxdXduR2ZSCk5zUDV3TXY5OHlzekJUN1JzNmhXZkM0U29SRTN1d0hldEtacjRkYjg3aEFuaFRiMkREc1NXdEtScEZqRWZESkYKUEl2ZkMxdUxJMFRGR1Bza01zWm5KeW5XZXkrYUNyeTQrYTNCaFFJREFRQUJBb0lCQUFiTXMyNGg2cERUZGdqNgpMNVRxcnEveXhnU0pLeS8wWWdVOUV2S0hQV1haMC9IRVB1blZjZUFtV2JYbm02YUpUOG5HRUtnRTIrTmdlYS9YCjBMb0w5UW1BYkNEUjAxUWVsMlZzaHI3aVJWbWFoU241enhFNUVGVVVqSXB1emtKZFRPUGp1Rjc5UkZOK0VES2cKYlJYeTI1Uzd6dkkwT3VCTFdaV0ZTbTNQWDdEalBLM2hyUmV4enhmbWFKRzdXdEVPYlYzaXlaYUU0STBCM0NGUwoyQVZHMUk2cXV3dmVWY2IybnNBOCttOWdyMmkzNGd0dFN6TE14dkNWUXNUSHMrOGNiaCtneVJqbENWbHRDS21rCkw1RzhMOTdyTmJSZERsTXNZbGdFKy9SL0QvMjdjTzlNRnpqN05PZzJEckkzRDlZMWtabnBCNmJEZ0FDcE1Ya3AKRzRLMGlpRUNnWUVBNklqMWU5dW9tSnZjb0ZJbGFtc3d2UkhNTkRBWXdiam93emRlWWZQMzdlK1dqQzdSa00yWgoyRXRCMy94RnRtWEZtZ3RFODhnU0lac1JWNlBNaHczRExaa243SklNRzRBUEtEd0ZlTUwzajliUjdOckZsc2JVCk9PM2VNYVJiRXJ6SWI0cTIrTFNEVDJNWWNaSHAxNEY5MzVhcnl6TXErZXJyZUJSMlNuQUt2ZmNDZ1lFQTN2OVMKaURDZE9HMjFGN2ErR0FUQVpCRHlBK2tJc2doK3ZwNHQvdWVIZHNGUHA3QjgyT25EV01vSlVMUVRObHB6UTh2YwpScDArTll1ajZ2VzNCbWNtVWJZWEl2aFFQK3MwRmlxajBpdmtxbGZ0UUloRHZIaEFsWUlNVzJ1a2xvemR3Snd2Ckl2c3UrV0laVmd1UXJwdUR6bkhvS0l0ZDlUYkNmT3JOelF2MVRXTUNnWUJvbjg3TDVwNHA0OTdWcDRBOExIUm0KYlU2U29KMVM5RTZPSHloeExDTCtSRkZaUzJ0eEdvREE1OE9BOHZHdXlvV285SzhXZkl0ajRYQmhIVXJWN1NSSApMT1RndFNtTGFnZm4wM0dtenhveDlyYkhPZnV6Qmt6UVBINC9tYko4Y01KQ0pZaUR5RHhZaUw2andzSVMrNmxWCjQyWHliRmhHMENNWUNTVGNBeU5pS3dLQmdDcGZiNnBMT29TdmEvcStrYTUwc0s0Uy9qbTUvUm4vUFQ0eFJtSXEKVW5aQlUwWjA1YlE5N3cyZkRsZ1ViNVlqWjhXWkpoa1E1Zkk4djMvMUFyc252SU5JUzJzc3Jlc0hZVlZZOHRvTAoxeDU3S0t6YmQ2dUhXaFpPcU5SeVJzc0lwMVpqRngzWmh0cUNSbmd3S0dUVlRoMWJUWmJ1cklMY3plWDRvL1J6CmNrQXJBb0dBVmZqSEFSVUtnVjZ5YXEycWRYYk10dGxyWWwrREtrczQvQ3lXM3hVUmZsNWFvanNiT01BWXc0V3cKcGZhL0xTRzg0S1B0aW1XUWtOZk50YU91N1VBNzRWbWNhd043c1Y2NEszTlNIcFJDVWpPWjFjc0Z2RlFzMHdndAphVHdKWnpUWmoxaUxPSEFYZFllYlQ1aEt3VHRlNEl2ZmxPQmp3TVI3MXhiekpVQ1FJaUk9Ci0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==", "encoding": "base64", "item": "/etc/kubernetes/ssl/apiserver.key", "source": "/etc/kubernetes/ssl/apiserver.key"}
ok: [kubemaster -> None] => (item=/etc/kubernetes/ssl/apiserver-kubelet-client.crt) => {"ansible_loop_var": "item", "changed": false, "content": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvekNDQWVlZ0F3SUJBZ0lJRm96ZFI2dFA1dXd3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB4T1RFd01USXhOalU1TlRkYUZ3MHlNREV3TVRFeE5qVTVOVGhhTUVFeApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1TWXdKQVlEVlFRREV4MXJkV0psTFdGd2FYTmxjblpsCmNpMXJkV0psYkdWMExXTnNhV1Z1ZERDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUIKQUtGNmo1ejhlTzljUDdvQThtbVVheHBqUEFBWXE1VXlzWEF3eDNCWUFyMmRxb3h6Y3VBcnRVZ2YzOWxwZVBVSwpOUld2Q1ljUVlDVzRlSlZ3WTVQc0orS0h4MWN3czJYckJyZk9sUStzZGhlQXl4RnJ4eDg1R29ML3pUYWRFcVloCi9ScDNabFE1cXRXLzhRY3FldjNGRis5ajVyeGtROW4rQTNvTGJhQUFjK01ya1o0ekVNL3NjRUZKZHN6Z1NsOFUKY2tWNHlmSTZUNkNkU08xb0h6TFl5eTlhWTNnRDZBOTRESmluVDlEZ3RISnEyQ2lrVkdvM2JUcmdXOW5EYnZYcwpjN2QyenB1ME16Q1RtdjZVTFZodmt3TUpQSG9GS3IzMnpSQzI0blYzRk9FUTQ3RXVBWnFzVUYya2RWRHc1cjRwClVMMkNrd2dWZGovejFZcHpqY3dBZWlNQ0F3RUFBYU1uTUNVd0RnWURWUjBQQVFIL0JBUURBZ1dnTUJNR0ExVWQKSlFRTU1Bb0dDQ3NHQVFVRkJ3TUNNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUFLaW9RRU9FVXB6UndYY2FOZAp6RXZVNzY0YzRBQXd3UGhNWERoc2dVdkpucytYRXQxekVlRkpSbmJjMlU1Q0kzT28xSkw1Uk42eTN1Z1Q1N09mCnFQdU0ybWVQaEs0QjRFK0hjaUZRYnBmekdCSk9GM0JRK0h4NURKR29rVkZ5ZUZSYUVYT3oxcGJvMWR6aFRnekIKVzZVVHFaVUpod25GRmEyTTlJVCtIWUZBUHBTTGdvelZVMmJiWGhldTY0UUI3OUZFcXlMQUw4dHQxb2t5T2FRUQpQN1FHcGhwYWVoL0FaQWE3MzRNc1piUTZYWUV0RzR6VVk0R250eDFCWXNtbTFJUmVLZ0tIQkF0K3k4UGhJbGhnCklkNXd2UUJ4YnlqUEoxRllkc1RMWHlZcGtmSHkzK1NsZTczcEhJSUtSUmZGUzZoWXk5SHdvZm9IZzBNQm04RDgKaFJLcQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==", "encoding": "base64", "item": "/etc/kubernetes/ssl/apiserver-kubelet-client.crt", "source": "/etc/kubernetes/ssl/apiserver-kubelet-client.crt"}
ok: [kubemaster -> None] => (item=/etc/kubernetes/ssl/apiserver-kubelet-client.key) => {"ansible_loop_var": "item", "changed": false, "content": "LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBb1hxUG5QeDQ3MXcvdWdEeWFaUnJHbU04QUJpcmxUS3hjRERIY0ZnQ3ZaMnFqSE55CjRDdTFTQi9mMldsNDlRbzFGYThKaHhCZ0piaDRsWEJqayt3bjRvZkhWekN6WmVzR3Q4NlZENngyRjRETEVXdkgKSHprYWd2L05OcDBTcGlIOUduZG1WRG1xMWIveEJ5cDYvY1VYNzJQbXZHUkQyZjREZWd0dG9BQno0eXVSbmpNUQp6K3h3UVVsMnpPQktYeFJ5UlhqSjhqcFBvSjFJN1dnZk10akxMMXBqZUFQb0QzZ01tS2RQME9DMGNtcllLS1JVCmFqZHRPdUJiMmNOdTlleHp0M2JPbTdRek1KT2EvcFF0V0crVEF3azhlZ1VxdmZiTkVMYmlkWGNVNFJEanNTNEIKbXF4UVhhUjFVUERtdmlsUXZZS1RDQlYyUC9QVmluT056QUI2SXdJREFRQUJBb0lCQURqZGQwdWFZV0F4aFVPVQpaOG1rZVo4ZVNSZmptREQvWEJzZEltNituNkIwc0lXcmRrM1pOb3JDcHNyaWFMVXFOZzEveVlRbDBTWVhjMmQwCjM2eE00ckxVaSs5NWVBT3hJM21CSjE4UXBibWJJbWZ0STM4c1ZkakNDbHRzTHhBUU1oaXE2WDlKbTQ2UWZHbkEKblBWVy9oblFmWVhrYlR5TVJsOHBQUjYyYWhrYVJIakJWaVJxQ2E3bXc0YU9ueHMxMkljV2MrTnhwL1NkeWJQcwpVUkdDYjFzVURkM3JmZWpJTTRqaVRnaUZPb3Q1Rm9FQitOcXNEZHlIa0tSaEU3MExmRXYrd1R3dzJkNmt1RnVjCmhzKy9KQk1qSkFkYkZtK1BYUnlORDhXY1liTEM0L0h3Y0JGa1hTdzdWNjYwT2ZZQThXRENVRHpFRm5zNmdKNi8Kb1pKbk40a0NnWUVBMFJyZHNtZjZQL1o1b2pIbEVrUEhoT24vOTJoaTFMbzVYMU5hZGR5RXNEa1RQV0l6K1dCMAp6VGJlK1dKUjcwamEvWmdFOW1lbTRRWGpCUmh2ZVVqV0VKNXRCSWRsL3h2YmpKc2JjUENHdE5Md0p3aVE4ejVSCjVKamRmNjN6S0Y0WG51bFlsZnM3RUlCS2h0YlUyYUN2eDBMR053aHNyNXF4VUF5WEJlKzgxSmNDZ1lFQXhiRmgKUC9NQ04wckx1TG9QZDJiQVIzNk9wd05saGlxd1NuRHBZNzd2VExkdmI4NGVWYnA0UjRBbnVMWSttdW82a0tmWQpONHUyRnQyazBmVlgwWWt6aGRzWkd2Y1pNVWU1NEJYaHpFRDUzOXdYNVNGcjVseXh4MkZ6cW1KT29nT1BoSUlKClZSdDJBMDRUdFBsTG1NRUZiZWcrSThWUFc1K0pVSDFjQ0pZcHZGVUNnWUJQQkNUMGJzQXMvWG1YSmQyWEdrL0IKMVI3cG9lTisyV1FtdG51L2YvWVFGWEZFeCtBMkF4SUFIaFVDRGZwQ1ByaXFEWkVLZjc2YTE3dWl5RW9QZmN1TwplaDllRDhBbU42T3VMVXFnVWZQRlR0UjdHNHlTcm1FSnpnNGRXbUZCUnRVTU1zZEwzYVRESm9lRERwbGVKNDlrCjc4R3hPQmpCL3VXMEJ4M21tMkl5dndLQmdHSkg1Y1lLQkk4Q2N6Y1o0TGlHYjQ3YmZ3WE9yWGsrM2JlVVhLTUcKK0NhV2h5TDVRS0Z6KzNub1gxYjMyelhldmxETjhMTGtsUmpYQmFxUG95SDdhSTdpTnAvdU9pc0Nzb2lSdUZoMwo4UUYyaExhWG4vRzlJWEN5cFBNT3ZuTFMvRlZ1NUQ3WVVSb2hoYkNSd3lCa3pWTXZneno3U1pHT1JaQ3JVY2EvCkNHNmxBb0dCQUpHTnVxeFVvSE1DQldnSmVoNDRpeERTc0RTc0d5YTVSMUVVSEtEMVBqZ2t5S2R6WXJubmxuS0wKZFJmSm40VlZFV1JxUmlKMGJZcTRqUVZlTjdPUVdwQkpQNlFBaG0xVG5sYjRPR0RpQkI3cm8wcXgxMGZkZnM0Mgo2bzdrVTBDRmI1Q0lETWhNYmZ2N2RqK01pOXhUenZ3ekFJc3RFdW5jQTl2TU1jcm5sZnFZCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==", "encoding": "base64", "item": "/etc/kubernetes/ssl/apiserver-kubelet-client.key", "source": "/etc/kubernetes/ssl/apiserver-kubelet-client.key"}
ok: [kubemaster -> None] => (item=/etc/kubernetes/ssl/ca.crt) => {"ansible_loop_var": "item", "changed": false, "content": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFNU1UQXhNakUyTlRrMU4xb1hEVEk1TVRBd09URTJOVGsxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTURPCnpTbTlEVHk0cjlWc0dHOVlJZGVkTE5RdWVYcjloUXJ2bGF4aEdQOGs5KytUN21uYzQ2RW5nWmhISGFmemFBNnkKMXd2UTJ2SEpQMG9QbWEvcitCaTRncGJldDZNMmwvTzB2TXl2Z1N0RXBaYmtWK1d6QTF3NUlTNzRNK1dDNlRlKwpEUmE1NGZsQmNEamhUSHdMRWw4Q3VZMDhxcUFvSGxsT3dtVEFlU3RqeUk4MVVIcndKd1pBZUk0SzQ0SENqVTdPClh3RHJHbklKWWVWdUdTVDBNV0FjYlUzSzZKOVYraFo0dE4vVzVINkRzTnNBczgvR0JaS3Y0QmNtQXJjSEJVMysKeVJwM0o2cnhZMXBMa3lQUFhsNmh0U3R4N0J4TzZCcTVNY3pGNFNjazRhL2pSb3JvYXBZU0ZCeWdxcmdQeWVpSgpuVHVoaGNRblBpdGxPZDUzNHZVQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFGemZHc2V2OWk0NzBEMHhhME9jK0M1TURDY3kKRVoxQkQ2SU9ySVYwdWdTWHFQZkZSakU5bWxUYks1YloyaHJOZjlaSFM4d0FDL0g5THZuNHVHMFVSM3lueStEYgpXQXN6cjBxaGFlQ2x6Q0FRdUZzblo2amx4b3JjVUI1OWRPd0xsdW1PRDZpN29yUUg0WkJZa2JIa3dVbG80cFVOClJHMXNWeVZoWW81d1lBZ2xveVAvQlNMclBYbzFLTGtRMDZLdnAzclhpRkpqZUtMQjlCdlVvVzVubjR3aDI0RTIKSFRXU05IaWxCL1FZTlZUNnc3MHlISFJrTUVIL2x0aS9qRnovR0RVTXlNbjNER09sRkNqK002U3o2M0RoMEU4WApWRk1SdXJ5YjhFeUFOb1JFRlpPbFBIclR3V3JtSk9aYkVJMEFRMmJlalNTNFhxcjBxSTN0eFFjVTFobz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=", "encoding": "base64", "item": "/etc/kubernetes/ssl/ca.crt", "source": "/etc/kubernetes/ssl/ca.crt"}
ok: [kubemaster -> None] => (item=/etc/kubernetes/ssl/ca.key) => {"ansible_loop_var": "item", "changed": false, "content": "LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBd003TktiME5QTGl2MVd3WWIxZ2gxNTBzMUM1NWV2MkZDdStWckdFWS95VDM3NVB1CmFkempvU2VCbUVjZHAvTm9EckxYQzlEYThjay9TZytacit2NEdMaUNsdDYzb3phWDg3Uzh6SytCSzBTbGx1UlgKNWJNRFhEa2hMdmd6NVlMcE43NE5Gcm5oK1VGd09PRk1mQXNTWHdLNWpUeXFvQ2dlV1U3Q1pNQjVLMlBJanpWUQpldkFuQmtCNGpncmpnY0tOVHM1ZkFPc2FjZ2xoNVc0WkpQUXhZQnh0VGNyb24xWDZGbmkwMzlia2ZvT3cyd0N6Cno4WUZrcS9nRnlZQ3R3Y0ZUZjdKR25jbnF2RmpXa3VUSTg5ZVhxRzFLM0hzSEU3b0dya3h6TVhoSnlUaHIrTkcKaXVocWxoSVVIS0NxdUEvSjZJbWRPNkdGeENjK0syVTUzbmZpOVFJREFRQUJBb0lCQVFDTEtIZHhhWTZWUmVNNgowYkx0SWhvQzZlRkNIMnowZE9oQ1ZWdWN0OVAzeXZIY0M5UW11aFBjNjR1Z29hdG8zY2c3MjU1VVcrQzZZL3IyCjlEMkRlSGR2T0RTZ3VZeW5FeThtNlVyZkhNakl0dG9oQnh2ZG9JK0xUM3lBendsOTNEZUJVUDFYYXNBTFM4eTQKY1B2b0I0R3g2MkNLN0w5bHFxOEUzRkIzc1dlREJWZ3dtdjZJK0RQRzZxNC8xd0RaME52QjNmMFl3aUdVUys1TgpjYVF4ZmRnODZyanJIdjRlNmthOUgwby9Bc2NKVmllYTArRHJpR0ZCcitVbnRmbVhZNGN6eVdIMXlrR0ZjTE54ClQzMGlGdlFtRzJwM0FsOG1NYzJzMUI0VlJTaGJ3WG1zT0hnV0Z6M0RKUTcrT1ViSFZvUkFlbStFbEdCajlUNmYKOXpVYU5HSUJBb0dCQU9EMmJVWnhMSE9wMmdnMGtIeWZqdFEvSzFwS21oMm9GZmJ1Vnc5dUxiU1hEclVsMzQ0NgpsNUpkZE01c2MyK3RiT294Sk9WQzJnSmd4WFNQVmpDdG1SM2pIOW1xVmtkVzNWSnJLWHFEUXpQTnM5ZDNJR0RtCkUzWVh4RllKM2ZoUlAyOFIrWkVscTQ4RmlobkppdG5iNU9INkFja1kzNGpkeHozTnExc05RYmgxQW9HQkFOdG8KcnNJZ2k2enFGOVpjTzBhNXdOTkhiVFNqV3pzWFdPTVBDSnh5aUlxZFhJMjRXcE9ydU5ENGVQYUVqT0FPenQxNQp6cFMyeTBsUEdrT242OTYzTE1LYU1taUJ2bWxSSTIwc1hIYThZRlpnakFMazgvWWtXY2xjTlphVXlFTGpnNTd1CjQ0S1B3UGdRRjA2WEZQemZxdXYydmhSOTFYNVJtNXp6QVR3T05EQ0JBb0dCQU5jd29vTDVpMDN3c2RHYmhvTFAKbGk3TlpranVoOUhLa1lndG9UWHZiT1VNZ2xNaHNXRTZCdmRKVW95TERUbE1xWnl6K2RrY1E3SjJveXJHV2VvaQpoUThqemJuR3lOVmc4RUVBcFNLK0lUQ1NtbStIVW4rZVZndG0zTFo0cGMzODJqTUMvTG10VVJoMGNoQU16RGR0CnN0bSs3cXBRa3JNM2dubEtIWm54YW1acEFvR0FLTDh4TFZZeittSGpHV2s1alBIMFVJaUVUSEZHQVJ4ZDRwVXAKS1R6UUJNWWlxU01pbHlnZUE1a1JoZDZwOXZacERwS1BmZWpRcWdXcHdBVFFkaVVZaCs5eEN4MkZTalBLVzMwUQprdkI3QjYrWWNQVXFKWDBzdTR1SFhEYTNWa3pBWmVUQmxiNnl2SjdvWlMwRHRjNm9CSGhuWnBEVDlFaWRtTEU4CmVuVzgxWUVDZ1lCV0FKbHdFNlBkTUoxK21IYWhwQ2h0TFJvSFhSZElyNmlLbUJmZm9WVlU1RHdVN2VLY1lIanUKZkYxazIwOGpIT0c2dHB5eEF2blQvYlJuN2VhUlY3aXBEUlBqblVySXlPaWdqdWNoK1JFc0MyT2hnTFZsc1RxcgprYnppalRIbmdrd0ltNm1jMXc2V3pnMGtVT1pmZnFTSjFxL1JCL0FUamJpVVNyL2xOR0V5cFE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=", "encoding": "base64", "item": "/etc/kubernetes/ssl/ca.key", "source": "/etc/kubernetes/ssl/ca.key"}
ok: [kubemaster -> None] => (item=/etc/kubernetes/ssl/front-proxy-ca.crt) => {"ansible_loop_var": "item", "changed": false, "content": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMwRENDQWJpZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFaTVJjd0ZRWURWUVFERXc1bWNtOXUKZEMxd2NtOTRlUzFqWVRBZUZ3MHhPVEV3TVRJeE5qVTVOVGhhRncweU9URXdNRGt4TmpVNU5UaGFNQmt4RnpBVgpCZ05WQkFNVERtWnliMjUwTFhCeWIzaDVMV05oTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCCkNnS0NBUUVBeGVHTnkyYjNHMUJXdk9VZFdSTm1EWXV6amlHWnB3RHZsbjVJOGw2TmkzQUpwTTd6TTk5Ym82TkYKN3FBNXNiY095dTc1T1dMV2orU2JPekF2TkZXOWV3aUgxZldqNVNaNUlLRTRhRjlwR0ZqMXhCa3VObmRES2cyTApSdXBiUEJ5U25nMDJJUmFnVnRPaEFYd01GZXhPVnAwQ2FQekdrNnlUcTRJZXRJc3Zpcmk5Zmd3dkNzV3l3dXcrCjJPbDNEcnA0M3RMYzFWRmx6L0JrL0prdDAraE8zMXZuci9VM1Q4QndVNXBBcmF4cDhQRnByeFVLSjROVnN1RUsKc1pNZWxIV25OSk5PRHB5OUxZZUZldjByMkhCTjhZMFM4d0FHZmk5TGdmczJBK002MStzWnZBVWFsVmFkR1FVMApic0J5VlVBblh5eXM4TDhhRmI3eHU3NnNMSHRZVVFJREFRQUJveU13SVRBT0JnTlZIUThCQWY4RUJBTUNBcVF3CkR3WURWUjBUQVFIL0JBVXdBd0VCL3pBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQVZyQVBJSHFjTzdKM0RFWUoKUnlsb2Nmb2JSNmhsTmVZU1lHOHpqaEp6R2dxc2M4Z1k2U3NFMkk0cVdrTlJFaytlQ2R0dGNTeGdVU0hOUzF2bwo2NEJqRVNhc2J6ZmpyN0JkTjBCNHVEL1k3RDRxNXhlcDIrN2lmQTRORVFuOVcvTjJoamdiWmNjYmwwMkQ3ZDBhCmFibkxndGZwR25xVVFtNkFkU051SktvMmMxMVpRQzlLQW9rZmlaWHVVUStNSlc3UGVJT0xhNVZ6aGtRS0VrdHoKaU5mYVZUdHQwU21NL3l5bjJCZlVIR2dRVC9yMXlxSkcvVlBTSWVsWkdSSXVHbEdmODd4VFNhQU9lUnI0azdLRApNeTVuSWZiL0FNb3RFS1ZvejU2YzJSQytlWWVWSVBES3QrSk1pbTZHem5XVlRCdWpwU3BBOUZ3ZzgyVVd6YXk1Clc5QUlTUT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K", "encoding": "base64", "item": "/etc/kubernetes/ssl/front-proxy-ca.crt", "source": "/etc/kubernetes/ssl/front-proxy-ca.crt"}
ok: [kubemaster -> None] => (item=/etc/kubernetes/ssl/front-proxy-ca.key) => {"ansible_loop_var": "item", "changed": false, "content": "LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcFFJQkFBS0NBUUVBeGVHTnkyYjNHMUJXdk9VZFdSTm1EWXV6amlHWnB3RHZsbjVJOGw2TmkzQUpwTTd6Ck05OWJvNk5GN3FBNXNiY095dTc1T1dMV2orU2JPekF2TkZXOWV3aUgxZldqNVNaNUlLRTRhRjlwR0ZqMXhCa3UKTm5kREtnMkxSdXBiUEJ5U25nMDJJUmFnVnRPaEFYd01GZXhPVnAwQ2FQekdrNnlUcTRJZXRJc3Zpcmk5Zmd3dgpDc1d5d3V3KzJPbDNEcnA0M3RMYzFWRmx6L0JrL0prdDAraE8zMXZuci9VM1Q4QndVNXBBcmF4cDhQRnByeFVLCko0TlZzdUVLc1pNZWxIV25OSk5PRHB5OUxZZUZldjByMkhCTjhZMFM4d0FHZmk5TGdmczJBK002MStzWnZBVWEKbFZhZEdRVTBic0J5VlVBblh5eXM4TDhhRmI3eHU3NnNMSHRZVVFJREFRQUJBb0lCQUFKblJ1MmRob3JTWXZ0UQpYUmJqNFcxd3NQR2MvdHpFYWlTTnpCMEZnUHFjbGNGbXNoV0ROUDI3MGZHZ0NKTmNBZFg5S1JIRlhZM0ozUWpQCkFRUEZ0dE56dXhLaU5TZG5YRGxGUEFZb2dvSkZNbGI3WjFCS0Nvck1OMURaTDlwL0RsalZHRDgyb3lQcTFoM0sKZitzVVA2OVZvTjNkbEJ2ZkVMd1lqSDdzNGRyeGRmNkVnQkFOZ2wzTDBMUDJ2eXcvbzAwNTJtVVNuUEhPNnp1NQpzc2YxOXZJV2RPR1d0Vk41Vnc5UVlhMy9DSGZIR1FtT3RKSDdtR3FOV0tQaWJKRzJJRU9rL1VLWFpNQ2dzcjNlCm0vV2xCbmQ5Rm1WcTdhVXp4ek5GYjNJalRVSVk0UGJpcDd0OGhWdlZ3OVRPVytoM0kzZjBoV2J0V2J1UUtYY3cKNUQ4K1FFRUNnWUVBemVSWTlrNjF2RkpKRWpXSDk1RmJyVEdSZ3VyK2FZWEJXZ25rcnFKZ1ZEY1Z4eFVIUGJ5NApNdnEvVStVb3lKblpWQmZYWjhxR0R6NDY5M3l1a0ZXODVKZWFpYnpMZHZsY2tlUDdBY0xOcjZjN2MyMWRSTlJlCmpGZTRBOEtNWVJLWE1nanBCNHFHcHpaWHRMejFQRTVxREgxSUhMRWlQVnlDZ1dQZkFBVE94bDBDZ1lFQTlnb2EKbDhDejJSTlFDZm0vcjJrcmU2UGcvYU0vQ2s0V1EveXlaNEVNMVZWOXhvcFdobUZLVWJnN0ZuaUdzbkkvczV4NApJRFpYMXlIQUVEenBqak9RUHRQaW0wbmpmc3pKWXE4RUppVEJkUFdPeEFYYVg5bzk5c21UWGlHVXl5dU9HS2p3Cng3MkxnZjM1Z1hERWpKaUNnaGFqeml5M3FrSkdaa0tJWURaTzBvVUNnWUVBazFBOW9zWUFYQ1VYNlRKcC9XV0EKamJQNHBBNVZSOVVPMmNoWVdlY09PMU5LZnEzNkIxOC9BQ0xPd0UwbldCaFVHa29Rc2srQ2llZUJzMC8vYW9BZAo3Z1RXNnBRdGxBdXBBeTEyaS9VQXpCeEFlajhXeThlVGVxRE1wY3dPT3VwZFNjeDhSd3dEZWlKMHNIbStrWTZkCmJDQTdkMlA1MXFPWk1ORHZYb2tTRlhVQ2dZRUE5RGY3Ulg2R0JvYmcvOFhqdXgzTmg2Q1hoNnd1dXUvZnNnUjIKb2I2N0dsY056YkVyU1ovd3dVVDVnY2lKQ3ovVWMxaHc5S2g4Y0pLVHJvcUgyaHJjcDBCeXdtbUpZeWVyRGtWaQovT21uRFkxZzlMcEtsTEp3cHpKWEVTeU1JVTlPY0xQZ3JLTThndit5VmdXMVUwbGU0Zm5QQnF2a2N4L3QrVHM1Cld5Y00vbkVDZ1lFQW1JWUs0YjVVSXUxOWJyMWQ0aEhRbndWT1p6azE4ZDdUSStReUlWSzZXOWU3K0ZTNzdYR3MKZXpaeVRyNytNNEdiYTFIaDczNUxPUkM3cDFNWURQRUdMc1BRRzhuTUZjaUt0YTA0N081U1p0UFVBZlE5RjlGeQpzdlNnb1dHUjVxZ0ZmR24zWDVlbVhWaVI3VERJK3hIWUxXUGZCdlZzRjRZVC9QdEx0UWFneUQ4PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=", "encoding": "base64", "item": "/etc/kubernetes/ssl/front-proxy-ca.key", "source": "/etc/kubernetes/ssl/front-proxy-ca.key"}
ok: [kubemaster -> None] => (item=/etc/kubernetes/ssl/front-proxy-client.crt) => {"ansible_loop_var": "item", "changed": false, "content": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMzekNDQWNlZ0F3SUJBZ0lJVUFjc3VrY1lpRVl3RFFZSktvWklodmNOQVFFTEJRQXdHVEVYTUJVR0ExVUUKQXhNT1puSnZiblF0Y0hKdmVIa3RZMkV3SGhjTk1Ua3hNREV5TVRZMU9UVTRXaGNOTWpBeE1ERXhNVFkxT1RVNApXakFkTVJzd0dRWURWUVFERXhKbWNtOXVkQzF3Y205NGVTMWpiR2xsYm5Rd2dnRWlNQTBHQ1NxR1NJYjNEUUVCCkFRVUFBNElCRHdBd2dnRUtBb0lCQVFEYmZLSGRjS3dNSEZlZXJiSWdIZ05OOGdZaktremFIbWFWUjh6cEY2bFYKaFBXdUNsVWFnalVxRHVKK2tpUWJrbjV0QVZtMmJvcUMwb1k1bUx4bUYyb1hJVjZ6WFNLUWZRUnRTOTZlNmNYagpjby9KU2h2Z2c3dXdIT2diYVhOSWQwT0kwYlVHbkhnZmhnUHpBVEM5dTc2RTdMemFOb3p0M0QrbTJSaW5saWMwCm42cmt3OEFXNllGUHJXSEkzZ3FVci9uUDJIcFcxUG5lNVBBKytyNks2SmVFQm5Vd0kwbTU3N0llWW9NU3B6THkKclJ2bmEzeTQ1Zzk3WnBUemdzVG41L1dxUDFkQkp1cE41MjBQdGFYSGVKU2x0ZWdGKzgzWVdUVFlpcUpBRytWMQpPeTR6Q1prQm5kSkgzOUhNS0w1K09WQ2lrU3ZkTFdoQTU5YXdFOGdPVnBCSEFnTUJBQUdqSnpBbE1BNEdBMVVkCkR3RUIvd1FFQXdJRm9EQVRCZ05WSFNVRUREQUtCZ2dyQmdFRkJRY0RBakFOQmdrcWhraUc5dzBCQVFzRkFBT0MKQVFFQVN5YUQ4R01NOG5vaUtYYmNYTmxqb1ZSTXdUK3J5aHFZeGpiZVNhY0NwaXIzWjU3MSt3eEdReWo4Tm9ZUgpackY5Q3RhODZtN2V3OWpKTGdJSjFydXI0cnZRSytyU3lUYjBhbGJkdUptSlVYZlNHREg3Z05PaFViRUU4Q0VmClBNVnZGUFJOdmIvVkhyeitId0NRSTc5WUJJVkpHQVF1eUdCN1hZd0lGa1hQM2F6dk9aMkd6NmRBRExGVTZkOUwKV2V6TkRsWER5WFRIZW5KSloxOVVPc0ttL0FPN1BGb2pHeXVCTzd1OTVVa3huQ3ZKSDF0bXdMYndTYTZTZU5mYwoxY2NmWFNHaytLY0xLMElFOGdiRGZ4NnJhMk9UUkYwV2RHbGV3SDVVanhlR2JmdmtIeWszYzZCbXZwRlhYR0JhCjBzdituQXVVZ2RXdldiLy9KNXRxbEhYTm13PT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=", "encoding": "base64", "item": "/etc/kubernetes/ssl/front-proxy-client.crt", "source": "/etc/kubernetes/ssl/front-proxy-client.crt"}
ok: [kubemaster -> None] => (item=/etc/kubernetes/ssl/front-proxy-client.key) => {"ansible_loop_var": "item", "changed": false, "content": "LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBMjN5aDNYQ3NEQnhYbnEyeUlCNERUZklHSXlwTTJoNW1sVWZNNlJlcFZZVDFyZ3BWCkdvSTFLZzdpZnBJa0c1SitiUUZadG02S2d0S0dPWmk4WmhkcUZ5RmVzMTBpa0gwRWJVdmVudW5GNDNLUHlVb2IKNElPN3NCem9HMmx6U0hkRGlORzFCcHg0SDRZRDh3RXd2YnUraE95ODJqYU03ZHcvcHRrWXA1WW5OSitxNU1QQQpGdW1CVDYxaHlONEtsSy81ejloNlZ0VDUzdVR3UHZxK2l1aVhoQVoxTUNOSnVlK3lIbUtERXFjeThxMGI1MnQ4CnVPWVBlMmFVODRMRTUrZjFxajlYUVNicVRlZHREN1dseDNpVXBiWG9CZnZOMkZrMDJJcWlRQnZsZFRzdU13bVoKQVozU1I5L1J6Q2krZmpsUW9wRXIzUzFvUU9mV3NCUElEbGFRUndJREFRQUJBb0lCQVFDZWZuQndkTFY0aFZycwpWZWVqODZoVk4ra2dxYUlYZkE1OXkveDNjY093NHJONmV6R2JVOTFLUXJkaHlPeFVFTVJFdU0zc3hBUTNsZ2loClBOOEhKZTlSUnFLWkNTODFnV0daLzBpQ0hyUHNtOFNqQ1dYNkNZYVdGVG15MmNJSzJUeW1aekMrMW15VDd2V1oKSEFuemFNaWFPazNoa2JUUTZ0UDRDeHNYb1lFc0tLM0dHTHV1SHA1eE43c0ZyaFlobFdjSFl5OHYrKzlaUDdnZApBUllFQ2VabHdmL3JhcEpxZThCU2VTeFNpUUpXbzZZT3FTZWxnbDNWZmZqNDdoM1hQUUM0UjZvZ2FNMGJXeFVyCk9HZCtBbVoyVmdXVWRqUy9NTEN6NGh3djVaWmgyekllSlV0NXdlN0pzaW4yOG9HTkt2UmROenpnWW5jc04zMzgKUnRUQmtqVUpBb0dCQU9JQXRUYmFQcEFFRmZramlTRmw1clhpdGJrSXpwcVQrNGxBVHRsWlNCK2dzV2l4RmZGVwpLRFdLL2grdzIzMVJ5TFZveGdoSjl5TFc3SzVwSkRxUUVWeHRmakdVRzZoUDRWTmtRd1kwOWRBbndmS2pQUmZ5CmZHc05kZDBhbEhWYzN3VXhqWWNoMno3czY1V2NKUFVqQzFEcGdHU2U1eCtlYzhJclh2cDYyUnRsQW9HQkFQaWUKaFlYRDAxSWFVN3I5cnREd2lZelJ0QnBIR2FaSTNkbnVEWDNiZTRmU0VERzdmUjJDRTQ5amNNVjlFQkE2SVR4MAo0WldHUTVSbVQvYmZQS2c2YUJSTXVmTlJ4WUdTMUgrbGFjVUtmMU1aL1crdkxsVWE5WE5IRU05ZXpoYXM0Z0sxCjlqU1RPY0lPZ2pUc0VKSlpodTZaNmptWVBJblRORzAyVXpOMFpFQTdBb0dBRTQ1cUgrZjh0K3VTbHk5ckExZlYKQWZJdk5XYThzZ1dTSTR0TUIyVEJaL000eHMxa0Q3RWI0NTQyWlRtaGxKT0tJYUNDV2w5RUltQ2t0R2JtdXFhUwp5Z096Z3NqRkoraFpvQ1N4NUF5YkpMcjcvbW4vdEYzSVJGOHl6c0lwRElyUHhaeEhHZVZsR21uWDdPZEpEMmxMCjVtRitPYmJDV3lzVDZ6NWpocHN3eEQwQ2dZQlJyOXdWS0ZOZGpaTG5SSVRVMThPS0JqalR6WHFxL1JRUGo3elYKRm5GUEhnRCtKaWtrc2VGMVltbmlLWVpHbnFMZTkxUkxTYk1ZcGtrdHh3bHFDTTY3cEw0UUtWVG9qdkxOQWU5dgpzMVJFWEt5OXJtUmR5QjdQZ3BFT1lZcjdQbENBTFlNNG9Rc0FMTDR4WDBMT295WEhiY0RjcloxdWQzdXBTb2twCnhkdE9rd0tCZ1FET2p6S1FyNXpPbHAwY0lmSzVzalZKU0UzOThtRWVEVFdBWThuWUxzM1VaVzJJZnJPVVJFY2cKU1hXenpQSTc1Z1E1U3pqM3BzZTVXZDl4UjROM2VMdzBnZzR0N0xwek9iL0FVYVN0cG5ManJMOTlKYUEvKzlOQgpJcFI1NXY3NFJjamFtbjBhOEloNzUyR2JmNjI2L29rRlhMVU5wWXhrQXlrcDBJdzZRQjFNYnc9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=", "encoding": "base64", "item": "/etc/kubernetes/ssl/front-proxy-client.key", "source": "/etc/kubernetes/ssl/front-proxy-client.key"}
ok: [kubemaster -> None] => (item=/etc/kubernetes/ssl/sa.key) => {"ansible_loop_var": "item", "changed": false, "content": "LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBc1AxMlUzTHVlU0pINkdRNDNxaXYzSlJpUUdrVElKbmhpcS8zL2RtQ0psVEdkaWx1CjErVUQzSUFOVFpMclg0ZER3TmtMOERwOE12THZUdHFmQktYNkhnbFhGRkZUV0tQVWRINXYxOFlkZTRyMFYyNXcKano0MjJETk1OcVd5OUdFRXFqQ2twcHlwUXdreUp4U2RseU1HNnFjbjZWTTRGcXVVcCtGSHR2aFNpTE5RWndFUwpLcXpXUEkvWTV4cVZnVHlweGpiUGJ6OER4bURRTDZneHYxWERuWFY0dnhKbVFKVmU0dXRVT3RRRXJtQWlNZHJtCnFIUVMvME45TmJQL3djNWlGbE9WSXozVDlxcVhaZ242SnNMNVFkejBlTW81a0RyaXBjaENPeWNveFdWMllNN3MKWXdIbSttQ0swdHRpMzN0RTBRWEw3WitJS2dOek5kNXhsQlJ2ZXdJREFRQUJBb0lCQUdjVVo0cTRnRTFLZXVqTgpIaFlIMXN5NEd2VDZmV3BhVlBKcUJCdnpSd3ErQ3d3Y20zTGY3YnpKVTNZZm9EZjExcThGZWgrQ1lIMW40WnBpClY0VGtpZVJlZWpuYUY1bWphUVFqODAyUEUvSzAvaFBCdllxZnFGK0ltTldtOGEvRXBXdmRncmVLdkVYUG9FY0UKaWJmUmJhRFNrN3NmZmFZdWJoak1hdkVVZFRVcGlmRktQQ2k2Um1jTEN5UFVwWUFUa1FoOHQzcm1BY1NqTVZGOQozWW5lSkNnN01CTWhHN0xVcUwyVnBsVllWUDVqMXRWcVdBQWJCVWlXV0JPaWFzWjhtYTRSUFRKL2NRZUl6NVI2CmJMTEhiQ3NFOEtmamZ0QjBOelkrN2xoSU5DN3lLemhzL3QrRkdRTSs5NFE0NHV4eEVVb0ZDNjBJTDllc3MzT2kKemZUWmVta0NnWUVBM3N1N3F2ZWlWSWRkZWRwNk9BbDAxTkdqZXBKcnFNUXVSWlNBYU9MSUo4S3BwQzJheDV2NApEbmhjQTZneG0yRW5vQVIvNCt6clFrVDZMWlB0Rm1zK283VjRhK1kzbmdJK09JQm9CaXdzSGQxekh1bXB4NDU0Cm9DTEN5cE1FWE44UlBtSXJzWDQ5eXZSRlk2UFBXUCtUN09xVXJHVmF2bThFSHJVMCtFV3RIZGNDZ1lFQXkxNGQKaUp0MXFldkJYUlFMdXBBblIxUm9NVFE0ci9MV3NHM1Z1eFpOZlZYVHpZenlEUGxMZkJjM1FrZTUzUTJiRitqWAptTzBvMUNNeGVnSEtraDB6V25ySGZOWWJpUmhYSjlnSm1HZ1p3RjNoUzRYdjV1ZGNIVDZhSnJjUm1uRHllZThkCnUwMTU1STMycGZ3OUd5bkJNaGVXc1BKUWNGMzgwaUVnSm9ZZlh2MENnWUVBMFpNejVZOHIwL2ZCcG00dHFhaTcKamJYbDRkYjZEcmI4RDlleE12aWc3MFhkN2VJNFJxd2dtdnY5RWpPVTltUmRzZGlKdFEyWC82NkM1M3pPK05mSgpCSllmd04rK1VaUXFrMXUyZnBmdXArazM4VXc3dU12OERDYW10WWUyQXVhU2dldzJ6YS9DMExjV3pyTzR5QzhpCkp5cEMwQ2hIUS91S0ZKakFTd3EvM2VFQ2dZQm1BSHVodEErdFBaRkx4UzBvMHZyUWZSYVJiMWJyY1VxYlRVdkIKUGF0NkxOMUNJT3FhOVVNV0ozTFE3TGJkbCtCdGVwMEFtQnM5WTdXQ00zbk9GWlZYVVhQVE9uU3ZLNlV6VjlsZwpOdjZMak1yZW5YdzVmVnVuVTJudjl5eCtJUjZOYUJDcWpHNlI0MTZOM0ZrdDBhQ1VKbDZ2dTZuNjRqbFhqOFBYCmJyZzNaUUtCZ0JMdk5ock5wRmczVXdoRzhNWjB6SjhoZGZMVXVMNFlxZis0Nmk0TGVmRWJBRkZ3RE9BV3JLblkKeVczVjhRamNxaWZnSDBvWEIvVjNtTG9iTUZiYWdXWVE2dGNXUTdJSzNZSERtdEJnQ0VhNEZQUFNCeVV3ZUQvYgpUZ0VoVXlSaWljMjVDWi9wUmZhN1lqWGZhRmZMcE5JWTRGVk94N2ozTzlXRno0Ny9aWk9oCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==", "encoding": "base64", "item": "/etc/kubernetes/ssl/sa.key", "source": "/etc/kubernetes/ssl/sa.key"}
ok: [kubemaster -> None] => (item=/etc/kubernetes/ssl/sa.pub) => {"ansible_loop_var": "item", "changed": false, "content": "LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUFzUDEyVTNMdWVTSkg2R1E0M3FpdgozSlJpUUdrVElKbmhpcS8zL2RtQ0psVEdkaWx1MStVRDNJQU5UWkxyWDRkRHdOa0w4RHA4TXZMdlR0cWZCS1g2CkhnbFhGRkZUV0tQVWRINXYxOFlkZTRyMFYyNXdqejQyMkROTU5xV3k5R0VFcWpDa3BweXBRd2t5SnhTZGx5TUcKNnFjbjZWTTRGcXVVcCtGSHR2aFNpTE5RWndFU0txeldQSS9ZNXhxVmdUeXB4amJQYno4RHhtRFFMNmd4djFYRApuWFY0dnhKbVFKVmU0dXRVT3RRRXJtQWlNZHJtcUhRUy8wTjlOYlAvd2M1aUZsT1ZJejNUOXFxWFpnbjZKc0w1ClFkejBlTW81a0RyaXBjaENPeWNveFdWMllNN3NZd0htK21DSzB0dGkzM3RFMFFYTDdaK0lLZ056TmQ1eGxCUnYKZXdJREFRQUIKLS0tLS1FTkQgUFVCTElDIEtFWS0tLS0tCg==", "encoding": "base64", "item": "/etc/kubernetes/ssl/sa.pub", "source": "/etc/kubernetes/ssl/sa.pub"}
TASK [kubernetes/master : kubeadm | write out kubeadm certs] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:24 +0900 (0:00:02.121) 0:11:25.663 ********
TASK [kubernetes/master : kubeadm | Init other uninitialized masters] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:24 +0900 (0:00:00.357) 0:11:26.020 ********
TASK [kubernetes/master : kubeadm | Upgrade first master] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:24 +0900 (0:00:00.090) 0:11:26.110 ********
TASK [kubernetes/master : kubeadm | Upgrade other masters] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:24 +0900 (0:00:00.086) 0:11:26.197 ********
TASK [kubernetes/master : kubeadm | Check serviceaccount key again] ************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:24 +0900 (0:00:00.091) 0:11:26.288 ********
ok: [kubemaster] => {"changed": false, "stat": {"atime": 1570899618.328, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "f4fabc93b10654796c59860ae8da30182652a4f7", "ctime": 1570899598.834, "dev": 64768, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 4216077, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0600", "mtime": 1570899598.834, "nlink": 1, "path": "/etc/kubernetes/ssl/sa.key", "pw_name": "root", "readable": true, "rgrp": false, "roth": false, "rusr": true, "size": 1675, "uid": 0, "version": "18446744071591654390", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}}
TASK [kubernetes/master : kubeadm | Set secret_changed if service account key was updated] *************************************************************************************************************************************************
Sunday 13 October 2019 02:00:25 +0900 (0:00:00.249) 0:11:26.537 ********
changed: [kubemaster] => {"changed": true, "cmd": ["/bin/true"], "delta": "0:00:00.001707", "end": "2019-10-13 02:00:25.254822", "rc": 0, "start": "2019-10-13 02:00:25.253115", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
TASK [kubernetes/master : kubeadm | Retrieve files to purge] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:25 +0900 (0:00:00.304) 0:11:26.842 ********
TASK [kubernetes/master : kubeadm | Purge old certs] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:25 +0900 (0:00:00.077) 0:11:26.919 ********
TASK [kubernetes/master : kubeadm | Purge old kubeconfig] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:25 +0900 (0:00:00.072) 0:11:26.991 ********
TASK [kubernetes/master : kubeadm | Remove taint for master with node role] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:25 +0900 (0:00:00.127) 0:11:27.119 ********
TASK [kubernetes/master : Include kubeadm etcd extra tasks] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:25 +0900 (0:00:00.083) 0:11:27.202 ********
TASK [kubernetes/master : Include kubeadm secondary server apiserver fixes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:25 +0900 (0:00:00.078) 0:11:27.281 ********
included: /root/kubespray/kubespray/roles/kubernetes/master/tasks/kubeadm-fix-apiserver.yml for kubemaster
TASK [kubernetes/master : Test if correct apiserver is set in all kubeconfigs] *************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:26 +0900 (0:00:00.205) 0:11:27.486 ********
ok: [kubemaster] => {"changed": false, "cmd": "grep -Fq \"https://127.0.0.1:6443\" /etc/kubernetes/admin.conf && grep -Fq \"https://127.0.0.1:6443\" /etc/kubernetes/controller-manager.conf && grep -Fq \"https://127.0.0.1:6443\" /etc/kubernetes/kubelet.conf && grep -Fq \"https://127.0.0.1:6443\" /etc/kubernetes/scheduler.conf", "delta": "0:00:00.003436", "end": "2019-10-13 02:00:26.487110", "failed_when_result": false, "msg": "non-zero return code", "rc": 1, "start": "2019-10-13 02:00:26.483674", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
TASK [kubernetes/master : Create temporary directory] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:26 +0900 (0:00:00.506) 0:11:27.993 ********
changed: [kubemaster] => {"changed": true, "gid": 0, "group": "root", "mode": "0700", "owner": "root", "path": "/tmp/ansible.PPr8Vz", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 6, "state": "directory", "uid": 0}
TASK [kubernetes/master : Generate new kubeconfigs with correct apiserver] *****************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:27 +0900 (0:00:00.533) 0:11:28.527 ********
changed: [kubemaster] => {"changed": true, "cmd": ["/usr/local/bin/kubeadm", "init", "phase", "kubeconfig", "all", "--config", "/etc/kubernetes/kubeadm-config.yaml", "--kubeconfig-dir", "/tmp/ansible.PPr8Vz"], "delta": "0:00:00.972729", "end": "2019-10-13 02:00:28.230269", "rc": 0, "start": "2019-10-13 02:00:27.257540", "stderr": "", "stderr_lines": [], "stdout": "[kubeconfig] Using kubeconfig folder \"/tmp/ansible.PPr8Vz\"\n[kubeconfig] Writing \"admin.conf\" kubeconfig file\n[kubeconfig] Writing \"kubelet.conf\" kubeconfig file\n[kubeconfig] Writing \"controller-manager.conf\" kubeconfig file\n[kubeconfig] Writing \"scheduler.conf\" kubeconfig file", "stdout_lines": ["[kubeconfig] Using kubeconfig folder \"/tmp/ansible.PPr8Vz\"", "[kubeconfig] Writing \"admin.conf\" kubeconfig file", "[kubeconfig] Writing \"kubelet.conf\" kubeconfig file", "[kubeconfig] Writing \"controller-manager.conf\" kubeconfig file", "[kubeconfig] Writing \"scheduler.conf\" kubeconfig file"]}
TASK [kubernetes/master : Copy new kubeconfigs to kube config dir] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:28 +0900 (0:00:01.206) 0:11:29.733 ********
changed: [kubemaster] => (item=admin.conf) => {"ansible_loop_var": "item", "changed": true, "checksum": "886b61952043e69964046a34cb2bad1abbe4fdeb", "dest": "/etc/kubernetes/admin.conf", "gid": 0, "group": "root", "item": "admin.conf", "md5sum": "afa625098e3f3f01bfd4cb12b5293a33", "mode": "0600", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 5466, "src": "/tmp/ansible.PPr8Vz/admin.conf", "state": "file", "uid": 0}
changed: [kubemaster] => (item=controller-manager.conf) => {"ansible_loop_var": "item", "changed": true, "checksum": "8ed44e1345beab39813e40790ef7c47bbd0fe38f", "dest": "/etc/kubernetes/controller-manager.conf", "gid": 0, "group": "root", "item": "controller-manager.conf", "md5sum": "a0b2e4b421a131c4890295d1ec50f20f", "mode": "0600", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 5498, "src": "/tmp/ansible.PPr8Vz/controller-manager.conf", "state": "file", "uid": 0}
changed: [kubemaster] => (item=kubelet.conf) => {"ansible_loop_var": "item", "changed": true, "checksum": "98ef5f4cb7a0aaa63d9f5f3551402d5341771896", "dest": "/etc/kubernetes/kubelet.conf", "gid": 0, "group": "root", "item": "kubelet.conf", "md5sum": "b113b4c8873454256bf40df73511b669", "mode": "0600", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 5494, "src": "/tmp/ansible.PPr8Vz/kubelet.conf", "state": "file", "uid": 0}
changed: [kubemaster] => (item=scheduler.conf) => {"ansible_loop_var": "item", "changed": true, "checksum": "484c02260ffa03fdeb95aafa057f21aa71729b0b", "dest": "/etc/kubernetes/scheduler.conf", "gid": 0, "group": "root", "item": "scheduler.conf", "md5sum": "2aacdfed903aae7936831c5c87a9ad38", "mode": "0600", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 5450, "src": "/tmp/ansible.PPr8Vz/scheduler.conf", "state": "file", "uid": 0}
TASK [kubernetes/master : Cleanup temporary directory] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:29 +0900 (0:00:01.658) 0:11:31.392 ********
changed: [kubemaster] => {"changed": true, "path": "/tmp/ansible.PPr8Vz", "state": "absent"}
TASK [kubernetes/client : Set external kube-apiserver endpoint] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:30 +0900 (0:00:00.225) 0:11:31.617 ********
ok: [kubemaster] => {"ansible_facts": {"external_apiserver_address": "192.168.56.107", "external_apiserver_port": "6443"}, "changed": false}
TASK [kubernetes/client : Create kube config dir for current/ansible become user] **********************************************************************************************************************************************************
Sunday 13 October 2019 02:00:30 +0900 (0:00:00.254) 0:11:31.872 ********
changed: [kubemaster] => {"changed": true, "gid": 0, "group": "root", "mode": "0700", "owner": "root", "path": "/root/.kube", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 51, "state": "directory", "uid": 0}
TASK [kubernetes/client : Copy admin kubeconfig to current/ansible become user home] *******************************************************************************************************************************************************
Sunday 13 October 2019 02:00:30 +0900 (0:00:00.231) 0:11:32.104 ********
changed: [kubemaster] => {"backup_file": "/root/.kube/config.1210.2019-10-13@02:00:30~", "changed": true, "checksum": "886b61952043e69964046a34cb2bad1abbe4fdeb", "dest": "/root/.kube/config", "gid": 0, "group": "root", "md5sum": "afa625098e3f3f01bfd4cb12b5293a33", "mode": "0600", "owner": "root", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 5466, "src": "/etc/kubernetes/admin.conf", "state": "file", "uid": 0}
TASK [kubernetes/client : Create kube artifacts dir] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:30 +0900 (0:00:00.293) 0:11:32.397 ********
TASK [kubernetes/client : Wait for k8s apiserver] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:31 +0900 (0:00:00.073) 0:11:32.470 ********
ok: [kubemaster] => {"changed": false, "elapsed": 0, "match_groupdict": {}, "match_groups": [], "path": null, "port": 6443, "search_regex": null, "state": "started"}
TASK [kubernetes/client : Generate admin kubeconfig with external api endpoint] ************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:31 +0900 (0:00:00.726) 0:11:33.197 ********
[WARNING]: Consider using the file module with state=directory rather than running 'mkdir'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in
ansible.cfg to get rid of this message.
changed: [kubemaster] => {"changed": true, "cmd": "mkdir -p /etc/kubernetes/external_kubeconfig && /usr/local/bin/kubeadm init phase kubeconfig admin --kubeconfig-dir /etc/kubernetes/external_kubeconfig --cert-dir /etc/kubernetes/ssl --apiserver-advertise-address 192.168.56.107 --apiserver-bind-port 6443 >/dev/null && cat /etc/kubernetes/external_kubeconfig/admin.conf && rm -rf /etc/kubernetes/external_kubeconfig", "delta": "0:00:01.085813", "end": "2019-10-13 02:00:33.040166", "rc": 0, "start": "2019-10-13 02:00:31.954353", "stderr": "", "stderr_lines": [], "stdout": "apiVersion: v1\nclusters:\n- cluster:\n certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFNU1UQXhNakUyTlRrMU4xb1hEVEk1TVRBd09URTJOVGsxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTURPCnpTbTlEVHk0cjlWc0dHOVlJZGVkTE5RdWVYcjloUXJ2bGF4aEdQOGs5KytUN21uYzQ2RW5nWmhISGFmemFBNnkKMXd2UTJ2SEpQMG9QbWEvcitCaTRncGJldDZNMmwvTzB2TXl2Z1N0RXBaYmtWK1d6QTF3NUlTNzRNK1dDNlRlKwpEUmE1NGZsQmNEamhUSHdMRWw4Q3VZMDhxcUFvSGxsT3dtVEFlU3RqeUk4MVVIcndKd1pBZUk0SzQ0SENqVTdPClh3RHJHbklKWWVWdUdTVDBNV0FjYlUzSzZKOVYraFo0dE4vVzVINkRzTnNBczgvR0JaS3Y0QmNtQXJjSEJVMysKeVJwM0o2cnhZMXBMa3lQUFhsNmh0U3R4N0J4TzZCcTVNY3pGNFNjazRhL2pSb3JvYXBZU0ZCeWdxcmdQeWVpSgpuVHVoaGNRblBpdGxPZDUzNHZVQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFGemZHc2V2OWk0NzBEMHhhME9jK0M1TURDY3kKRVoxQkQ2SU9ySVYwdWdTWHFQZkZSakU5bWxUYks1YloyaHJOZjlaSFM4d0FDL0g5THZuNHVHMFVSM3lueStEYgpXQXN6cjBxaGFlQ2x6Q0FRdUZzblo2amx4b3JjVUI1OWRPd0xsdW1PRDZpN29yUUg0WkJZa2JIa3dVbG80cFVOClJHMXNWeVZoWW81d1lBZ2xveVAvQlNMclBYbzFLTGtRMDZLdnAzclhpRkpqZUtMQjlCdlVvVzVubjR3aDI0RTIKSFRXU05IaWxCL1FZTlZUNnc3MHlISFJrTUVIL2x0aS9qRnovR0RVTXlNbjNER09sRkNqK002U3o2M0RoMEU4WApWRk1SdXJ5YjhFeUFOb1JFRlpPbFBIclR3V3JtSk9aYkVJMEFRMmJlalNTNFhxcjBxSTN0eFFjVTFobz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=\n server: https://192.168.56.107:6443\n name: kubernetes\ncontexts:\n- context:\n cluster: kubernetes\n user: kubernetes-admin\n name: kubernetes-admin@kubernetes\ncurrent-context: kubernetes-admin@kubernetes\nkind: Config\npreferences: {}\nusers:\n- name: kubernetes-admin\n user:\n client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4akNDQWRxZ0F3SUJBZ0lJUTRsNEh1RnhyY2d3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB4T1RFd01USXhOalU1TlRkYUZ3MHlNREV3TVRFeE56QXdNek5hTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTNrOEpHSGdTNlV3MDNNb0kKc09Ybzhqc2QyQ0RBYnJOQ1JpUk5TTHJwUG5FSS9KazFieHlyVWhBeDdPZnlySG9xbmozK0d4WlpHbGRFMkJmRQpjUHk0K0xxdXpQUDZPbDkvajVqdUUwZUxjUjVYWlFVNTIvNHBqcGFMQlV6RVAzN0g0NFFBQVhoTE5pUklldHlOCmNPQzUzdlhmY2l1RFR2NFhRMmw1UGpaZDBQOFBWTGZMTlJNa3FTTEdQOVRuSmZ1MU54TWRDbEtuTVZHNWg4NlYKOWZpVXFBYWdEQzJXVW54VC80TDB6dHA3OG1IZy9yTjRqVlhGTWRhbWdtV203UGhFVmRYTkpwZUExK3NmdVdkQwpIZmZIZE55K0dYUFQwVldVeHFjQU54MGRGNVBnWitwRUpMQldTYVQxZUpveTZldVc2TVhMc0FMdEZUekVuYW00CmswR2JPd0lEQVFBQm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFDdmliWVJwZEhMeDFtclQwY2lVTEV5elNJWm9vYnZoeXJ3VApTWTFnVWMyTDR6ckpuNTBmcHJFZ2xnaU9tdXFUTE1JU1p0K01CWlR6dUJtNjZ6WWhYRWpvZzRXbUlkUnhKdC9ECmM4LzRzc1VQY3Q5N0I4R1pJRVMxZmRhN3VQRHRnRmp0MjVuZFFrNDRrR1VBSHBRYi9OV2xJdzQvWkVhaHpNY0wKc2hnR0FXWUdtK1QyT29mV0FjNlp6eEFZamRySllURFUxYmxxakt0YnI3dy8vRDdIMmFZU29QNzBRZFE2ekJtKwpEYU13bmdUcTg3SElUNHQ1RnZJS1I5cnJ6eFYvSnRqQ2x2NXRRTEc2MXB6QXhHekNkRlJMN1pzU01nbThVMExuCllnYUZacUdGZ2JaSFlMRTJsWStOYzB2dERiMnZlendxSk5RdFRyY1hLenJmWEZpTjdiMD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=\n client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBM2s4SkdIZ1M2VXcwM01vSXNPWG84anNkMkNEQWJyTkNSaVJOU0xycFBuRUkvSmsxCmJ4eXJVaEF4N09meXJIb3FuajMrR3haWkdsZEUyQmZFY1B5NCtMcXV6UFA2T2w5L2o1anVFMGVMY1I1WFpRVTUKMi80cGpwYUxCVXpFUDM3SDQ0UUFBWGhMTmlSSWV0eU5jT0M1M3ZYZmNpdURUdjRYUTJsNVBqWmQwUDhQVkxmTApOUk1rcVNMR1A5VG5KZnUxTnhNZENsS25NVkc1aDg2VjlmaVVxQWFnREMyV1VueFQvNEwwenRwNzhtSGcvck40CmpWWEZNZGFtZ21XbTdQaEVWZFhOSnBlQTErc2Z1V2RDSGZmSGROeStHWFBUMFZXVXhxY0FOeDBkRjVQZ1orcEUKSkxCV1NhVDFlSm95NmV1VzZNWExzQUx0RlR6RW5hbTRrMEdiT3dJREFRQUJBb0lCQUYxRjJmSWJSd3Y5Zmx4NApwQWJjbjgwcVFvNVF4NHhxeDg2NEluajJlekJJcXRXWXpwT00rdENXbk92aU1hNXVIMzEvVXlNNFVEQStGMU4zClBXT09sV2srb21LNXB1NWc3UnRwdGRIKy9hRjdjbjJmSUxTTndnL2lWL2ZrU3A2OWFjNnFybm52WjljU2lCcU0KZVFlNE1pUTdFd0pMQVcvZUtoS05xV0lSMjNENUdzbGZFTkxzdXFoWmc0SXJJOWtmVk1vSk5xb3dKb2FxUzVYMApRSlQvMzl4aVVrSVRUaGdySGJCS1IrVWRUNWxYUUdYQmprUDJkdVlSTkpBWUFwaTlMbVoyL25RN2RIWUI1NzF5Cm1lNE5SNnA5WDlQbVc1ZUZvYk5RYmVsUlJnQlNYSnVVZGsxV2dKT29rWE5jbmdHMTJSZVZvYTlrWVk2bDBXRUYKWE1nMDRqa0NnWUVBNkF6MkZscWlWdFVMOVhad21TOXBrY3F0UG41QmFmWEE5T3R4RkN6OXNEZFlJT3c2UUhQSAptL2JVQnEvc0h2bHcwVCs2bjlaZnZBeG02RE5GVzRJZFp4WTZwa0ljQ3BXWjVjRzdGSjZ5OXZHUW1vc25LWStICmRmSC91cnN3TkREN2tuSWl2OGZ4emtmUjR5Y1F6WXYxZjFBRzZsQ0o2MkQ1b1NJVDA4dWpzMzhDZ1lFQTlVQ3UKcnlDdUpKUk4waTFHSHZ0ZmlleWQzMVd5c3AzQVpXd0U5bjZGMnpZZzhKTDJYQkRwMHpjMHlnV1lyUFZ6cGNKNQpISmNBZ3NtQjdmSTB1VTZTcWIrTkdvMzBla0lTWXJTSXdqRXNkajZGOWdSL3RUS1NYYmlRU0Z6KzIwL1FlM3l1Cm9kbG9qYUVCNGJyT01iaUlCNm05T0h6RlhsNE1GQkVVanp1eXhrVUNnWUJuZmloZkRlc3JTUGQ5Y1dhRW00VzgKOVNUUkkyUXZTdURWR3hDaXkvSVZIb25hblNEWjBBbHRnT2ROV0RhNkZjd0FOeWtYMG5kK0doWG5aNzUrTW9qRQppa0I4R0M0RGRuZGQ1a3AyeHRUUW1DcERFdDdUaFdVSDhIZDZvN083QTBHZ0xmRmJrV1JCK1B3ZUVEVkpSeXVsCnJWNTRNSlZpZWl5N2RjUmNDSTJwSndLQmdRQ3hucTdrMXQ1SVQ4QXN2KzFROHUwMmpzK251QUU3cTFSMEg4OVkKVmpuTGtXQTUyN1hnNTc3dzRjS3hyV3Z0bDdheURpNTRya1FJZExoTkJHVjY3NVQ0cTJvZUdVRSthaXUyb05tTApDdUlRb1NlVURaNFFOUmdjR0ZjbWtYTmlrbU9EaThGT0pvNjZSQ21BdWpJcmdHSEVhd0pIcnRXaWlxcGt2aG9OCndrYXZvUUtCZ1FDYVBpQ0FZZjRxWlRDeVlzVHRQazZJQjNWSHQ4L3BrdU9lN3V4VTlKaTcvMk5FVGg0WEFuM0YKNmVvTE55NHFhS0gvaXZlQndLRmFDcFlTWVgzbTA2dndlM2lMdFIrQXYrUGx4MFBod21DQnZJbnFTd0JaNjdFMgpDdVJhZlVwamJUVGRpN2tYbnc3WWtFb0ZpUzhJT2Z1aTBYWUoxTnZoanQ0OXB3Tk8rTFBQdUE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=", "stdout_lines": ["apiVersion: v1", "clusters:", "- cluster:", " certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFNU1UQXhNakUyTlRrMU4xb1hEVEk1TVRBd09URTJOVGsxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTURPCnpTbTlEVHk0cjlWc0dHOVlJZGVkTE5RdWVYcjloUXJ2bGF4aEdQOGs5KytUN21uYzQ2RW5nWmhISGFmemFBNnkKMXd2UTJ2SEpQMG9QbWEvcitCaTRncGJldDZNMmwvTzB2TXl2Z1N0RXBaYmtWK1d6QTF3NUlTNzRNK1dDNlRlKwpEUmE1NGZsQmNEamhUSHdMRWw4Q3VZMDhxcUFvSGxsT3dtVEFlU3RqeUk4MVVIcndKd1pBZUk0SzQ0SENqVTdPClh3RHJHbklKWWVWdUdTVDBNV0FjYlUzSzZKOVYraFo0dE4vVzVINkRzTnNBczgvR0JaS3Y0QmNtQXJjSEJVMysKeVJwM0o2cnhZMXBMa3lQUFhsNmh0U3R4N0J4TzZCcTVNY3pGNFNjazRhL2pSb3JvYXBZU0ZCeWdxcmdQeWVpSgpuVHVoaGNRblBpdGxPZDUzNHZVQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFGemZHc2V2OWk0NzBEMHhhME9jK0M1TURDY3kKRVoxQkQ2SU9ySVYwdWdTWHFQZkZSakU5bWxUYks1YloyaHJOZjlaSFM4d0FDL0g5THZuNHVHMFVSM3lueStEYgpXQXN6cjBxaGFlQ2x6Q0FRdUZzblo2amx4b3JjVUI1OWRPd0xsdW1PRDZpN29yUUg0WkJZa2JIa3dVbG80cFVOClJHMXNWeVZoWW81d1lBZ2xveVAvQlNMclBYbzFLTGtRMDZLdnAzclhpRkpqZUtMQjlCdlVvVzVubjR3aDI0RTIKSFRXU05IaWxCL1FZTlZUNnc3MHlISFJrTUVIL2x0aS9qRnovR0RVTXlNbjNER09sRkNqK002U3o2M0RoMEU4WApWRk1SdXJ5YjhFeUFOb1JFRlpPbFBIclR3V3JtSk9aYkVJMEFRMmJlalNTNFhxcjBxSTN0eFFjVTFobz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=", " server: https://192.168.56.107:6443", " name: kubernetes", "contexts:", "- context:", " cluster: kubernetes", " user: kubernetes-admin", " name: kubernetes-admin@kubernetes", "current-context: kubernetes-admin@kubernetes", "kind: Config", "preferences: {}", "users:", "- name: kubernetes-admin", " user:", " client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4akNDQWRxZ0F3SUJBZ0lJUTRsNEh1RnhyY2d3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB4T1RFd01USXhOalU1TlRkYUZ3MHlNREV3TVRFeE56QXdNek5hTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTNrOEpHSGdTNlV3MDNNb0kKc09Ybzhqc2QyQ0RBYnJOQ1JpUk5TTHJwUG5FSS9KazFieHlyVWhBeDdPZnlySG9xbmozK0d4WlpHbGRFMkJmRQpjUHk0K0xxdXpQUDZPbDkvajVqdUUwZUxjUjVYWlFVNTIvNHBqcGFMQlV6RVAzN0g0NFFBQVhoTE5pUklldHlOCmNPQzUzdlhmY2l1RFR2NFhRMmw1UGpaZDBQOFBWTGZMTlJNa3FTTEdQOVRuSmZ1MU54TWRDbEtuTVZHNWg4NlYKOWZpVXFBYWdEQzJXVW54VC80TDB6dHA3OG1IZy9yTjRqVlhGTWRhbWdtV203UGhFVmRYTkpwZUExK3NmdVdkQwpIZmZIZE55K0dYUFQwVldVeHFjQU54MGRGNVBnWitwRUpMQldTYVQxZUpveTZldVc2TVhMc0FMdEZUekVuYW00CmswR2JPd0lEQVFBQm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFDdmliWVJwZEhMeDFtclQwY2lVTEV5elNJWm9vYnZoeXJ3VApTWTFnVWMyTDR6ckpuNTBmcHJFZ2xnaU9tdXFUTE1JU1p0K01CWlR6dUJtNjZ6WWhYRWpvZzRXbUlkUnhKdC9ECmM4LzRzc1VQY3Q5N0I4R1pJRVMxZmRhN3VQRHRnRmp0MjVuZFFrNDRrR1VBSHBRYi9OV2xJdzQvWkVhaHpNY0wKc2hnR0FXWUdtK1QyT29mV0FjNlp6eEFZamRySllURFUxYmxxakt0YnI3dy8vRDdIMmFZU29QNzBRZFE2ekJtKwpEYU13bmdUcTg3SElUNHQ1RnZJS1I5cnJ6eFYvSnRqQ2x2NXRRTEc2MXB6QXhHekNkRlJMN1pzU01nbThVMExuCllnYUZacUdGZ2JaSFlMRTJsWStOYzB2dERiMnZlendxSk5RdFRyY1hLenJmWEZpTjdiMD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=", " client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBM2s4SkdIZ1M2VXcwM01vSXNPWG84anNkMkNEQWJyTkNSaVJOU0xycFBuRUkvSmsxCmJ4eXJVaEF4N09meXJIb3FuajMrR3haWkdsZEUyQmZFY1B5NCtMcXV6UFA2T2w5L2o1anVFMGVMY1I1WFpRVTUKMi80cGpwYUxCVXpFUDM3SDQ0UUFBWGhMTmlSSWV0eU5jT0M1M3ZYZmNpdURUdjRYUTJsNVBqWmQwUDhQVkxmTApOUk1rcVNMR1A5VG5KZnUxTnhNZENsS25NVkc1aDg2VjlmaVVxQWFnREMyV1VueFQvNEwwenRwNzhtSGcvck40CmpWWEZNZGFtZ21XbTdQaEVWZFhOSnBlQTErc2Z1V2RDSGZmSGROeStHWFBUMFZXVXhxY0FOeDBkRjVQZ1orcEUKSkxCV1NhVDFlSm95NmV1VzZNWExzQUx0RlR6RW5hbTRrMEdiT3dJREFRQUJBb0lCQUYxRjJmSWJSd3Y5Zmx4NApwQWJjbjgwcVFvNVF4NHhxeDg2NEluajJlekJJcXRXWXpwT00rdENXbk92aU1hNXVIMzEvVXlNNFVEQStGMU4zClBXT09sV2srb21LNXB1NWc3UnRwdGRIKy9hRjdjbjJmSUxTTndnL2lWL2ZrU3A2OWFjNnFybm52WjljU2lCcU0KZVFlNE1pUTdFd0pMQVcvZUtoS05xV0lSMjNENUdzbGZFTkxzdXFoWmc0SXJJOWtmVk1vSk5xb3dKb2FxUzVYMApRSlQvMzl4aVVrSVRUaGdySGJCS1IrVWRUNWxYUUdYQmprUDJkdVlSTkpBWUFwaTlMbVoyL25RN2RIWUI1NzF5Cm1lNE5SNnA5WDlQbVc1ZUZvYk5RYmVsUlJnQlNYSnVVZGsxV2dKT29rWE5jbmdHMTJSZVZvYTlrWVk2bDBXRUYKWE1nMDRqa0NnWUVBNkF6MkZscWlWdFVMOVhad21TOXBrY3F0UG41QmFmWEE5T3R4RkN6OXNEZFlJT3c2UUhQSAptL2JVQnEvc0h2bHcwVCs2bjlaZnZBeG02RE5GVzRJZFp4WTZwa0ljQ3BXWjVjRzdGSjZ5OXZHUW1vc25LWStICmRmSC91cnN3TkREN2tuSWl2OGZ4emtmUjR5Y1F6WXYxZjFBRzZsQ0o2MkQ1b1NJVDA4dWpzMzhDZ1lFQTlVQ3UKcnlDdUpKUk4waTFHSHZ0ZmlleWQzMVd5c3AzQVpXd0U5bjZGMnpZZzhKTDJYQkRwMHpjMHlnV1lyUFZ6cGNKNQpISmNBZ3NtQjdmSTB1VTZTcWIrTkdvMzBla0lTWXJTSXdqRXNkajZGOWdSL3RUS1NYYmlRU0Z6KzIwL1FlM3l1Cm9kbG9qYUVCNGJyT01iaUlCNm05T0h6RlhsNE1GQkVVanp1eXhrVUNnWUJuZmloZkRlc3JTUGQ5Y1dhRW00VzgKOVNUUkkyUXZTdURWR3hDaXkvSVZIb25hblNEWjBBbHRnT2ROV0RhNkZjd0FOeWtYMG5kK0doWG5aNzUrTW9qRQppa0I4R0M0RGRuZGQ1a3AyeHRUUW1DcERFdDdUaFdVSDhIZDZvN083QTBHZ0xmRmJrV1JCK1B3ZUVEVkpSeXVsCnJWNTRNSlZpZWl5N2RjUmNDSTJwSndLQmdRQ3hucTdrMXQ1SVQ4QXN2KzFROHUwMmpzK251QUU3cTFSMEg4OVkKVmpuTGtXQTUyN1hnNTc3dzRjS3hyV3Z0bDdheURpNTRya1FJZExoTkJHVjY3NVQ0cTJvZUdVRSthaXUyb05tTApDdUlRb1NlVURaNFFOUmdjR0ZjbWtYTmlrbU9EaThGT0pvNjZSQ21BdWpJcmdHSEVhd0pIcnRXaWlxcGt2aG9OCndrYXZvUUtCZ1FDYVBpQ0FZZjRxWlRDeVlzVHRQazZJQjNWSHQ4L3BrdU9lN3V4VTlKaTcvMk5FVGg0WEFuM0YKNmVvTE55NHFhS0gvaXZlQndLRmFDcFlTWVgzbTA2dndlM2lMdFIrQXYrUGx4MFBod21DQnZJbnFTd0JaNjdFMgpDdVJhZlVwamJUVGRpN2tYbnc3WWtFb0ZpUzhJT2Z1aTBYWUoxTnZoanQ0OXB3Tk8rTFBQdUE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo="]}
TASK [kubernetes/client : Write admin kubeconfig on ansible host] **************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:33 +0900 (0:00:01.369) 0:11:34.566 ********
TASK [kubernetes/client : Copy kubectl binary to ansible host] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:33 +0900 (0:00:00.079) 0:11:34.646 ********
TASK [kubernetes/client : create helper script kubectl.sh on ansible host] *****************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:33 +0900 (0:00:00.079) 0:11:34.726 ********
TASK [kubernetes-apps/cluster_roles : Kubernetes Apps | Wait for kube-apiserver] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:00:33 +0900 (0:00:00.082) 0:11:34.808 ********
ok: [kubemaster] => {"attempts": 1, "cache_control": "no-cache, private", "changed": false, "connection": "close", "content_length": "2", "content_type": "text/plain; charset=utf-8", "cookies": {}, "cookies_string": "", "date": "Sat, 12 Oct 2019 17:00:33 GMT", "elapsed": 0, "msg": "OK (2 bytes)", "redirected": false, "status": 200, "url": "https://127.0.0.1:6443/healthz", "x_content_type_options": "nosniff"}
TASK [kubernetes-apps/cluster_roles : Kubernetes Apps | Check AppArmor status] *************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:34 +0900 (0:00:00.649) 0:11:35.458 ********
TASK [kubernetes-apps/cluster_roles : Kubernetes Apps | Set apparmor_enabled] **************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:34 +0900 (0:00:00.072) 0:11:35.530 ********
TASK [kubernetes-apps/cluster_roles : Kubernetes Apps | Render templates for PodSecurityPolicy] ********************************************************************************************************************************************
Sunday 13 October 2019 02:00:34 +0900 (0:00:00.070) 0:11:35.600 ********
TASK [kubernetes-apps/cluster_roles : Kubernetes Apps | Add policies, roles, bindings for PodSecurityPolicy] *******************************************************************************************************************************
Sunday 13 October 2019 02:00:34 +0900 (0:00:00.101) 0:11:35.702 ********
TASK [kubernetes-apps/cluster_roles : Kubernetes Apps | Add ClusterRoleBinding to admit nodes] *********************************************************************************************************************************************
Sunday 13 October 2019 02:00:34 +0900 (0:00:00.143) 0:11:35.845 ********
changed: [kubemaster] => {"changed": true, "checksum": "7a5d333673421310e034bd1931c65d490d541632", "dest": "/etc/kubernetes/node-crb.yml", "gid": 0, "group": "root", "md5sum": "63565185c3821bd7eb783657a86d84c9", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 408, "src": "/root/.ansible/tmp/ansible-tmp-1570899634.4519124-150021029360174/source", "state": "file", "uid": 0}
TASK [kubernetes-apps/cluster_roles : Apply workaround to allow all nodes with cert O=system:nodes to register] ****************************************************************************************************************************
Sunday 13 October 2019 02:00:34 +0900 (0:00:00.513) 0:11:36.359 ********
ok: [kubemaster] => {"attempts": 1, "changed": false, "msg": "success: clusterrolebinding.rbac.authorization.k8s.io/kubespray:system:node created"}
TASK [kubernetes-apps/cluster_roles : Kubernetes Apps | Add webhook ClusterRole that grants access to proxy, stats, log, spec, and metrics on a kubelet] ***********************************************************************************
Sunday 13 October 2019 02:00:35 +0900 (0:00:01.051) 0:11:37.411 ********
changed: [kubemaster] => {"changed": true, "checksum": "3ae0c3673b72c2b21c807ff3f30e5d3ba7686e13", "dest": "/etc/kubernetes/node-webhook-cr.yml", "gid": 0, "group": "root", "md5sum": "43fcdedeedcf4cc2671178b1ef7ba5bf", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 403, "src": "/root/.ansible/tmp/ansible-tmp-1570899636.0169783-94751982757242/source", "state": "file", "uid": 0}
TASK [kubernetes-apps/cluster_roles : Apply webhook ClusterRole] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:36 +0900 (0:00:00.502) 0:11:37.914 ********
ok: [kubemaster] => {"changed": false, "msg": "success: clusterrole.rbac.authorization.k8s.io/system:node-webhook created"}
TASK [kubernetes-apps/cluster_roles : Kubernetes Apps | Add ClusterRoleBinding for system:nodes to webhook ClusterRole] ********************************************************************************************************************
Sunday 13 October 2019 02:00:36 +0900 (0:00:00.439) 0:11:38.353 ********
changed: [kubemaster] => {"changed": true, "checksum": "629bb96860047ab61d6d966a0d77412a8093baad", "dest": "/etc/kubernetes/node-webhook-crb.yml", "gid": 0, "group": "root", "md5sum": "33a0a30140c67a8fdd4678d19b7029c5", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 414, "src": "/root/.ansible/tmp/ansible-tmp-1570899636.969988-129740347956921/source", "state": "file", "uid": 0}
TASK [kubernetes-apps/cluster_roles : Grant system:nodes the webhook ClusterRole] **********************************************************************************************************************************************************
Sunday 13 October 2019 02:00:37 +0900 (0:00:00.544) 0:11:38.898 ********
ok: [kubemaster] => {"changed": false, "msg": "success: clusterrolebinding.rbac.authorization.k8s.io/system:node-webhook created"}
TASK [kubernetes-apps/cluster_roles : include_tasks] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:37 +0900 (0:00:00.466) 0:11:39.365 ********
TASK [kubernetes-apps/cluster_roles : PriorityClass | Copy k8s-cluster-critical-pc.yml file] ***********************************************************************************************************************************************
Sunday 13 October 2019 02:00:38 +0900 (0:00:00.086) 0:11:39.451 ********
changed: [kubemaster] => {"changed": true, "checksum": "2abb3233dd0241e931c969f251bdcd54183b1f88", "dest": "/etc/kubernetes/k8s-cluster-critical-pc.yml", "gid": 0, "group": "root", "md5sum": "71b709e503a59f773fe8f097e394325d", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 235, "src": "/root/.ansible/tmp/ansible-tmp-1570899638.0532534-206360080400976/source", "state": "file", "uid": 0}
TASK [kubernetes-apps/cluster_roles : PriorityClass | Create k8s-cluster-critical] *********************************************************************************************************************************************************
Sunday 13 October 2019 02:00:38 +0900 (0:00:00.468) 0:11:39.920 ********
ok: [kubemaster] => {"changed": false, "msg": "success: priorityclass.scheduling.k8s.io/k8s-cluster-critical created"}
RUNNING HANDLER [kubernetes/master : Master | restart kubelet] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:38 +0900 (0:00:00.398) 0:11:40.319 ********
changed: [kubemaster] => {"changed": true, "cmd": ["/bin/true"], "delta": "0:00:00.002047", "end": "2019-10-13 02:00:39.085028", "rc": 0, "start": "2019-10-13 02:00:39.082981", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
RUNNING HANDLER [kubernetes/master : Master | wait for master static pods] *****************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:39 +0900 (0:00:00.381) 0:11:40.701 ********
changed: [kubemaster] => {"changed": true, "cmd": ["/bin/true"], "delta": "0:00:00.002126", "end": "2019-10-13 02:00:39.471130", "rc": 0, "start": "2019-10-13 02:00:39.469004", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
RUNNING HANDLER [kubernetes/master : Master | Restart kube-scheduler] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:39 +0900 (0:00:00.395) 0:11:41.096 ********
changed: [kubemaster] => {"changed": true, "cmd": ["/bin/true"], "delta": "0:00:00.010120", "end": "2019-10-13 02:00:39.932575", "rc": 0, "start": "2019-10-13 02:00:39.922455", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
RUNNING HANDLER [kubernetes/master : Master | Restart kube-controller-manager] *************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:40 +0900 (0:00:00.383) 0:11:41.480 ********
changed: [kubemaster] => {"changed": true, "cmd": ["/bin/true"], "delta": "0:00:00.001814", "end": "2019-10-13 02:00:40.242142", "rc": 0, "start": "2019-10-13 02:00:40.240328", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
RUNNING HANDLER [kubernetes/master : Master | reload systemd] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:40 +0900 (0:00:00.313) 0:11:41.793 ********
ok: [kubemaster] => {"changed": false, "name": null, "status": {}}
RUNNING HANDLER [kubernetes/master : Master | reload kubelet] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:40 +0900 (0:00:00.514) 0:11:42.307 ********
changed: [kubemaster] => {"changed": true, "name": "kubelet", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-10-13 02:00:08 KST", "ActiveEnterTimestampMonotonic": "21949046149", "ActiveExitTimestamp": "Sun 2019-10-13 01:59:57 KST", "ActiveExitTimestampMonotonic": "21938938588", "ActiveState": "active", "After": "system.slice systemd-journald.socket basic.target docker.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-10-13 02:00:08 KST", "AssertTimestampMonotonic": "21949040042", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-10-13 02:00:08 KST", "ConditionTimestampMonotonic": "21949040042", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/kubelet.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Kubernetes Kubelet Server", "DevicePolicy": "auto", "Documentation": "https://github.com/GoogleCloudPlatform/kubernetes", "DropInPaths": "/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf", "Environment": "KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml", "EnvironmentFile": "/etc/sysconfig/kubelet (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "477", "ExecMainStartTimestamp": "Sun 2019-10-13 02:00:08 KST", "ExecMainStartTimestampMonotonic": "21949046097", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/bin/kubelet ; argv[]=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/etc/systemd/system/kubelet.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "kubelet.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2019-10-13 02:00:08 KST", "InactiveEnterTimestampMonotonic": "21949039994", "InactiveExitTimestamp": "Sun 2019-10-13 02:00:08 KST", "InactiveExitTimestampMonotonic": "21949046149", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "7259", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7259", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "477", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "kubelet.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "system.slice basic.target", "Restart": "always", "RestartUSec": "10s", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "root", "WantedBy": "multi-user.target", "Wants": "docker.socket", "WatchdogTimestamp": "Sun 2019-10-13 02:00:08 KST", "WatchdogTimestampMonotonic": "21949046135", "WatchdogUSec": "0"}}
RUNNING HANDLER [kubernetes/master : Master | Remove scheduler container] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:41 +0900 (0:00:00.450) 0:11:42.758 ********
changed: [kubemaster] => {"attempts": 1, "changed": true, "cmd": "docker ps -af name=k8s_kube-scheduler* -q | xargs --no-run-if-empty docker rm -f", "delta": "0:00:00.375573", "end": "2019-10-13 02:00:41.917406", "rc": 0, "start": "2019-10-13 02:00:41.541833", "stderr": "", "stderr_lines": [], "stdout": "bbe35ba92914", "stdout_lines": ["bbe35ba92914"]}
RUNNING HANDLER [kubernetes/master : Master | Remove controller manager container] *********************************************************************************************************************************************************
Sunday 13 October 2019 02:00:41 +0900 (0:00:00.619) 0:11:43.377 ********
changed: [kubemaster] => {"attempts": 1, "changed": true, "cmd": "docker ps -af name=k8s_kube-controller-manager* -q | xargs --no-run-if-empty docker rm -f", "delta": "0:00:00.194041", "end": "2019-10-13 02:00:42.344941", "rc": 0, "start": "2019-10-13 02:00:42.150900", "stderr": "", "stderr_lines": [], "stdout": "9d8b334a528f", "stdout_lines": ["9d8b334a528f"]}
RUNNING HANDLER [kubernetes/master : Master | wait for kube-scheduler] *********************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:42 +0900 (0:00:00.431) 0:11:43.809 ********
FAILED - RETRYING: Master | wait for kube-scheduler (60 retries left).
ok: [kubemaster] => {"attempts": 2, "cache_control": "no-cache, private", "changed": false, "connection": "close", "content_length": "2", "content_type": "text/plain; charset=utf-8", "cookies": {}, "cookies_string": "", "date": "Sat, 12 Oct 2019 17:00:44 GMT", "elapsed": 0, "msg": "OK (2 bytes)", "redirected": false, "status": 200, "url": "http://localhost:10251/healthz", "x_content_type_options": "nosniff"}
RUNNING HANDLER [kubernetes/master : Master | wait for kube-controller-manager] ************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:44 +0900 (0:00:01.831) 0:11:45.640 ********
ok: [kubemaster] => {"attempts": 1, "cache_control": "no-cache, private", "changed": false, "connection": "close", "content_length": "2", "content_type": "text/plain; charset=utf-8", "cookies": {}, "cookies_string": "", "date": "Sat, 12 Oct 2019 17:00:44 GMT", "elapsed": 0, "msg": "OK (2 bytes)", "redirected": false, "status": 200, "url": "http://localhost:10252/healthz", "x_content_type_options": "nosniff"}
RUNNING HANDLER [kubernetes/master : Master | wait for the apiserver to be running] ********************************************************************************************************************************************************
Sunday 13 October 2019 02:00:44 +0900 (0:00:00.331) 0:11:45.972 ********
ok: [kubemaster] => {"attempts": 1, "cache_control": "no-cache, private", "changed": false, "connection": "close", "content_length": "2", "content_type": "text/plain; charset=utf-8", "cookies": {}, "cookies_string": "", "date": "Sat, 12 Oct 2019 17:00:45 GMT", "elapsed": 0, "msg": "OK (2 bytes)", "redirected": false, "status": 200, "url": "https://127.0.0.1:6443/healthz", "x_content_type_options": "nosniff"}
RUNNING HANDLER [kubernetes/master : Master | set secret_changed] **************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:45 +0900 (0:00:00.639) 0:11:46.612 ********
changed: [kubemaster] => {"changed": true, "cmd": ["/bin/true"], "delta": "0:00:00.001861", "end": "2019-10-13 02:00:45.453110", "rc": 0, "start": "2019-10-13 02:00:45.451249", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
RUNNING HANDLER [kubernetes/master : Master | set secret_changed to true] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:45 +0900 (0:00:00.415) 0:11:47.027 ********
ok: [kubemaster] => {"ansible_facts": {"secret_changed": true}, "changed": false}
RUNNING HANDLER [kubernetes/master : Master | Copy new kubeconfig for root user] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:00:45 +0900 (0:00:00.085) 0:11:47.112 ********
ok: [kubemaster] => {"changed": false, "checksum": "886b61952043e69964046a34cb2bad1abbe4fdeb", "dest": "/root/.kube/config", "gid": 0, "group": "root", "md5sum": "afa625098e3f3f01bfd4cb12b5293a33", "mode": "0600", "owner": "root", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 5466, "src": "/etc/kubernetes/admin.conf", "state": "file", "uid": 0}
PLAY [k8s-cluster] *************************************************************************************************************************************************************************************************************************
TASK [download : prep_download | Set a few facts] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:46 +0900 (0:00:00.478) 0:11:47.591 ********
TASK [download : Set image info command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:46 +0900 (0:00:00.163) 0:11:47.754 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:46 +0900 (0:00:00.146) 0:11:47.901 ********
TASK [download : prep_download | Create staging directory on remote node] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:46 +0900 (0:00:00.140) 0:11:48.042 ********
TASK [download : prep_download | Create local cache for files and images] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:46 +0900 (0:00:00.195) 0:11:48.237 ********
TASK [download : prep_download | On localhost, check if passwordless root is possible] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:00:46 +0900 (0:00:00.099) 0:11:48.336 ********
TASK [download : prep_download | On localhost, check if user has access to docker without using sudo] **************************************************************************************************************************************
Sunday 13 October 2019 02:00:47 +0900 (0:00:00.105) 0:11:48.442 ********
TASK [download : prep_download | Parse the outputs of the previous commands] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:47 +0900 (0:00:00.091) 0:11:48.534 ********
TASK [download : prep_download | Check that local user is in group or can become root] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:00:47 +0900 (0:00:00.138) 0:11:48.673 ********
TASK [download : include_tasks] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:47 +0900 (0:00:00.152) 0:11:48.825 ********
TASK [download : download | Get kubeadm binary and list of required images] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:47 +0900 (0:00:00.169) 0:11:48.995 ********
TASK [download : download | Download files / images] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:47 +0900 (0:00:00.157) 0:11:49.152 ********
TASK [download : download | Sync files / images from ansible host to nodes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:49 +0900 (0:00:01.735) 0:11:50.887 ********
TASK [kubespray-defaults : Configure defaults] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:51 +0900 (0:00:01.632) 0:11:52.520 ********
ok: [kubeworker] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
ok: [kubemaster] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
TASK [kubernetes/kubeadm : Set kubeadm_discovery_address] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:51 +0900 (0:00:00.198) 0:11:52.719 ********
ok: [kubeworker] => {"ansible_facts": {"kubeadm_discovery_address": "192.168.56.107:6443"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"kubeadm_discovery_address": "192.168.56.107:6443"}, "changed": false}
TASK [kubernetes/kubeadm : Check if kubelet.conf exists] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:51 +0900 (0:00:00.455) 0:11:53.174 ********
ok: [kubeworker] => {"changed": false, "stat": {"exists": false}}
ok: [kubemaster] => {"changed": false, "stat": {"atime": 1570899641.429, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 16, "charset": "us-ascii", "checksum": "98ef5f4cb7a0aaa63d9f5f3551402d5341771896", "ctime": 1570899628.994, "dev": 64768, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 8449621, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0600", "mtime": 1570899627.661999, "nlink": 1, "path": "/etc/kubernetes/kubelet.conf", "pw_name": "root", "readable": true, "rgrp": false, "roth": false, "rusr": true, "size": 5494, "uid": 0, "version": "18446744071756953557", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}}
TASK [kubernetes/kubeadm : Check if kubeadm CA cert is accessible] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:52 +0900 (0:00:00.378) 0:11:53.552 ********
ok: [kubeworker -> None] => {"changed": false, "stat": {"atime": 1570899597.957, "attr_flags": "", "attributes": [], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "checksum": "eaf4ba74fa3605dbae5fdace9b64679c723cf798", "ctime": 1570899597.74, "dev": 64768, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 4216064, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1570899597.74, "nlink": 1, "path": "/etc/kubernetes/ssl/ca.crt", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 1025, "uid": 0, "version": "1438275706", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}}
TASK [kubernetes/kubeadm : Calculate kubeadm CA cert hash] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:52 +0900 (0:00:00.294) 0:11:53.846 ********
changed: [kubeworker -> None] => {"changed": true, "cmd": "openssl x509 -pubkey -in /etc/kubernetes/ssl/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'", "delta": "0:00:00.021522", "end": "2019-10-13 02:00:52.626339", "rc": 0, "start": "2019-10-13 02:00:52.604817", "stderr": "", "stderr_lines": [], "stdout": "27b4e9454913a040325c2c88deea4d3e41bb5047f7b0a522cc2ac3d6b67102be", "stdout_lines": ["27b4e9454913a040325c2c88deea4d3e41bb5047f7b0a522cc2ac3d6b67102be"]}
TASK [kubernetes/kubeadm : Create kubeadm token for joining nodes with 24h expiration (default)] *******************************************************************************************************************************************
Sunday 13 October 2019 02:00:52 +0900 (0:00:00.333) 0:11:54.180 ********
changed: [kubeworker -> None] => {"changed": true, "cmd": ["/usr/local/bin/kubeadm", "token", "create"], "delta": "0:00:00.083637", "end": "2019-10-13 02:00:52.996903", "rc": 0, "start": "2019-10-13 02:00:52.913266", "stderr": "", "stderr_lines": [], "stdout": "iddbgq.1xfzs36vikecyrbl", "stdout_lines": ["iddbgq.1xfzs36vikecyrbl"]}
TASK [kubernetes/kubeadm : Set kubeadm_token to generated token] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:53 +0900 (0:00:00.315) 0:11:54.496 ********
ok: [kubeworker] => {"ansible_facts": {"kubeadm_token": "iddbgq.1xfzs36vikecyrbl"}, "changed": false}
TASK [kubernetes/kubeadm : gets the kubeadm version] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:53 +0900 (0:00:00.141) 0:11:54.638 ********
changed: [kubeworker] => {"changed": true, "cmd": ["/usr/local/bin/kubeadm", "version", "-o", "short"], "delta": "0:00:00.046784", "end": "2019-10-13 02:00:53.118538", "rc": 0, "start": "2019-10-13 02:00:53.071754", "stderr": "", "stderr_lines": [], "stdout": "v1.16.0", "stdout_lines": ["v1.16.0"]}
changed: [kubemaster] => {"changed": true, "cmd": ["/usr/local/bin/kubeadm", "version", "-o", "short"], "delta": "0:00:00.059875", "end": "2019-10-13 02:00:53.478328", "rc": 0, "start": "2019-10-13 02:00:53.418453", "stderr": "", "stderr_lines": [], "stdout": "v1.16.0", "stdout_lines": ["v1.16.0"]}
TASK [kubernetes/kubeadm : sets kubeadm api version to v1beta1] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:53 +0900 (0:00:00.341) 0:11:54.980 ********
ok: [kubeworker] => {"ansible_facts": {"kubeadmConfig_api_version": "v1beta1"}, "changed": false}
ok: [kubemaster] => {"ansible_facts": {"kubeadmConfig_api_version": "v1beta1"}, "changed": false}
TASK [kubernetes/kubeadm : Create kubeadm client config] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:53 +0900 (0:00:00.148) 0:11:55.128 ********
changed: [kubeworker] => {"changed": true, "checksum": "f8e8493e220eab345d9c150c7f94e1fbab4f1f34", "dest": "/etc/kubernetes/kubeadm-client.conf", "gid": 0, "group": "root", "md5sum": "4a0c9861953e401216fa0c2e542e6afe", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 440, "src": "/root/.ansible/tmp/ansible-tmp-1570899653.734125-21410870044175/source", "state": "file", "uid": 0}
TASK [kubernetes/kubeadm : Join to cluster] ************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:00:54 +0900 (0:00:00.519) 0:11:55.647 ********
changed: [kubeworker] => {"changed": true, "cmd": ["timeout", "-k", "120s", "120s", "/usr/local/bin/kubeadm", "join", "--config", "/etc/kubernetes/kubeadm-client.conf", "--ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests"], "delta": "0:00:22.302299", "end": "2019-10-13 02:01:16.397766", "rc": 0, "start": "2019-10-13 02:00:54.095467", "stderr": "\t[WARNING DirAvailable--etc-kubernetes-manifests]: /etc/kubernetes/manifests is not empty\n\t[WARNING IsDockerSystemdCheck]: detected \"cgroupfs\" as the Docker cgroup driver. The recommended driver is \"systemd\". Please follow the guide at https://kubernetes.io/docs/setup/cri/", "stderr_lines": ["\t[WARNING DirAvailable--etc-kubernetes-manifests]: /etc/kubernetes/manifests is not empty", "\t[WARNING IsDockerSystemdCheck]: detected \"cgroupfs\" as the Docker cgroup driver. The recommended driver is \"systemd\". Please follow the guide at https://kubernetes.io/docs/setup/cri/"], "stdout": "[preflight] Running pre-flight checks\n[preflight] Reading configuration from the cluster...\n[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'\n[kubelet-start] Downloading configuration for the kubelet from the \"kubelet-config-1.16\" ConfigMap in the kube-system namespace\n[kubelet-start] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"\n[kubelet-start] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"\n[kubelet-start] Activating the kubelet service\n[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...\n\nThis node has joined the cluster:\n* Certificate signing request was sent to apiserver and a response was received.\n* The Kubelet was informed of the new secure connection details.\n\nRun 'kubectl get nodes' on the control-plane to see this node join the cluster.", "stdout_lines": ["[preflight] Running pre-flight checks", "[preflight] Reading configuration from the cluster...", "[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'", "[kubelet-start] Downloading configuration for the kubelet from the \"kubelet-config-1.16\" ConfigMap in the kube-system namespace", "[kubelet-start] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"", "[kubelet-start] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"", "[kubelet-start] Activating the kubelet service", "[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...", "", "This node has joined the cluster:", "* Certificate signing request was sent to apiserver and a response was received.", "* The Kubelet was informed of the new secure connection details.", "", "Run 'kubectl get nodes' on the control-plane to see this node join the cluster."]}
TASK [kubernetes/kubeadm : Display kubeadm join stderr if any] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:16 +0900 (0:00:22.555) 0:12:18.203 ********
TASK [kubernetes/kubeadm : Update server field in kubelet kubeconfig] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:16 +0900 (0:00:00.130) 0:12:18.334 ********
changed: [kubeworker] => {"backup": "/etc/kubernetes/kubelet.conf.18700.2019-10-13@02:01:17~", "changed": true, "msg": "line replaced"}
TASK [kubernetes/kubeadm : Update server field in kube-proxy kubeconfig] *******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:17 +0900 (0:00:00.701) 0:12:19.035 ********
TASK [kubernetes/kubeadm : Restart all kube-proxy pods to ensure that they load the new configmap] *****************************************************************************************************************************************
Sunday 13 October 2019 02:01:17 +0900 (0:00:00.083) 0:12:19.118 ********
TASK [kubernetes/kubeadm : Symlink kubelet kubeconfig for calico/canal] ********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:17 +0900 (0:00:00.080) 0:12:19.199 ********
TASK [kubernetes/kubeadm : Delete kube-proxy daemonset if kube_proxy_remove set, e.g. kube_network_plugin providing proxy services] ********************************************************************************************************
Sunday 13 October 2019 02:01:17 +0900 (0:00:00.135) 0:12:19.334 ********
TASK [kubernetes/kubeadm : Extract etcd certs from control plane if using etcd kubeadm mode] ***********************************************************************************************************************************************
Sunday 13 October 2019 02:01:17 +0900 (0:00:00.075) 0:12:19.410 ********
TASK [network_plugin/cilium : Cilium | Ensure BFPFS mounted] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:18 +0900 (0:00:00.126) 0:12:19.537 ********
TASK [network_plugin/cilium : Cilium | Create Cilium certs directory] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:18 +0900 (0:00:00.143) 0:12:19.680 ********
TASK [network_plugin/cilium : Cilium | Link etcd certificates for cilium] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:18 +0900 (0:00:00.132) 0:12:19.812 ********
TASK [network_plugin/cilium : Cilium | Create Cilium node manifests] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:18 +0900 (0:00:00.213) 0:12:20.027 ********
TASK [network_plugin/cilium : Cilium | Set CNI directory permissions] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:18 +0900 (0:00:00.225) 0:12:20.252 ********
TASK [network_plugin/cilium : Cilium | Copy CNI plugins] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:18 +0900 (0:00:00.141) 0:12:20.393 ********
TASK [network_plugin/cilium : Cilium | Enable portmap addon] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:19 +0900 (0:00:00.133) 0:12:20.527 ********
TASK [network_plugin/calico : Check vars defined correctly] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:19 +0900 (0:00:00.130) 0:12:20.657 ********
ok: [kubeworker] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [network_plugin/calico : Get current version of calico cluster version] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:19 +0900 (0:00:00.107) 0:12:20.765 ********
ok: [kubeworker] => {"changed": false, "cmd": "/usr/local/bin/calicoctl.sh version | grep 'Cluster Version:' | awk '{ print $3}'", "delta": "0:00:00.004835", "end": "2019-10-13 02:01:19.213497", "rc": 0, "start": "2019-10-13 02:01:19.208662", "stderr": "/bin/sh: /usr/local/bin/calicoctl.sh: No such file or directory", "stderr_lines": ["/bin/sh: /usr/local/bin/calicoctl.sh: No such file or directory"], "stdout": "", "stdout_lines": []}
TASK [network_plugin/calico : Check that calico version is enough for upgrade] *************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:19 +0900 (0:00:00.235) 0:12:21.000 ********
TASK [network_plugin/calico : Set upgrade flag when version needs to be updated] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:01:19 +0900 (0:00:00.099) 0:12:21.100 ********
TASK [network_plugin/calico : Calico | Get kubelet hostname] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:19 +0900 (0:00:00.147) 0:12:21.247 ********
TASK [network_plugin/calico : include_tasks] ***********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:20 +0900 (0:00:00.238) 0:12:21.486 ********
TASK [network_plugin/calico : include_tasks] ***********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:20 +0900 (0:00:00.078) 0:12:21.564 ********
included: /root/kubespray/kubespray/roles/network_plugin/calico/tasks/install.yml for kubeworker, kubemaster
TASK [network_plugin/calico : Calico | Copy calicoctl binary from download dir] ************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:20 +0900 (0:00:00.420) 0:12:21.985 ********
changed: [kubeworker] => {"changed": true, "checksum": "84f9e061d85fddf6437700a90af18084654cbb21", "dest": "/usr/local/bin/calicoctl", "gid": 0, "group": "root", "md5sum": "7840346b42d3df44cecb256266104e88", "mode": "0755", "owner": "root", "secontext": "system_u:object_r:bin_t:s0", "size": 35861536, "src": "/tmp/releases/calicoctl", "state": "file", "uid": 0}
changed: [kubemaster] => {"changed": true, "checksum": "84f9e061d85fddf6437700a90af18084654cbb21", "dest": "/usr/local/bin/calicoctl", "gid": 0, "group": "root", "md5sum": "7840346b42d3df44cecb256266104e88", "mode": "0755", "owner": "root", "secontext": "system_u:object_r:bin_t:s0", "size": 35861536, "src": "/tmp/releases/calicoctl", "state": "file", "uid": 0}
TASK [network_plugin/calico : Calico | Check if host has NetworkManager] *******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:21 +0900 (0:00:00.506) 0:12:22.491 ********
ok: [kubeworker] => {"changed": false, "cmd": ["systemctl", "show", "NetworkManager"], "delta": "0:00:00.008694", "end": "2019-10-13 02:01:20.929455", "failed_when_result": false, "rc": 0, "start": "2019-10-13 02:01:20.920761", "stderr": "", "stderr_lines": [], "stdout": "Type=dbus\nRestart=on-failure\nNotifyAccess=none\nRestartUSec=100ms\nTimeoutStartUSec=1min 30s\nTimeoutStopUSec=1min 30s\nWatchdogUSec=0\nWatchdogTimestamp=Sat 2019-10-12 19:54:27 KST\nWatchdogTimestampMonotonic=4923434\nStartLimitInterval=10000000\nStartLimitBurst=5\nStartLimitAction=none\nFailureAction=none\nPermissionsStartOnly=no\nRootDirectoryStartOnly=no\nRemainAfterExit=no\nGuessMainPID=yes\nMainPID=2891\nControlPID=0\nBusName=org.freedesktop.NetworkManager\nFileDescriptorStoreMax=0\nStatusErrno=0\nResult=success\nExecMainStartTimestamp=Sat 2019-10-12 19:54:26 KST\nExecMainStartTimestampMonotonic=4879101\nExecMainExitTimestampMonotonic=0\nExecMainPID=2891\nExecMainCode=0\nExecMainStatus=0\nExecStart={ path=/usr/sbin/NetworkManager ; argv[]=/usr/sbin/NetworkManager --no-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }\nExecReload={ path=/usr/bin/dbus-send ; argv[]=/usr/bin/dbus-send --print-reply --system --type=method_call --dest=org.freedesktop.NetworkManager /org/freedesktop/NetworkManager org.freedesktop.NetworkManager.Reload uint32:0 ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }\nSlice=system.slice\nControlGroup=/system.slice/NetworkManager.service\nMemoryCurrent=24576\nTasksCurrent=5\nDelegate=no\nCPUAccounting=no\nCPUShares=18446744073709551615\nStartupCPUShares=18446744073709551615\nCPUQuotaPerSecUSec=infinity\nBlockIOAccounting=no\nBlockIOWeight=18446744073709551615\nStartupBlockIOWeight=18446744073709551615\nMemoryAccounting=no\nMemoryLimit=18446744073709551615\nDevicePolicy=auto\nTasksAccounting=no\nTasksMax=18446744073709551615\nUMask=0022\nLimitCPU=18446744073709551615\nLimitFSIZE=18446744073709551615\nLimitDATA=18446744073709551615\nLimitSTACK=18446744073709551615\nLimitCORE=18446744073709551615\nLimitRSS=18446744073709551615\nLimitNOFILE=4096\nLimitAS=18446744073709551615\nLimitNPROC=7257\nLimitMEMLOCK=65536\nLimitLOCKS=18446744073709551615\nLimitSIGPENDING=7257\nLimitMSGQUEUE=819200\nLimitNICE=0\nLimitRTPRIO=0\nLimitRTTIME=18446744073709551615\nOOMScoreAdjust=0\nNice=0\nIOScheduling=0\nCPUSchedulingPolicy=0\nCPUSchedulingPriority=0\nTimerSlackNSec=50000\nCPUSchedulingResetOnFork=no\nNonBlocking=no\nStandardInput=null\nStandardOutput=journal\nStandardError=inherit\nTTYReset=no\nTTYVHangup=no\nTTYVTDisallocate=no\nSyslogPriority=30\nSyslogLevelPrefix=yes\nSecureBits=0\nCapabilityBoundingSet=539309282\nAmbientCapabilities=0\nMountFlags=0\nPrivateTmp=no\nPrivateNetwork=no\nPrivateDevices=no\nProtectHome=read-only\nProtectSystem=yes\nSameProcessGroup=no\nIgnoreSIGPIPE=yes\nNoNewPrivileges=no\nSystemCallErrorNumber=0\nRuntimeDirectoryMode=0755\nKillMode=process\nKillSignal=15\nSendSIGKILL=yes\nSendSIGHUP=no\nId=NetworkManager.service\nNames=NetworkManager.service\nRequires=basic.target\nWants=network.target system.slice\nRequiredBy=NetworkManager-wait-online.service\nWantedBy=multi-user.target\nConflicts=shutdown.target\nBefore=shutdown.target multi-user.target NetworkManager-wait-online.service network.service network.target\nAfter=network-pre.target system.slice systemd-journald.socket dbus.service basic.target\nDocumentation=man:NetworkManager(8)\nDescription=Network Manager\nLoadState=loaded\nActiveState=active\nSubState=running\nFragmentPath=/usr/lib/systemd/system/NetworkManager.service\nUnitFileState=enabled\nUnitFilePreset=enabled\nInactiveExitTimestamp=Sat 2019-10-12 19:54:26 KST\nInactiveExitTimestampMonotonic=4879126\nActiveEnterTimestamp=Sat 2019-10-12 19:54:27 KST\nActiveEnterTimestampMonotonic=4923594\nActiveExitTimestampMonotonic=0\nInactiveEnterTimestampMonotonic=0\nCanStart=yes\nCanStop=yes\nCanReload=yes\nCanIsolate=no\nStopWhenUnneeded=no\nRefuseManualStart=no\nRefuseManualStop=no\nAllowIsolate=no\nDefaultDependencies=yes\nOnFailureJobMode=replace\nIgnoreOnIsolate=no\nIgnoreOnSnapshot=no\nNeedDaemonReload=no\nJobTimeoutUSec=0\nJobTimeoutAction=none\nConditionResult=yes\nAssertResult=yes\nConditionTimestamp=Sat 2019-10-12 19:54:26 KST\nConditionTimestampMonotonic=4878618\nAssertTimestamp=Sat 2019-10-12 19:54:26 KST\nAssertTimestampMonotonic=4878618\nTransient=no", "stdout_lines": ["Type=dbus", "Restart=on-failure", "NotifyAccess=none", "RestartUSec=100ms", "TimeoutStartUSec=1min 30s", "TimeoutStopUSec=1min 30s", "WatchdogUSec=0", "WatchdogTimestamp=Sat 2019-10-12 19:54:27 KST", "WatchdogTimestampMonotonic=4923434", "StartLimitInterval=10000000", "StartLimitBurst=5", "StartLimitAction=none", "FailureAction=none", "PermissionsStartOnly=no", "RootDirectoryStartOnly=no", "RemainAfterExit=no", "GuessMainPID=yes", "MainPID=2891", "ControlPID=0", "BusName=org.freedesktop.NetworkManager", "FileDescriptorStoreMax=0", "StatusErrno=0", "Result=success", "ExecMainStartTimestamp=Sat 2019-10-12 19:54:26 KST", "ExecMainStartTimestampMonotonic=4879101", "ExecMainExitTimestampMonotonic=0", "ExecMainPID=2891", "ExecMainCode=0", "ExecMainStatus=0", "ExecStart={ path=/usr/sbin/NetworkManager ; argv[]=/usr/sbin/NetworkManager --no-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReload={ path=/usr/bin/dbus-send ; argv[]=/usr/bin/dbus-send --print-reply --system --type=method_call --dest=org.freedesktop.NetworkManager /org/freedesktop/NetworkManager org.freedesktop.NetworkManager.Reload uint32:0 ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "Slice=system.slice", "ControlGroup=/system.slice/NetworkManager.service", "MemoryCurrent=24576", "TasksCurrent=5", "Delegate=no", "CPUAccounting=no", "CPUShares=18446744073709551615", "StartupCPUShares=18446744073709551615", "CPUQuotaPerSecUSec=infinity", "BlockIOAccounting=no", "BlockIOWeight=18446744073709551615", "StartupBlockIOWeight=18446744073709551615", "MemoryAccounting=no", "MemoryLimit=18446744073709551615", "DevicePolicy=auto", "TasksAccounting=no", "TasksMax=18446744073709551615", "UMask=0022", "LimitCPU=18446744073709551615", "LimitFSIZE=18446744073709551615", "LimitDATA=18446744073709551615", "LimitSTACK=18446744073709551615", "LimitCORE=18446744073709551615", "LimitRSS=18446744073709551615", "LimitNOFILE=4096", "LimitAS=18446744073709551615", "LimitNPROC=7257", "LimitMEMLOCK=65536", "LimitLOCKS=18446744073709551615", "LimitSIGPENDING=7257", "LimitMSGQUEUE=819200", "LimitNICE=0", "LimitRTPRIO=0", "LimitRTTIME=18446744073709551615", "OOMScoreAdjust=0", "Nice=0", "IOScheduling=0", "CPUSchedulingPolicy=0", "CPUSchedulingPriority=0", "TimerSlackNSec=50000", "CPUSchedulingResetOnFork=no", "NonBlocking=no", "StandardInput=null", "StandardOutput=journal", "StandardError=inherit", "TTYReset=no", "TTYVHangup=no", "TTYVTDisallocate=no", "SyslogPriority=30", "SyslogLevelPrefix=yes", "SecureBits=0", "CapabilityBoundingSet=539309282", "AmbientCapabilities=0", "MountFlags=0", "PrivateTmp=no", "PrivateNetwork=no", "PrivateDevices=no", "ProtectHome=read-only", "ProtectSystem=yes", "SameProcessGroup=no", "IgnoreSIGPIPE=yes", "NoNewPrivileges=no", "SystemCallErrorNumber=0", "RuntimeDirectoryMode=0755", "KillMode=process", "KillSignal=15", "SendSIGKILL=yes", "SendSIGHUP=no", "Id=NetworkManager.service", "Names=NetworkManager.service", "Requires=basic.target", "Wants=network.target system.slice", "RequiredBy=NetworkManager-wait-online.service", "WantedBy=multi-user.target", "Conflicts=shutdown.target", "Before=shutdown.target multi-user.target NetworkManager-wait-online.service network.service network.target", "After=network-pre.target system.slice systemd-journald.socket dbus.service basic.target", "Documentation=man:NetworkManager(8)", "Description=Network Manager", "LoadState=loaded", "ActiveState=active", "SubState=running", "FragmentPath=/usr/lib/systemd/system/NetworkManager.service", "UnitFileState=enabled", "UnitFilePreset=enabled", "InactiveExitTimestamp=Sat 2019-10-12 19:54:26 KST", "InactiveExitTimestampMonotonic=4879126", "ActiveEnterTimestamp=Sat 2019-10-12 19:54:27 KST", "ActiveEnterTimestampMonotonic=4923594", "ActiveExitTimestampMonotonic=0", "InactiveEnterTimestampMonotonic=0", "CanStart=yes", "CanStop=yes", "CanReload=yes", "CanIsolate=no", "StopWhenUnneeded=no", "RefuseManualStart=no", "RefuseManualStop=no", "AllowIsolate=no", "DefaultDependencies=yes", "OnFailureJobMode=replace", "IgnoreOnIsolate=no", "IgnoreOnSnapshot=no", "NeedDaemonReload=no", "JobTimeoutUSec=0", "JobTimeoutAction=none", "ConditionResult=yes", "AssertResult=yes", "ConditionTimestamp=Sat 2019-10-12 19:54:26 KST", "ConditionTimestampMonotonic=4878618", "AssertTimestamp=Sat 2019-10-12 19:54:26 KST", "AssertTimestampMonotonic=4878618", "Transient=no"]}
ok: [kubemaster] => {"changed": false, "cmd": ["systemctl", "show", "NetworkManager"], "delta": "0:00:00.010563", "end": "2019-10-13 02:01:21.290352", "failed_when_result": false, "rc": 0, "start": "2019-10-13 02:01:21.279789", "stderr": "", "stderr_lines": [], "stdout": "Type=dbus\nRestart=on-failure\nNotifyAccess=none\nRestartUSec=100ms\nTimeoutStartUSec=1min 30s\nTimeoutStopUSec=1min 30s\nWatchdogUSec=0\nWatchdogTimestamp=Sat 2019-10-12 19:54:23 KST\nWatchdogTimestampMonotonic=4299715\nStartLimitInterval=10000000\nStartLimitBurst=5\nStartLimitAction=none\nFailureAction=none\nPermissionsStartOnly=no\nRootDirectoryStartOnly=no\nRemainAfterExit=no\nGuessMainPID=yes\nMainPID=768\nControlPID=0\nBusName=org.freedesktop.NetworkManager\nFileDescriptorStoreMax=0\nStatusErrno=0\nResult=success\nExecMainStartTimestamp=Sat 2019-10-12 19:54:23 KST\nExecMainStartTimestampMonotonic=4172143\nExecMainExitTimestampMonotonic=0\nExecMainPID=768\nExecMainCode=0\nExecMainStatus=0\nExecStart={ path=/usr/sbin/NetworkManager ; argv[]=/usr/sbin/NetworkManager --no-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }\nExecReload={ path=/usr/bin/dbus-send ; argv[]=/usr/bin/dbus-send --print-reply --system --type=method_call --dest=org.freedesktop.NetworkManager /org/freedesktop/NetworkManager org.freedesktop.NetworkManager.Reload uint32:0 ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }\nSlice=system.slice\nControlGroup=/system.slice/NetworkManager.service\nMemoryCurrent=4149248\nTasksCurrent=5\nDelegate=no\nCPUAccounting=no\nCPUShares=18446744073709551615\nStartupCPUShares=18446744073709551615\nCPUQuotaPerSecUSec=infinity\nBlockIOAccounting=no\nBlockIOWeight=18446744073709551615\nStartupBlockIOWeight=18446744073709551615\nMemoryAccounting=no\nMemoryLimit=18446744073709551615\nDevicePolicy=auto\nTasksAccounting=no\nTasksMax=18446744073709551615\nUMask=0022\nLimitCPU=18446744073709551615\nLimitFSIZE=18446744073709551615\nLimitDATA=18446744073709551615\nLimitSTACK=18446744073709551615\nLimitCORE=18446744073709551615\nLimitRSS=18446744073709551615\nLimitNOFILE=4096\nLimitAS=18446744073709551615\nLimitNPROC=7259\nLimitMEMLOCK=65536\nLimitLOCKS=18446744073709551615\nLimitSIGPENDING=7259\nLimitMSGQUEUE=819200\nLimitNICE=0\nLimitRTPRIO=0\nLimitRTTIME=18446744073709551615\nOOMScoreAdjust=0\nNice=0\nIOScheduling=0\nCPUSchedulingPolicy=0\nCPUSchedulingPriority=0\nTimerSlackNSec=50000\nCPUSchedulingResetOnFork=no\nNonBlocking=no\nStandardInput=null\nStandardOutput=journal\nStandardError=inherit\nTTYReset=no\nTTYVHangup=no\nTTYVTDisallocate=no\nSyslogPriority=30\nSyslogLevelPrefix=yes\nSecureBits=0\nCapabilityBoundingSet=539309282\nAmbientCapabilities=0\nMountFlags=0\nPrivateTmp=no\nPrivateNetwork=no\nPrivateDevices=no\nProtectHome=read-only\nProtectSystem=yes\nSameProcessGroup=no\nIgnoreSIGPIPE=yes\nNoNewPrivileges=no\nSystemCallErrorNumber=0\nRuntimeDirectoryMode=0755\nKillMode=process\nKillSignal=15\nSendSIGKILL=yes\nSendSIGHUP=no\nId=NetworkManager.service\nNames=NetworkManager.service\nRequires=system.slice basic.target\nWants=network.target\nRequiredBy=NetworkManager-wait-online.service\nWantedBy=multi-user.target\nConflicts=shutdown.target\nBefore=network.target NetworkManager-wait-online.service shutdown.target multi-user.target network.service\nAfter=basic.target system.slice dbus.service systemd-journald.socket network-pre.target\nDocumentation=man:NetworkManager(8)\nDescription=Network Manager\nLoadState=loaded\nActiveState=active\nSubState=running\nFragmentPath=/usr/lib/systemd/system/NetworkManager.service\nUnitFileState=enabled\nUnitFilePreset=enabled\nInactiveExitTimestamp=Sat 2019-10-12 19:54:23 KST\nInactiveExitTimestampMonotonic=4172168\nActiveEnterTimestamp=Sat 2019-10-12 19:54:23 KST\nActiveEnterTimestampMonotonic=4299977\nActiveExitTimestampMonotonic=0\nInactiveEnterTimestampMonotonic=0\nCanStart=yes\nCanStop=yes\nCanReload=yes\nCanIsolate=no\nStopWhenUnneeded=no\nRefuseManualStart=no\nRefuseManualStop=no\nAllowIsolate=no\nDefaultDependencies=yes\nOnFailureJobMode=replace\nIgnoreOnIsolate=no\nIgnoreOnSnapshot=no\nNeedDaemonReload=no\nJobTimeoutUSec=0\nJobTimeoutAction=none\nConditionResult=yes\nAssertResult=yes\nConditionTimestamp=Sat 2019-10-12 19:54:23 KST\nConditionTimestampMonotonic=4170998\nAssertTimestamp=Sat 2019-10-12 19:54:23 KST\nAssertTimestampMonotonic=4170999\nTransient=no", "stdout_lines": ["Type=dbus", "Restart=on-failure", "NotifyAccess=none", "RestartUSec=100ms", "TimeoutStartUSec=1min 30s", "TimeoutStopUSec=1min 30s", "WatchdogUSec=0", "WatchdogTimestamp=Sat 2019-10-12 19:54:23 KST", "WatchdogTimestampMonotonic=4299715", "StartLimitInterval=10000000", "StartLimitBurst=5", "StartLimitAction=none", "FailureAction=none", "PermissionsStartOnly=no", "RootDirectoryStartOnly=no", "RemainAfterExit=no", "GuessMainPID=yes", "MainPID=768", "ControlPID=0", "BusName=org.freedesktop.NetworkManager", "FileDescriptorStoreMax=0", "StatusErrno=0", "Result=success", "ExecMainStartTimestamp=Sat 2019-10-12 19:54:23 KST", "ExecMainStartTimestampMonotonic=4172143", "ExecMainExitTimestampMonotonic=0", "ExecMainPID=768", "ExecMainCode=0", "ExecMainStatus=0", "ExecStart={ path=/usr/sbin/NetworkManager ; argv[]=/usr/sbin/NetworkManager --no-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReload={ path=/usr/bin/dbus-send ; argv[]=/usr/bin/dbus-send --print-reply --system --type=method_call --dest=org.freedesktop.NetworkManager /org/freedesktop/NetworkManager org.freedesktop.NetworkManager.Reload uint32:0 ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "Slice=system.slice", "ControlGroup=/system.slice/NetworkManager.service", "MemoryCurrent=4149248", "TasksCurrent=5", "Delegate=no", "CPUAccounting=no", "CPUShares=18446744073709551615", "StartupCPUShares=18446744073709551615", "CPUQuotaPerSecUSec=infinity", "BlockIOAccounting=no", "BlockIOWeight=18446744073709551615", "StartupBlockIOWeight=18446744073709551615", "MemoryAccounting=no", "MemoryLimit=18446744073709551615", "DevicePolicy=auto", "TasksAccounting=no", "TasksMax=18446744073709551615", "UMask=0022", "LimitCPU=18446744073709551615", "LimitFSIZE=18446744073709551615", "LimitDATA=18446744073709551615", "LimitSTACK=18446744073709551615", "LimitCORE=18446744073709551615", "LimitRSS=18446744073709551615", "LimitNOFILE=4096", "LimitAS=18446744073709551615", "LimitNPROC=7259", "LimitMEMLOCK=65536", "LimitLOCKS=18446744073709551615", "LimitSIGPENDING=7259", "LimitMSGQUEUE=819200", "LimitNICE=0", "LimitRTPRIO=0", "LimitRTTIME=18446744073709551615", "OOMScoreAdjust=0", "Nice=0", "IOScheduling=0", "CPUSchedulingPolicy=0", "CPUSchedulingPriority=0", "TimerSlackNSec=50000", "CPUSchedulingResetOnFork=no", "NonBlocking=no", "StandardInput=null", "StandardOutput=journal", "StandardError=inherit", "TTYReset=no", "TTYVHangup=no", "TTYVTDisallocate=no", "SyslogPriority=30", "SyslogLevelPrefix=yes", "SecureBits=0", "CapabilityBoundingSet=539309282", "AmbientCapabilities=0", "MountFlags=0", "PrivateTmp=no", "PrivateNetwork=no", "PrivateDevices=no", "ProtectHome=read-only", "ProtectSystem=yes", "SameProcessGroup=no", "IgnoreSIGPIPE=yes", "NoNewPrivileges=no", "SystemCallErrorNumber=0", "RuntimeDirectoryMode=0755", "KillMode=process", "KillSignal=15", "SendSIGKILL=yes", "SendSIGHUP=no", "Id=NetworkManager.service", "Names=NetworkManager.service", "Requires=system.slice basic.target", "Wants=network.target", "RequiredBy=NetworkManager-wait-online.service", "WantedBy=multi-user.target", "Conflicts=shutdown.target", "Before=network.target NetworkManager-wait-online.service shutdown.target multi-user.target network.service", "After=basic.target system.slice dbus.service systemd-journald.socket network-pre.target", "Documentation=man:NetworkManager(8)", "Description=Network Manager", "LoadState=loaded", "ActiveState=active", "SubState=running", "FragmentPath=/usr/lib/systemd/system/NetworkManager.service", "UnitFileState=enabled", "UnitFilePreset=enabled", "InactiveExitTimestamp=Sat 2019-10-12 19:54:23 KST", "InactiveExitTimestampMonotonic=4172168", "ActiveEnterTimestamp=Sat 2019-10-12 19:54:23 KST", "ActiveEnterTimestampMonotonic=4299977", "ActiveExitTimestampMonotonic=0", "InactiveEnterTimestampMonotonic=0", "CanStart=yes", "CanStop=yes", "CanReload=yes", "CanIsolate=no", "StopWhenUnneeded=no", "RefuseManualStart=no", "RefuseManualStop=no", "AllowIsolate=no", "DefaultDependencies=yes", "OnFailureJobMode=replace", "IgnoreOnIsolate=no", "IgnoreOnSnapshot=no", "NeedDaemonReload=no", "JobTimeoutUSec=0", "JobTimeoutAction=none", "ConditionResult=yes", "AssertResult=yes", "ConditionTimestamp=Sat 2019-10-12 19:54:23 KST", "ConditionTimestampMonotonic=4170998", "AssertTimestamp=Sat 2019-10-12 19:54:23 KST", "AssertTimestampMonotonic=4170999", "Transient=no"]}
TASK [network_plugin/calico : Calico | Ensure NetworkManager conf.d dir] *******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:21 +0900 (0:00:00.313) 0:12:22.804 ********
ok: [kubeworker] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/NetworkManager/conf.d", "secontext": "system_u:object_r:NetworkManager_etc_t:s0", "size": 6, "state": "directory", "uid": 0}
ok: [kubemaster] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/NetworkManager/conf.d", "secontext": "system_u:object_r:NetworkManager_etc_t:s0", "size": 6, "state": "directory", "uid": 0}
TASK [network_plugin/calico : Calico | Prevent NetworkManager from managing Calico interfaces] *********************************************************************************************************************************************
Sunday 13 October 2019 02:01:21 +0900 (0:00:00.342) 0:12:23.147 ********
changed: [kubeworker] => {"changed": true, "checksum": "65f6506ad05fe2e569147c5cc56acf9bdbf259af", "dest": "/etc/NetworkManager/conf.d/calico.conf", "gid": 0, "group": "root", "md5sum": "3a38215407fdec0468079714c8ced63c", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:NetworkManager_etc_t:s0", "size": 70, "src": "/root/.ansible/tmp/ansible-tmp-1570899681.7534988-32102385122642/source", "state": "file", "uid": 0}
changed: [kubemaster] => {"changed": true, "checksum": "65f6506ad05fe2e569147c5cc56acf9bdbf259af", "dest": "/etc/NetworkManager/conf.d/calico.conf", "gid": 0, "group": "root", "md5sum": "3a38215407fdec0468079714c8ced63c", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:NetworkManager_etc_t:s0", "size": 70, "src": "/root/.ansible/tmp/ansible-tmp-1570899681.807815-34226713157848/source", "state": "file", "uid": 0}
TASK [network_plugin/calico : Calico | Write Calico cni config] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:22 +0900 (0:00:00.665) 0:12:23.813 ********
changed: [kubeworker] => {"changed": true, "checksum": "a7d6be8481b1c94174461fdbc0423c974008d7e2", "dest": "/etc/cni/net.d/calico.conflist.template", "gid": 0, "group": "root", "md5sum": "4d2fa186452a25487606b109c0242100", "mode": "0644", "owner": "kube", "secontext": "system_u:object_r:etc_t:s0", "size": 734, "src": "/root/.ansible/tmp/ansible-tmp-1570899682.4185996-204503487220288/source", "state": "file", "uid": 997}
changed: [kubemaster] => {"changed": true, "checksum": "275561630832ccbf596888f9f61fbbbe0501b8ff", "dest": "/etc/cni/net.d/calico.conflist.template", "gid": 0, "group": "root", "md5sum": "15691c92a767af9ae6c5da3cc7e37f11", "mode": "0644", "owner": "kube", "secontext": "system_u:object_r:etc_t:s0", "size": 734, "src": "/root/.ansible/tmp/ansible-tmp-1570899682.473002-162552971258952/source", "state": "file", "uid": 997}
TASK [network_plugin/calico : Calico | Create calico certs directory] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:23 +0900 (0:00:00.954) 0:12:24.767 ********
changed: [kubeworker] => {"changed": true, "gid": 0, "group": "root", "mode": "0750", "owner": "root", "path": "/etc/calico/certs", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 6, "state": "directory", "uid": 0}
changed: [kubemaster] => {"changed": true, "gid": 0, "group": "root", "mode": "0750", "owner": "root", "path": "/etc/calico/certs", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 6, "state": "directory", "uid": 0}
TASK [network_plugin/calico : Calico | Link etcd certificates for calico-node] *************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:23 +0900 (0:00:00.337) 0:12:25.105 ********
changed: [kubemaster] => (item={'s': 'ca.pem', 'd': 'ca_cert.crt'}) => {"ansible_loop_var": "item", "changed": true, "dest": "/etc/calico/certs/ca_cert.crt", "gid": 0, "group": "root", "item": {"d": "ca_cert.crt", "s": "ca.pem"}, "mode": "0640", "owner": "kube", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 1090, "src": "/etc/ssl/etcd/ssl/ca.pem", "state": "hard", "uid": 997}
changed: [kubeworker] => (item={'s': 'ca.pem', 'd': 'ca_cert.crt'}) => {"ansible_loop_var": "item", "changed": true, "dest": "/etc/calico/certs/ca_cert.crt", "gid": 0, "group": "root", "item": {"d": "ca_cert.crt", "s": "ca.pem"}, "mode": "0640", "owner": "kube", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 1090, "src": "/etc/ssl/etcd/ssl/ca.pem", "state": "hard", "uid": 997}
changed: [kubeworker] => (item={'s': 'node-kubeworker.pem', 'd': 'cert.crt'}) => {"ansible_loop_var": "item", "changed": true, "dest": "/etc/calico/certs/cert.crt", "gid": 0, "group": "root", "item": {"d": "cert.crt", "s": "node-kubeworker.pem"}, "mode": "0640", "owner": "kube", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 1371, "src": "/etc/ssl/etcd/ssl/node-kubeworker.pem", "state": "hard", "uid": 997}
changed: [kubemaster] => (item={'s': 'node-kubemaster.pem', 'd': 'cert.crt'}) => {"ansible_loop_var": "item", "changed": true, "dest": "/etc/calico/certs/cert.crt", "gid": 0, "group": "root", "item": {"d": "cert.crt", "s": "node-kubemaster.pem"}, "mode": "0640", "owner": "kube", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 1371, "src": "/etc/ssl/etcd/ssl/node-kubemaster.pem", "state": "hard", "uid": 997}
changed: [kubeworker] => (item={'s': 'node-kubeworker-key.pem', 'd': 'key.pem'}) => {"ansible_loop_var": "item", "changed": true, "dest": "/etc/calico/certs/key.pem", "gid": 0, "group": "root", "item": {"d": "key.pem", "s": "node-kubeworker-key.pem"}, "mode": "0640", "owner": "kube", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 1675, "src": "/etc/ssl/etcd/ssl/node-kubeworker-key.pem", "state": "hard", "uid": 997}
changed: [kubemaster] => (item={'s': 'node-kubemaster-key.pem', 'd': 'key.pem'}) => {"ansible_loop_var": "item", "changed": true, "dest": "/etc/calico/certs/key.pem", "gid": 0, "group": "root", "item": {"d": "key.pem", "s": "node-kubemaster-key.pem"}, "mode": "0640", "owner": "kube", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 1675, "src": "/etc/ssl/etcd/ssl/node-kubemaster-key.pem", "state": "hard", "uid": 997}
TASK [network_plugin/calico : Calico | Install calicoctl wrapper script] *******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:24 +0900 (0:00:00.737) 0:12:25.842 ********
changed: [kubeworker] => {"changed": true, "checksum": "b08cc4a9d7e3fe7d584e533f39ccab7a994c2769", "dest": "/usr/local/bin/calicoctl.sh", "gid": 0, "group": "root", "md5sum": "e82b5c37670c6a4f3b980023cbd7e24b", "mode": "0755", "owner": "root", "secontext": "system_u:object_r:bin_t:s0", "size": 223, "src": "/root/.ansible/tmp/ansible-tmp-1570899684.445514-165257258766780/source", "state": "file", "uid": 0}
changed: [kubemaster] => {"changed": true, "checksum": "b08cc4a9d7e3fe7d584e533f39ccab7a994c2769", "dest": "/usr/local/bin/calicoctl.sh", "gid": 0, "group": "root", "md5sum": "e82b5c37670c6a4f3b980023cbd7e24b", "mode": "0755", "owner": "root", "secontext": "system_u:object_r:bin_t:s0", "size": 223, "src": "/root/.ansible/tmp/ansible-tmp-1570899684.504597-38718046158936/source", "state": "file", "uid": 0}
TASK [network_plugin/calico : Calico | wait for etcd] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:25 +0900 (0:00:00.867) 0:12:26.709 ********
ok: [kubeworker] => {"attempts": 1, "changed": false, "connection": "close", "content_length": "17", "content_type": "text/plain; charset=utf-8", "cookies": {}, "cookies_string": "", "date": "Sat, 12 Oct 2019 17:01:25 GMT", "elapsed": 0, "json": {"health": "true"}, "msg": "OK (17 bytes)", "redirected": false, "status": 200, "url": "https://192.168.56.107:2379/health"}
TASK [network_plugin/calico : Calico | Check if calico network pool has already been configured] *******************************************************************************************************************************************
Sunday 13 October 2019 02:01:25 +0900 (0:00:00.645) 0:12:27.355 ********
ok: [kubemaster] => {"changed": false, "cmd": "/usr/local/bin/calicoctl.sh get ippool | grep -w \"10.233.64.0/18\" | wc -l\n", "delta": "0:00:00.037257", "end": "2019-10-13 02:01:26.182677", "rc": 0, "start": "2019-10-13 02:01:26.145420", "stderr": "", "stderr_lines": [], "stdout": "0", "stdout_lines": ["0"]}
TASK [network_plugin/calico : Calico | Ensure that calico_pool_cidr is within kube_pods_subnet when defined] *******************************************************************************************************************************
Sunday 13 October 2019 02:01:26 +0900 (0:00:00.341) 0:12:27.696 ********
TASK [network_plugin/calico : Calico | Create calico manifests for kdd] ********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:26 +0900 (0:00:00.175) 0:12:27.872 ********
TASK [network_plugin/calico : Calico | Create Calico Kubernetes datastore resources] *******************************************************************************************************************************************************
Sunday 13 October 2019 02:01:26 +0900 (0:00:00.153) 0:12:28.026 ********
TASK [network_plugin/calico : Calico | Configure calico network pool (version < v3.3.0)] ***************************************************************************************************************************************************
Sunday 13 October 2019 02:01:26 +0900 (0:00:00.154) 0:12:28.181 ********
TASK [network_plugin/calico : Calico | Configure calico network pool (version >= v3.3.0)] **************************************************************************************************************************************************
Sunday 13 October 2019 02:01:26 +0900 (0:00:00.170) 0:12:28.352 ********
changed: [kubemaster] => {"changed": true, "cmd": "echo \"\n { \"kind\": \"IPPool\",\n \"apiVersion\": \"projectcalico.org/v3\",\n \"metadata\": {\n \"name\": \"default-pool\",\n },\n \"spec\": {\n \"blockSize\": \"24\",\n \"cidr\": \"10.233.64.0/18\",\n \"ipipMode\": \"Always\",\n \"natOutgoing\": True }} \" | /usr/local/bin/calicoctl.sh apply -f -\n", "delta": "0:00:00.040502", "end": "2019-10-13 02:01:27.226433", "rc": 0, "start": "2019-10-13 02:01:27.185931", "stderr": "", "stderr_lines": [], "stdout": "Successfully applied 1 'IPPool' resource(s)", "stdout_lines": ["Successfully applied 1 'IPPool' resource(s)"]}
TASK [network_plugin/calico : Determine nodeToNodeMesh needed state] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:27 +0900 (0:00:00.379) 0:12:28.731 ********
TASK [network_plugin/calico : Calico | Set global as_num] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:27 +0900 (0:00:00.101) 0:12:28.832 ********
ok: [kubemaster] => {"changed": false, "cmd": "echo ' { \"kind\": \"BGPConfiguration\", \"apiVersion\": \"projectcalico.org/v3\", \"metadata\": {\n \"name\": \"default\",\n}, \"spec\": {\n \"logSeverityScreen\": \"Info\",\n \"nodeToNodeMeshEnabled\": true ,\n \"asNumber\": 64512 }} ' | /usr/local/bin/calicoctl.sh apply -f -\n", "delta": "0:00:00.035168", "end": "2019-10-13 02:01:27.680078", "rc": 0, "start": "2019-10-13 02:01:27.644910", "stderr": "", "stderr_lines": [], "stdout": "Successfully applied 1 'BGPConfiguration' resource(s)", "stdout_lines": ["Successfully applied 1 'BGPConfiguration' resource(s)"]}
TASK [network_plugin/calico : Calico | Configure peering with router(s) at global scope] ***************************************************************************************************************************************************
Sunday 13 October 2019 02:01:27 +0900 (0:00:00.355) 0:12:29.188 ********
TASK [network_plugin/calico : Calico | Configure peering with route reflectors at global scope] ********************************************************************************************************************************************
Sunday 13 October 2019 02:01:27 +0900 (0:00:00.156) 0:12:29.344 ********
TASK [network_plugin/calico : Calico | Configure route reflectors to peer with each other] *************************************************************************************************************************************************
Sunday 13 October 2019 02:01:28 +0900 (0:00:00.140) 0:12:29.484 ********
TASK [network_plugin/calico : Calico | Create calico manifests] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:28 +0900 (0:00:00.125) 0:12:29.610 ********
changed: [kubemaster] => (item={'name': 'calico-config', 'file': 'calico-config.yml', 'type': 'cm'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "b4600e09630b2e884d9a1b9f4a33b65d9e819206", "dest": "/etc/kubernetes/calico-config.yml", "gid": 0, "group": "root", "item": {"file": "calico-config.yml", "name": "calico-config", "type": "cm"}, "md5sum": "f38ec3957fc50db20bb1f5400b56c5e6", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 318, "src": "/root/.ansible/tmp/ansible-tmp-1570899688.3032653-269065909344950/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'calico-node', 'file': 'calico-node.yml', 'type': 'ds'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "6d8ea8b5dacd672db970b32b9ac00b0592db17e2", "dest": "/etc/kubernetes/calico-node.yml", "gid": 0, "group": "root", "item": {"file": "calico-node.yml", "name": "calico-node", "type": "ds"}, "md5sum": "5c79ec2f36ebf9373431476265533e2b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 7806, "src": "/root/.ansible/tmp/ansible-tmp-1570899689.1618235-102993529416715/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'calico', 'file': 'calico-node-sa.yml', 'type': 'sa'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "c94d2aa63492bb9c1462f1d8440e047cfd0b5f51", "dest": "/etc/kubernetes/calico-node-sa.yml", "gid": 0, "group": "root", "item": {"file": "calico-node-sa.yml", "name": "calico", "type": "sa"}, "md5sum": "04e0a37d7e036cdafb2b32f23645ec6f", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 147, "src": "/root/.ansible/tmp/ansible-tmp-1570899689.7301574-195463305561096/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'calico', 'file': 'calico-cr.yml', 'type': 'clusterrole'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "7cc31ac157450407c4ffcc3107e5d395e5c6331f", "dest": "/etc/kubernetes/calico-cr.yml", "gid": 0, "group": "root", "item": {"file": "calico-cr.yml", "name": "calico", "type": "clusterrole"}, "md5sum": "1bce069e28113ed76ef5e20400761d98", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 549, "src": "/root/.ansible/tmp/ansible-tmp-1570899690.2421033-234499104160560/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'calico', 'file': 'calico-crb.yml', 'type': 'clusterrolebinding'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "4c242c0edb1c62b5a759af67900bdda8df83420a", "dest": "/etc/kubernetes/calico-crb.yml", "gid": 0, "group": "root", "item": {"file": "calico-crb.yml", "name": "calico", "type": "clusterrolebinding"}, "md5sum": "4890b13200e0ef5c268ecd3598af9523", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 270, "src": "/root/.ansible/tmp/ansible-tmp-1570899690.7139022-273923842799253/source", "state": "file", "uid": 0}
TASK [network_plugin/calico : Calico | Create calico manifests for typha] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:31 +0900 (0:00:03.019) 0:12:32.629 ********
TASK [network_plugin/calico : Start Calico resources] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:31 +0900 (0:00:00.149) 0:12:32.778 ********
ok: [kubemaster] => (item=calico-config.yml) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "b4600e09630b2e884d9a1b9f4a33b65d9e819206", "dest": "/etc/kubernetes/calico-config.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "calico-config.yml.j2", "attributes": null, "backup": false, "checksum": "b4600e09630b2e884d9a1b9f4a33b65d9e819206", "content": null, "delimiter": null, "dest": "/etc/kubernetes/calico-config.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899688.3032653-269065909344950/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "calico-config.yml", "name": "calico-config", "type": "cm"}, "md5sum": "f38ec3957fc50db20bb1f5400b56c5e6", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 318, "src": "/root/.ansible/tmp/ansible-tmp-1570899688.3032653-269065909344950/source", "state": "file", "uid": 0}, "msg": "success: configmap/calico-config created"}
ok: [kubemaster] => (item=calico-node.yml) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "6d8ea8b5dacd672db970b32b9ac00b0592db17e2", "dest": "/etc/kubernetes/calico-node.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "calico-node.yml.j2", "attributes": null, "backup": false, "checksum": "6d8ea8b5dacd672db970b32b9ac00b0592db17e2", "content": null, "delimiter": null, "dest": "/etc/kubernetes/calico-node.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899689.1618235-102993529416715/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "calico-node.yml", "name": "calico-node", "type": "ds"}, "md5sum": "5c79ec2f36ebf9373431476265533e2b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 7806, "src": "/root/.ansible/tmp/ansible-tmp-1570899689.1618235-102993529416715/source", "state": "file", "uid": 0}, "msg": "success: daemonset.apps/calico-node created"}
ok: [kubemaster] => (item=calico-node-sa.yml) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "c94d2aa63492bb9c1462f1d8440e047cfd0b5f51", "dest": "/etc/kubernetes/calico-node-sa.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "calico-node-sa.yml.j2", "attributes": null, "backup": false, "checksum": "c94d2aa63492bb9c1462f1d8440e047cfd0b5f51", "content": null, "delimiter": null, "dest": "/etc/kubernetes/calico-node-sa.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899689.7301574-195463305561096/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "calico-node-sa.yml", "name": "calico", "type": "sa"}, "md5sum": "04e0a37d7e036cdafb2b32f23645ec6f", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 147, "src": "/root/.ansible/tmp/ansible-tmp-1570899689.7301574-195463305561096/source", "state": "file", "uid": 0}, "msg": "success: serviceaccount/calico-node created"}
ok: [kubemaster] => (item=calico-cr.yml) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "7cc31ac157450407c4ffcc3107e5d395e5c6331f", "dest": "/etc/kubernetes/calico-cr.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "calico-cr.yml.j2", "attributes": null, "backup": false, "checksum": "7cc31ac157450407c4ffcc3107e5d395e5c6331f", "content": null, "delimiter": null, "dest": "/etc/kubernetes/calico-cr.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899690.2421033-234499104160560/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "calico-cr.yml", "name": "calico", "type": "clusterrole"}, "md5sum": "1bce069e28113ed76ef5e20400761d98", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 549, "src": "/root/.ansible/tmp/ansible-tmp-1570899690.2421033-234499104160560/source", "state": "file", "uid": 0}, "msg": "success: clusterrole.rbac.authorization.k8s.io/calico-node created"}
ok: [kubemaster] => (item=calico-crb.yml) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "4c242c0edb1c62b5a759af67900bdda8df83420a", "dest": "/etc/kubernetes/calico-crb.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "calico-crb.yml.j2", "attributes": null, "backup": false, "checksum": "4c242c0edb1c62b5a759af67900bdda8df83420a", "content": null, "delimiter": null, "dest": "/etc/kubernetes/calico-crb.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899690.7139022-273923842799253/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "calico-crb.yml", "name": "calico", "type": "clusterrolebinding"}, "md5sum": "4890b13200e0ef5c268ecd3598af9523", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 270, "src": "/root/.ansible/tmp/ansible-tmp-1570899690.7139022-273923842799253/source", "state": "file", "uid": 0}, "msg": "success: clusterrolebinding.rbac.authorization.k8s.io/calico-node created"}
TASK [network_plugin/calico : Wait for calico kubeconfig to be created] ********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:33 +0900 (0:00:02.242) 0:12:35.021 ********
TASK [network_plugin/calico : Calico | Configure node asNumber for per node peering] *******************************************************************************************************************************************************
Sunday 13 October 2019 02:01:33 +0900 (0:00:00.145) 0:12:35.166 ********
TASK [network_plugin/calico : Calico | Configure peering with router(s) at node scope] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:01:33 +0900 (0:00:00.141) 0:12:35.308 ********
TASK [network_plugin/flannel : Flannel | Create Flannel manifests] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:34 +0900 (0:00:00.149) 0:12:35.457 ********
TASK [network_plugin/weave : Weave | Set cni directory permissions] ************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:34 +0900 (0:00:00.177) 0:12:35.635 ********
TASK [network_plugin/weave : Weave | Copy cni plugins] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:34 +0900 (0:00:00.162) 0:12:35.797 ********
TASK [network_plugin/weave : Weave | Create manifest] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:34 +0900 (0:00:00.144) 0:12:35.942 ********
TASK [network_plugin/weave : Weave | Fix nodePort for Weave] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:34 +0900 (0:00:00.133) 0:12:36.075 ********
TASK [network_plugin/canal : Canal | Write Canal cni config] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:34 +0900 (0:00:00.134) 0:12:36.210 ********
TASK [network_plugin/canal : Canal | Create canal certs directory] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:34 +0900 (0:00:00.143) 0:12:36.353 ********
TASK [network_plugin/canal : Canal | Link etcd certificates for canal-node] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:35 +0900 (0:00:00.132) 0:12:36.486 ********
TASK [network_plugin/canal : Canal | Set Flannel etcd configuration] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:35 +0900 (0:00:00.246) 0:12:36.732 ********
TASK [network_plugin/canal : Canal | Create canal node manifests] **************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:35 +0900 (0:00:00.075) 0:12:36.808 ********
TASK [network_plugin/canal : Canal | Install calicoctl wrapper script] *********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:35 +0900 (0:00:00.260) 0:12:37.068 ********
TASK [network_plugin/canal : Canal | Create network policy directory] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:35 +0900 (0:00:00.145) 0:12:37.213 ********
TASK [network_plugin/cni : CNI | make sure /opt/cni/bin exists] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:35 +0900 (0:00:00.144) 0:12:37.358 ********
TASK [network_plugin/cni : CNI | Copy cni plugins] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:36 +0900 (0:00:00.158) 0:12:37.517 ********
TASK [network_plugin/macvlan : Macvlan | Set cni directory permissions] ********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:36 +0900 (0:00:00.127) 0:12:37.644 ********
TASK [network_plugin/macvlan : Macvlan | Copy cni plugins] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:36 +0900 (0:00:00.129) 0:12:37.773 ********
TASK [network_plugin/macvlan : Macvlan | Retrieve Pod Cidr] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:36 +0900 (0:00:00.231) 0:12:38.005 ********
TASK [network_plugin/macvlan : Macvlan | set node_pod_cidr] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:36 +0900 (0:00:00.215) 0:12:38.221 ********
TASK [network_plugin/macvlan : Macvlan | Retrieve default gateway network interface] *******************************************************************************************************************************************************
Sunday 13 October 2019 02:01:36 +0900 (0:00:00.161) 0:12:38.382 ********
TASK [network_plugin/macvlan : Macvlan | set node_default_gateway_interface] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:37 +0900 (0:00:00.141) 0:12:38.524 ********
TASK [network_plugin/macvlan : Macvlan | Install network gateway interface on debian] ******************************************************************************************************************************************************
Sunday 13 October 2019 02:01:37 +0900 (0:00:00.136) 0:12:38.660 ********
TASK [network_plugin/macvlan : Macvlan | Install macvlan script on centos] *****************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:37 +0900 (0:00:00.156) 0:12:38.817 ********
TASK [network_plugin/macvlan : Macvlan | Install post-up script on centos] *****************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:37 +0900 (0:00:00.230) 0:12:39.048 ********
TASK [network_plugin/macvlan : Macvlan | Install network gateway interface on centos] ******************************************************************************************************************************************************
Sunday 13 October 2019 02:01:37 +0900 (0:00:00.140) 0:12:39.188 ********
TASK [network_plugin/macvlan : Macvlan | Install service nat via gateway on coreos] ********************************************************************************************************************************************************
Sunday 13 October 2019 02:01:37 +0900 (0:00:00.194) 0:12:39.383 ********
TASK [network_plugin/macvlan : Macvlan | Enable service nat via gateway on coreos] *********************************************************************************************************************************************************
Sunday 13 October 2019 02:01:38 +0900 (0:00:00.146) 0:12:39.529 ********
TASK [network_plugin/macvlan : Macvlan | Install network gateway interface on coreos] ******************************************************************************************************************************************************
Sunday 13 October 2019 02:01:38 +0900 (0:00:00.161) 0:12:39.690 ********
TASK [network_plugin/macvlan : Macvlan | Install cni definition for Macvlan] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:38 +0900 (0:00:00.181) 0:12:39.872 ********
TASK [network_plugin/macvlan : Macvlan | Install loopback definition for Macvlan] **********************************************************************************************************************************************************
Sunday 13 October 2019 02:01:38 +0900 (0:00:00.155) 0:12:40.028 ********
TASK [network_plugin/macvlan : Enable net.ipv4.conf.all.arp_notify in sysctl] **************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:38 +0900 (0:00:00.154) 0:12:40.182 ********
TASK [network_plugin/contiv : Contiv | Load openvswitch kernel module] *********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:38 +0900 (0:00:00.134) 0:12:40.317 ********
TASK [network_plugin/contiv : Contiv | Create contiv etcd directories] *********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:39 +0900 (0:00:00.158) 0:12:40.475 ********
TASK [network_plugin/contiv : Contiv | Workaround https://github.com/contiv/netplugin/issues/1152] *****************************************************************************************************************************************
Sunday 13 October 2019 02:01:39 +0900 (0:00:00.167) 0:12:40.643 ********
TASK [network_plugin/contiv : Contiv | Set necessary facts] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:39 +0900 (0:00:00.131) 0:12:40.775 ********
TASK [network_plugin/contiv : Contiv | Add another manifest if contiv_enable_api_proxy is true] ********************************************************************************************************************************************
Sunday 13 October 2019 02:01:39 +0900 (0:00:00.133) 0:12:40.908 ********
TASK [network_plugin/contiv : Contiv | Create /var/contiv] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:39 +0900 (0:00:00.151) 0:12:41.060 ********
TASK [network_plugin/contiv : Contiv | Create contiv config directory] *********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:39 +0900 (0:00:00.168) 0:12:41.228 ********
TASK [network_plugin/contiv : Contiv | Install all Kubernetes resources] *******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:39 +0900 (0:00:00.139) 0:12:41.368 ********
TASK [network_plugin/contiv : Contiv | Copy certs generation script] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:40 +0900 (0:00:00.205) 0:12:41.574 ********
TASK [network_plugin/contiv : Contiv | Check for cert key existence] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:40 +0900 (0:00:00.132) 0:12:41.706 ********
TASK [network_plugin/contiv : Contiv | Generate contiv-api-proxy certificates] *************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:40 +0900 (0:00:00.146) 0:12:41.853 ********
TASK [network_plugin/contiv : Contiv | Fetch the generated certificate] ********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:40 +0900 (0:00:00.150) 0:12:42.004 ********
TASK [network_plugin/contiv : Contiv | Copy the generated certificate on nodes] ************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:40 +0900 (0:00:00.105) 0:12:42.109 ********
TASK [network_plugin/contiv : Contiv | Set cni directory permissions] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:40 +0900 (0:00:00.167) 0:12:42.277 ********
TASK [network_plugin/contiv : Contiv | Copy cni plugins] ***********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:40 +0900 (0:00:00.153) 0:12:42.430 ********
TASK [network_plugin/contiv : Contiv | Copy netctl binary from docker container] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:01:41 +0900 (0:00:00.143) 0:12:42.574 ********
TASK [network_plugin/kube-ovn : Kube-OVN | Label ovn-db node] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:41 +0900 (0:00:00.145) 0:12:42.720 ********
TASK [network_plugin/kube-ovn : Kube-OVN | Create Kube-OVN manifests] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:41 +0900 (0:00:00.132) 0:12:42.852 ********
TASK [network_plugin/kube-router : kube-router | Add annotations on kube-master] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:01:41 +0900 (0:00:00.189) 0:12:43.041 ********
TASK [network_plugin/kube-router : kube-router | Add annotations on kube-node] *************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:41 +0900 (0:00:00.167) 0:12:43.209 ********
TASK [network_plugin/kube-router : kube-router | Add common annotations on all servers] ****************************************************************************************************************************************************
Sunday 13 October 2019 02:01:41 +0900 (0:00:00.176) 0:12:43.386 ********
TASK [network_plugin/kube-router : kube-router | Set cni directory permissions] ************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:42 +0900 (0:00:00.172) 0:12:43.558 ********
TASK [network_plugin/kube-router : kube-router | Copy cni plugins] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:42 +0900 (0:00:00.200) 0:12:43.758 ********
TASK [network_plugin/kube-router : kube-router | Create manifest] **************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:42 +0900 (0:00:00.204) 0:12:43.963 ********
TASK [network_plugin/cloud : Cloud | Set cni directory permissions] ************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:42 +0900 (0:00:00.193) 0:12:44.156 ********
TASK [network_plugin/cloud : Canal | Copy cni plugins] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:42 +0900 (0:00:00.190) 0:12:44.347 ********
TASK [network_plugin/multus : Multus | Copy manifest files] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:43 +0900 (0:00:00.204) 0:12:44.552 ********
TASK [network_plugin/multus : Multus | Copy manifest templates] ****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:43 +0900 (0:00:00.242) 0:12:44.795 ********
RUNNING HANDLER [network_plugin/cilium : restart kubelet] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:43 +0900 (0:00:00.112) 0:12:44.908 ********
RUNNING HANDLER [network_plugin/calico : reset_calico_cni] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:43 +0900 (0:00:00.113) 0:12:45.021 ********
changed: [kubeworker] => {"changed": true, "cmd": ["/bin/true"], "delta": "0:00:00.002230", "end": "2019-10-13 02:01:43.603060", "rc": 0, "start": "2019-10-13 02:01:43.600830", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
changed: [kubemaster] => {"changed": true, "cmd": ["/bin/true"], "delta": "0:00:00.008101", "end": "2019-10-13 02:01:43.982691", "rc": 0, "start": "2019-10-13 02:01:43.974590", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
RUNNING HANDLER [network_plugin/calico : delete 10-calico.conflist] ************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:44 +0900 (0:00:00.718) 0:12:45.739 ********
ok: [kubeworker] => {"changed": false, "path": "/etc/cni/net.d/10-calico.conflist", "state": "absent"}
ok: [kubemaster] => {"changed": false, "path": "/etc/cni/net.d/10-calico.conflist", "state": "absent"}
RUNNING HANDLER [network_plugin/calico : docker | delete calico-node containers] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:01:44 +0900 (0:00:00.357) 0:12:46.097 ********
changed: [kubeworker] => {"changed": true, "cmd": "docker ps -af name=k8s_POD_calico-node* -q | xargs --no-run-if-empty docker rm -f", "delta": "0:00:00.311842", "end": "2019-10-13 02:01:44.958001", "rc": 0, "start": "2019-10-13 02:01:44.646159", "stderr": "", "stderr_lines": [], "stdout": "ec59231bb93e", "stdout_lines": ["ec59231bb93e"]}
changed: [kubemaster] => {"changed": true, "cmd": "docker ps -af name=k8s_POD_calico-node* -q | xargs --no-run-if-empty docker rm -f", "delta": "0:00:00.328238", "end": "2019-10-13 02:01:45.279776", "rc": 0, "start": "2019-10-13 02:01:44.951538", "stderr": "", "stderr_lines": [], "stdout": "a5ffe540a420", "stdout_lines": ["a5ffe540a420"]}
RUNNING HANDLER [network_plugin/calico : containerd | delete calico-node containers] *******************************************************************************************************************************************************
Sunday 13 October 2019 02:01:45 +0900 (0:00:00.638) 0:12:46.735 ********
RUNNING HANDLER [network_plugin/calico : Calico | Reload NetworkManager] *******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:45 +0900 (0:00:00.185) 0:12:46.921 ********
changed: [kubeworker] => {"changed": true, "name": "NetworkManager", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2019-10-12 19:54:27 KST", "ActiveEnterTimestampMonotonic": "4923594", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network-pre.target system.slice systemd-journald.socket dbus.service basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sat 2019-10-12 19:54:26 KST", "AssertTimestampMonotonic": "4878618", "Before": "shutdown.target multi-user.target NetworkManager-wait-online.service network.service network.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.freedesktop.NetworkManager", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "539309282", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2019-10-12 19:54:26 KST", "ConditionTimestampMonotonic": "4878618", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/NetworkManager.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Manager", "DevicePolicy": "auto", "Documentation": "man:NetworkManager(8)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2891", "ExecMainStartTimestamp": "Sat 2019-10-12 19:54:26 KST", "ExecMainStartTimestampMonotonic": "4879101", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/bin/dbus-send ; argv[]=/usr/bin/dbus-send --print-reply --system --type=method_call --dest=org.freedesktop.NetworkManager /org/freedesktop/NetworkManager org.freedesktop.NetworkManager.Reload uint32:0 ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/NetworkManager ; argv[]=/usr/sbin/NetworkManager --no-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/NetworkManager.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "NetworkManager.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2019-10-12 19:54:26 KST", "InactiveExitTimestampMonotonic": "4879126", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "process", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "7257", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7257", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "2891", "MemoryAccounting": "no", "MemoryCurrent": "24576", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "NetworkManager.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "read-only", "ProtectSystem": "yes", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RequiredBy": "NetworkManager-wait-online.service", "Requires": "basic.target", "Restart": "on-failure", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "5", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "network.target system.slice", "WatchdogTimestamp": "Sat 2019-10-12 19:54:27 KST", "WatchdogTimestampMonotonic": "4923434", "WatchdogUSec": "0"}}
changed: [kubemaster] => {"changed": true, "name": "NetworkManager", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2019-10-12 19:54:23 KST", "ActiveEnterTimestampMonotonic": "4299977", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target system.slice dbus.service systemd-journald.socket network-pre.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sat 2019-10-12 19:54:23 KST", "AssertTimestampMonotonic": "4170999", "Before": "network.target NetworkManager-wait-online.service shutdown.target multi-user.target network.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.freedesktop.NetworkManager", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "539309282", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2019-10-12 19:54:23 KST", "ConditionTimestampMonotonic": "4170998", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/NetworkManager.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Manager", "DevicePolicy": "auto", "Documentation": "man:NetworkManager(8)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "768", "ExecMainStartTimestamp": "Sat 2019-10-12 19:54:23 KST", "ExecMainStartTimestampMonotonic": "4172143", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/bin/dbus-send ; argv[]=/usr/bin/dbus-send --print-reply --system --type=method_call --dest=org.freedesktop.NetworkManager /org/freedesktop/NetworkManager org.freedesktop.NetworkManager.Reload uint32:0 ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/NetworkManager ; argv[]=/usr/sbin/NetworkManager --no-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/NetworkManager.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "NetworkManager.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2019-10-12 19:54:23 KST", "InactiveExitTimestampMonotonic": "4172168", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "process", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "7259", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "7259", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "768", "MemoryAccounting": "no", "MemoryCurrent": "3747840", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "NetworkManager.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "read-only", "ProtectSystem": "yes", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RequiredBy": "NetworkManager-wait-online.service", "Requires": "system.slice basic.target", "Restart": "on-failure", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "5", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "network.target", "WatchdogTimestamp": "Sat 2019-10-12 19:54:23 KST", "WatchdogTimestampMonotonic": "4299715", "WatchdogUSec": "0"}}
[WARNING]: Could not match supplied host pattern, ignoring: calico-rr
PLAY [calico-rr] ***************************************************************************************************************************************************************************************************************************
skipping: no hosts matched
PLAY [kube-master[0]] **********************************************************************************************************************************************************************************************************************
TASK [download : prep_download | Set a few facts] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:46 +0900 (0:00:00.760) 0:12:47.681 ********
TASK [download : Set image info command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:46 +0900 (0:00:00.124) 0:12:47.805 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:46 +0900 (0:00:00.100) 0:12:47.905 ********
TASK [download : prep_download | Create staging directory on remote node] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:46 +0900 (0:00:00.112) 0:12:48.017 ********
TASK [download : prep_download | Create local cache for files and images] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:46 +0900 (0:00:00.102) 0:12:48.120 ********
TASK [download : prep_download | On localhost, check if passwordless root is possible] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:01:46 +0900 (0:00:00.135) 0:12:48.255 ********
TASK [download : prep_download | On localhost, check if user has access to docker without using sudo] **************************************************************************************************************************************
Sunday 13 October 2019 02:01:46 +0900 (0:00:00.116) 0:12:48.371 ********
TASK [download : prep_download | Parse the outputs of the previous commands] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:47 +0900 (0:00:00.133) 0:12:48.505 ********
TASK [download : prep_download | Check that local user is in group or can become root] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:01:47 +0900 (0:00:00.108) 0:12:48.614 ********
TASK [download : include_tasks] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:47 +0900 (0:00:00.116) 0:12:48.731 ********
TASK [download : download | Get kubeadm binary and list of required images] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:47 +0900 (0:00:00.085) 0:12:48.817 ********
TASK [download : download | Download files / images] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:47 +0900 (0:00:00.102) 0:12:48.919 ********
TASK [download : download | Sync files / images from ansible host to nodes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:49 +0900 (0:00:01.655) 0:12:50.575 ********
TASK [kubespray-defaults : Configure defaults] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:51 +0900 (0:00:01.869) 0:12:52.445 ********
ok: [kubemaster] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
TASK [kubernetes-apps/rotate_tokens : Rotate Tokens | Get default token name] **************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:51 +0900 (0:00:00.091) 0:12:52.536 ********
ok: [kubemaster] => {"attempts": 1, "changed": false, "cmd": "/usr/local/bin/kubectl --kubeconfig /etc/kubernetes/admin.conf get secrets -o custom-columns=name:{.metadata.name} --no-headers | grep -m1 default-token", "delta": "0:00:00.361587", "end": "2019-10-13 02:01:51.634343", "rc": 0, "start": "2019-10-13 02:01:51.272756", "stderr": "", "stderr_lines": [], "stdout": "default-token-hgt4v", "stdout_lines": ["default-token-hgt4v"]}
TASK [kubernetes-apps/rotate_tokens : Rotate Tokens | Get default token data] **************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:51 +0900 (0:00:00.623) 0:12:53.160 ********
ok: [kubemaster] => {"changed": false, "cmd": ["/usr/local/bin/kubectl", "--kubeconfig", "/etc/kubernetes/admin.conf", "get", "secrets", "default-token-hgt4v", "-ojson"], "delta": "0:00:00.214691", "end": "2019-10-13 02:01:52.141771", "rc": 0, "start": "2019-10-13 02:01:51.927080", "stderr": "", "stderr_lines": [], "stdout": "{\n \"apiVersion\": \"v1\",\n \"data\": {\n \"ca.crt\": \"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFNU1UQXhNakUyTlRrMU4xb1hEVEk1TVRBd09URTJOVGsxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTURPCnpTbTlEVHk0cjlWc0dHOVlJZGVkTE5RdWVYcjloUXJ2bGF4aEdQOGs5KytUN21uYzQ2RW5nWmhISGFmemFBNnkKMXd2UTJ2SEpQMG9QbWEvcitCaTRncGJldDZNMmwvTzB2TXl2Z1N0RXBaYmtWK1d6QTF3NUlTNzRNK1dDNlRlKwpEUmE1NGZsQmNEamhUSHdMRWw4Q3VZMDhxcUFvSGxsT3dtVEFlU3RqeUk4MVVIcndKd1pBZUk0SzQ0SENqVTdPClh3RHJHbklKWWVWdUdTVDBNV0FjYlUzSzZKOVYraFo0dE4vVzVINkRzTnNBczgvR0JaS3Y0QmNtQXJjSEJVMysKeVJwM0o2cnhZMXBMa3lQUFhsNmh0U3R4N0J4TzZCcTVNY3pGNFNjazRhL2pSb3JvYXBZU0ZCeWdxcmdQeWVpSgpuVHVoaGNRblBpdGxPZDUzNHZVQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFGemZHc2V2OWk0NzBEMHhhME9jK0M1TURDY3kKRVoxQkQ2SU9ySVYwdWdTWHFQZkZSakU5bWxUYks1YloyaHJOZjlaSFM4d0FDL0g5THZuNHVHMFVSM3lueStEYgpXQXN6cjBxaGFlQ2x6Q0FRdUZzblo2amx4b3JjVUI1OWRPd0xsdW1PRDZpN29yUUg0WkJZa2JIa3dVbG80cFVOClJHMXNWeVZoWW81d1lBZ2xveVAvQlNMclBYbzFLTGtRMDZLdnAzclhpRkpqZUtMQjlCdlVvVzVubjR3aDI0RTIKSFRXU05IaWxCL1FZTlZUNnc3MHlISFJrTUVIL2x0aS9qRnovR0RVTXlNbjNER09sRkNqK002U3o2M0RoMEU4WApWRk1SdXJ5YjhFeUFOb1JFRlpPbFBIclR3V3JtSk9aYkVJMEFRMmJlalNTNFhxcjBxSTN0eFFjVTFobz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=\",\n \"namespace\": \"ZGVmYXVsdA==\",\n \"token\": \"ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNklsWm5WbmR1WnpocGNIZ3RTbWx4U2pobGJEZG5TV2xtUkZoR1l6SlZkV2ROYVZwbVNEaGZabWRpTkhNaWZRLmV5SnBjM01pT2lKcmRXSmxjbTVsZEdWekwzTmxjblpwWTJWaFkyTnZkVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXVZVzFsYzNCaFkyVWlPaUprWldaaGRXeDBJaXdpYTNWaVpYSnVaWFJsY3k1cGJ5OXpaWEoyYVdObFlXTmpiM1Z1ZEM5elpXTnlaWFF1Ym1GdFpTSTZJbVJsWm1GMWJIUXRkRzlyWlc0dGFHZDBOSFlpTENKcmRXSmxjbTVsZEdWekxtbHZMM05sY25acFkyVmhZMk52ZFc1MEwzTmxjblpwWTJVdFlXTmpiM1Z1ZEM1dVlXMWxJam9pWkdWbVlYVnNkQ0lzSW10MVltVnlibVYwWlhNdWFXOHZjMlZ5ZG1salpXRmpZMjkxYm5RdmMyVnlkbWxqWlMxaFkyTnZkVzUwTG5WcFpDSTZJbVJrTm1FNE9URXdMVGRpTXpndE5EazFNQzFpTnpCbUxUVm1aRGMwWmpCak9URTVNU0lzSW5OMVlpSTZJbk41YzNSbGJUcHpaWEoyYVdObFlXTmpiM1Z1ZERwa1pXWmhkV3gwT21SbFptRjFiSFFpZlEuY3FEUWlRNUFyN3QzSlptQ0YtSG1CZ1Z2Rlotb3VPWTY2MlpwLW9MQWNnY2RDSTVqQ2ZPYmJDcm1tNG5GblRqQXBRVy1yZXowUzFJWmhtU0tfVVlyUUlnOEFUVFlha3JjY1ZiTkltZkpvN3FQWmIwdEp2dFlvR2ZoTnNVeDl2UGQxdDloaXJzU3Nqc0Z2ajJXX1RwT2RoVC1zdElOd2dzTVBYckJUMEwyRm54dzBkN3lYTzB6ZmZXclZPTmdFZnEwbE85ZHlxWWVuU19NZllLT2dEdG5XazRJWUREMlJWRWhpM3d3UnpzUkpsRWRWTlhhQVVvQjdQR0wyQUY4QkdJbFJLVllxTVR1bjBxaVlwMTRWWVdVMHZhek9pcXJ3SlZIQkpRcGdOcGNpNW5vRFVkbE51VmlBU0FRa1FoSnhCOUU0eDBIb1dFQVNYc2Zfc0RQV1RXYmln\"\n },\n \"kind\": \"Secret\",\n \"metadata\": {\n \"annotations\": {\n \"kubernetes.io/service-account.name\": \"default\",\n \"kubernetes.io/service-account.uid\": \"dd6a8910-7b38-4950-b70f-5fd74f0c9191\"\n },\n \"creationTimestamp\": \"2019-10-12T17:00:34Z\",\n \"name\": \"default-token-hgt4v\",\n \"namespace\": \"default\",\n \"resourceVersion\": \"309\",\n \"selfLink\": \"/api/v1/namespaces/default/secrets/default-token-hgt4v\",\n \"uid\": \"2a1ef9ff-f20b-4077-9884-236151760a9e\"\n },\n \"type\": \"kubernetes.io/service-account-token\"\n}", "stdout_lines": ["{", " \"apiVersion\": \"v1\",", " \"data\": {", " \"ca.crt\": \"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFNU1UQXhNakUyTlRrMU4xb1hEVEk1TVRBd09URTJOVGsxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTURPCnpTbTlEVHk0cjlWc0dHOVlJZGVkTE5RdWVYcjloUXJ2bGF4aEdQOGs5KytUN21uYzQ2RW5nWmhISGFmemFBNnkKMXd2UTJ2SEpQMG9QbWEvcitCaTRncGJldDZNMmwvTzB2TXl2Z1N0RXBaYmtWK1d6QTF3NUlTNzRNK1dDNlRlKwpEUmE1NGZsQmNEamhUSHdMRWw4Q3VZMDhxcUFvSGxsT3dtVEFlU3RqeUk4MVVIcndKd1pBZUk0SzQ0SENqVTdPClh3RHJHbklKWWVWdUdTVDBNV0FjYlUzSzZKOVYraFo0dE4vVzVINkRzTnNBczgvR0JaS3Y0QmNtQXJjSEJVMysKeVJwM0o2cnhZMXBMa3lQUFhsNmh0U3R4N0J4TzZCcTVNY3pGNFNjazRhL2pSb3JvYXBZU0ZCeWdxcmdQeWVpSgpuVHVoaGNRblBpdGxPZDUzNHZVQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFGemZHc2V2OWk0NzBEMHhhME9jK0M1TURDY3kKRVoxQkQ2SU9ySVYwdWdTWHFQZkZSakU5bWxUYks1YloyaHJOZjlaSFM4d0FDL0g5THZuNHVHMFVSM3lueStEYgpXQXN6cjBxaGFlQ2x6Q0FRdUZzblo2amx4b3JjVUI1OWRPd0xsdW1PRDZpN29yUUg0WkJZa2JIa3dVbG80cFVOClJHMXNWeVZoWW81d1lBZ2xveVAvQlNMclBYbzFLTGtRMDZLdnAzclhpRkpqZUtMQjlCdlVvVzVubjR3aDI0RTIKSFRXU05IaWxCL1FZTlZUNnc3MHlISFJrTUVIL2x0aS9qRnovR0RVTXlNbjNER09sRkNqK002U3o2M0RoMEU4WApWRk1SdXJ5YjhFeUFOb1JFRlpPbFBIclR3V3JtSk9aYkVJMEFRMmJlalNTNFhxcjBxSTN0eFFjVTFobz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=\",", " \"namespace\": \"ZGVmYXVsdA==\",", " \"token\": \"ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNklsWm5WbmR1WnpocGNIZ3RTbWx4U2pobGJEZG5TV2xtUkZoR1l6SlZkV2ROYVZwbVNEaGZabWRpTkhNaWZRLmV5SnBjM01pT2lKcmRXSmxjbTVsZEdWekwzTmxjblpwWTJWaFkyTnZkVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXVZVzFsYzNCaFkyVWlPaUprWldaaGRXeDBJaXdpYTNWaVpYSnVaWFJsY3k1cGJ5OXpaWEoyYVdObFlXTmpiM1Z1ZEM5elpXTnlaWFF1Ym1GdFpTSTZJbVJsWm1GMWJIUXRkRzlyWlc0dGFHZDBOSFlpTENKcmRXSmxjbTVsZEdWekxtbHZMM05sY25acFkyVmhZMk52ZFc1MEwzTmxjblpwWTJVdFlXTmpiM1Z1ZEM1dVlXMWxJam9pWkdWbVlYVnNkQ0lzSW10MVltVnlibVYwWlhNdWFXOHZjMlZ5ZG1salpXRmpZMjkxYm5RdmMyVnlkbWxqWlMxaFkyTnZkVzUwTG5WcFpDSTZJbVJrTm1FNE9URXdMVGRpTXpndE5EazFNQzFpTnpCbUxUVm1aRGMwWmpCak9URTVNU0lzSW5OMVlpSTZJbk41YzNSbGJUcHpaWEoyYVdObFlXTmpiM1Z1ZERwa1pXWmhkV3gwT21SbFptRjFiSFFpZlEuY3FEUWlRNUFyN3QzSlptQ0YtSG1CZ1Z2Rlotb3VPWTY2MlpwLW9MQWNnY2RDSTVqQ2ZPYmJDcm1tNG5GblRqQXBRVy1yZXowUzFJWmhtU0tfVVlyUUlnOEFUVFlha3JjY1ZiTkltZkpvN3FQWmIwdEp2dFlvR2ZoTnNVeDl2UGQxdDloaXJzU3Nqc0Z2ajJXX1RwT2RoVC1zdElOd2dzTVBYckJUMEwyRm54dzBkN3lYTzB6ZmZXclZPTmdFZnEwbE85ZHlxWWVuU19NZllLT2dEdG5XazRJWUREMlJWRWhpM3d3UnpzUkpsRWRWTlhhQVVvQjdQR0wyQUY4QkdJbFJLVllxTVR1bjBxaVlwMTRWWVdVMHZhek9pcXJ3SlZIQkpRcGdOcGNpNW5vRFVkbE51VmlBU0FRa1FoSnhCOUU0eDBIb1dFQVNYc2Zfc0RQV1RXYmln\"", " },", " \"kind\": \"Secret\",", " \"metadata\": {", " \"annotations\": {", " \"kubernetes.io/service-account.name\": \"default\",", " \"kubernetes.io/service-account.uid\": \"dd6a8910-7b38-4950-b70f-5fd74f0c9191\"", " },", " \"creationTimestamp\": \"2019-10-12T17:00:34Z\",", " \"name\": \"default-token-hgt4v\",", " \"namespace\": \"default\",", " \"resourceVersion\": \"309\",", " \"selfLink\": \"/api/v1/namespaces/default/secrets/default-token-hgt4v\",", " \"uid\": \"2a1ef9ff-f20b-4077-9884-236151760a9e\"", " },", " \"type\": \"kubernetes.io/service-account-token\"", "}"]}
TASK [kubernetes-apps/rotate_tokens : Rotate Tokens | Test if default certificate is expired] **********************************************************************************************************************************************
Sunday 13 October 2019 02:01:52 +0900 (0:00:00.503) 0:12:53.663 ********
ok: [kubemaster] => {"cache_control": "no-cache, private", "changed": false, "connection": "close", "content": "{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"nodes is forbidden: User \\\"system:serviceaccount:default:default\\\" cannot list resource \\\"nodes\\\" in API group \\\"\\\" at the cluster scope\",\"reason\":\"Forbidden\",\"details\":{\"kind\":\"nodes\"},\"code\":403}\n", "content_length": "277", "content_type": "application/json", "date": "Sat, 12 Oct 2019 17:01:52 GMT", "elapsed": 0, "failed_when_result": false, "json": {"apiVersion": "v1", "code": 403, "details": {"kind": "nodes"}, "kind": "Status", "message": "nodes is forbidden: User \"system:serviceaccount:default:default\" cannot list resource \"nodes\" in API group \"\" at the cluster scope", "metadata": {}, "reason": "Forbidden", "status": "Failure"}, "msg": "Status code was 403 and not [200]: HTTP Error 403: Forbidden", "redirected": false, "status": 403, "url": "https://10.233.0.1/api/v1/nodes", "x_content_type_options": "nosniff"}
TASK [kubernetes-apps/rotate_tokens : Rotate Tokens | Determine if certificate is expired] *************************************************************************************************************************************************
Sunday 13 October 2019 02:01:52 +0900 (0:00:00.412) 0:12:54.076 ********
ok: [kubemaster] => {"ansible_facts": {"needs_rotation": false}, "changed": false}
TASK [kubernetes-apps/rotate_tokens : Rotate Tokens | Get all serviceaccount tokens to expire] *********************************************************************************************************************************************
Sunday 13 October 2019 02:01:52 +0900 (0:00:00.120) 0:12:54.196 ********
TASK [kubernetes-apps/rotate_tokens : Rotate Tokens | Delete expired tokens] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:52 +0900 (0:00:00.108) 0:12:54.305 ********
TASK [kubernetes-apps/rotate_tokens : Rotate Tokens | Delete pods in system namespace] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:01:52 +0900 (0:00:00.081) 0:12:54.387 ********
TASK [win_nodes/kubernetes_patch : Ensure that user manifests directory exists] ************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:53 +0900 (0:00:00.082) 0:12:54.470 ********
changed: [kubemaster] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/kube-manifests/kubernetes", "secontext": "unconfined_u:object_r:admin_home_t:s0", "size": 6, "state": "directory", "uid": 0}
TASK [win_nodes/kubernetes_patch : Copy kube-proxy daemonset nodeselector patch] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:01:53 +0900 (0:00:00.266) 0:12:54.737 ********
changed: [kubemaster] => {"changed": true, "checksum": "7a623f3b1dbed6c45b190d209c0ff765511fe176", "dest": "/root/kube-manifests/nodeselector-os-linux-patch.json", "gid": 0, "group": "root", "md5sum": "cadd7f8620482e830875188201fc2f3b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:admin_home_t:s0", "size": 81, "src": "/root/.ansible/tmp/ansible-tmp-1570899713.3456807-97864253328106/source", "state": "file", "uid": 0}
TASK [win_nodes/kubernetes_patch : Check current nodeselector for kube-proxy daemonset] ****************************************************************************************************************************************************
Sunday 13 October 2019 02:01:53 +0900 (0:00:00.564) 0:12:55.301 ********
changed: [kubemaster] => {"changed": true, "cmd": "/usr/local/bin/kubectl --kubeconfig /etc/kubernetes/admin.conf get ds kube-proxy --namespace=kube-system -o jsonpath='{.spec.template.spec.nodeSelector.beta.kubernetes.io/os}'", "delta": "0:00:00.140768", "end": "2019-10-13 02:01:54.182657", "rc": 0, "start": "2019-10-13 02:01:54.041889", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
TASK [win_nodes/kubernetes_patch : Apply nodeselector patch for kube-proxy daemonset] ******************************************************************************************************************************************************
Sunday 13 October 2019 02:01:54 +0900 (0:00:00.390) 0:12:55.692 ********
changed: [kubemaster] => {"changed": true, "cmd": "/usr/local/bin/kubectl --kubeconfig /etc/kubernetes/admin.conf patch ds kube-proxy --namespace=kube-system --type=strategic -p \"$(cat nodeselector-os-linux-patch.json)\"", "delta": "0:00:00.174232", "end": "2019-10-13 02:01:54.629286", "rc": 0, "start": "2019-10-13 02:01:54.455054", "stderr": "", "stderr_lines": [], "stdout": "daemonset.apps/kube-proxy patched (no change)", "stdout_lines": ["daemonset.apps/kube-proxy patched (no change)"]}
TASK [win_nodes/kubernetes_patch : debug] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:54 +0900 (0:00:00.457) 0:12:56.150 ********
ok: [kubemaster] => {
"msg": [
"daemonset.apps/kube-proxy patched (no change)"
]
}
TASK [win_nodes/kubernetes_patch : debug] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:54 +0900 (0:00:00.114) 0:12:56.264 ********
ok: [kubemaster] => {
"msg": []
}
PLAY [kube-master] *************************************************************************************************************************************************************************************************************************
TASK [download : prep_download | Set a few facts] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:55 +0900 (0:00:00.202) 0:12:56.467 ********
TASK [download : Set image info command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:55 +0900 (0:00:00.078) 0:12:56.545 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:55 +0900 (0:00:00.109) 0:12:56.655 ********
TASK [download : prep_download | Create staging directory on remote node] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:55 +0900 (0:00:00.080) 0:12:56.735 ********
TASK [download : prep_download | Create local cache for files and images] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:55 +0900 (0:00:00.092) 0:12:56.828 ********
TASK [download : prep_download | On localhost, check if passwordless root is possible] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:01:55 +0900 (0:00:00.116) 0:12:56.944 ********
TASK [download : prep_download | On localhost, check if user has access to docker without using sudo] **************************************************************************************************************************************
Sunday 13 October 2019 02:01:55 +0900 (0:00:00.106) 0:12:57.051 ********
TASK [download : prep_download | Parse the outputs of the previous commands] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:55 +0900 (0:00:00.093) 0:12:57.145 ********
TASK [download : prep_download | Check that local user is in group or can become root] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:01:55 +0900 (0:00:00.083) 0:12:57.229 ********
TASK [download : include_tasks] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:55 +0900 (0:00:00.079) 0:12:57.309 ********
TASK [download : download | Get kubeadm binary and list of required images] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:55 +0900 (0:00:00.075) 0:12:57.384 ********
TASK [download : download | Download files / images] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:56 +0900 (0:00:00.075) 0:12:57.460 ********
TASK [download : download | Sync files / images from ansible host to nodes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:57 +0900 (0:00:01.517) 0:12:58.977 ********
TASK [kubespray-defaults : Configure defaults] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:59 +0900 (0:00:01.457) 0:13:00.435 ********
ok: [kubemaster] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
TASK [kubernetes-apps/network_plugin/cilium : Cilium | Start Resources] ********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:59 +0900 (0:00:00.087) 0:13:00.523 ********
TASK [kubernetes-apps/network_plugin/cilium : Cilium | Wait for pods to run] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:59 +0900 (0:00:00.167) 0:13:00.691 ********
TASK [kubernetes-apps/network_plugin/calico : calico upgrade complete] *********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:59 +0900 (0:00:00.095) 0:13:00.786 ********
TASK [kubernetes-apps/network_plugin/canal : Canal | Start Resources] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:59 +0900 (0:00:00.091) 0:13:00.878 ********
TASK [kubernetes-apps/network_plugin/flannel : Flannel | Start Resources] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:01:59 +0900 (0:00:00.305) 0:13:01.184 ********
TASK [kubernetes-apps/network_plugin/flannel : Flannel | Wait for flannel subnet.env file presence] ****************************************************************************************************************************************
Sunday 13 October 2019 02:01:59 +0900 (0:00:00.150) 0:13:01.334 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Create Kubernetes resources] ********************************************************************************************************************************************************
Sunday 13 October 2019 02:01:59 +0900 (0:00:00.091) 0:13:01.425 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Wait for netmaster] *****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:00 +0900 (0:00:00.108) 0:13:01.534 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Get global configuration] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:00 +0900 (0:00:00.115) 0:13:01.650 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Set contiv_global_config] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:00 +0900 (0:00:00.106) 0:13:01.756 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Set global forwarding mode] *********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:00 +0900 (0:00:00.100) 0:13:01.857 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Set global fabric mode] *************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:00 +0900 (0:00:00.094) 0:13:01.952 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Set peer hostname] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:00 +0900 (0:00:00.117) 0:13:02.069 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Get BGP configuration] **************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:00 +0900 (0:00:00.141) 0:13:02.211 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Configure peering with router(s)] ***************************************************************************************************************************************************
Sunday 13 October 2019 02:02:00 +0900 (0:00:00.098) 0:13:02.309 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Get existing networks] **************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:00 +0900 (0:00:00.104) 0:13:02.413 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Create networks] ********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:01 +0900 (0:00:00.082) 0:13:02.497 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Check if default group exists] ******************************************************************************************************************************************************
Sunday 13 October 2019 02:02:01 +0900 (0:00:00.094) 0:13:02.591 ********
TASK [kubernetes-apps/network_plugin/contiv : Contiv | Create default group] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:01 +0900 (0:00:00.086) 0:13:02.678 ********
TASK [kubernetes-apps/network_plugin/cni : CNI | make sure /opt/cni/bin exists] ************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:01 +0900 (0:00:00.105) 0:13:02.783 ********
TASK [kubernetes-apps/network_plugin/cni : CNI | Copy cni plugins] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:01 +0900 (0:00:00.084) 0:13:02.868 ********
TASK [kubernetes-apps/network_plugin/kube-ovn : Kube-OVN | Start Resources] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:01 +0900 (0:00:00.090) 0:13:02.959 ********
TASK [kubernetes-apps/network_plugin/weave : Weave | Start Resources] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:01 +0900 (0:00:00.185) 0:13:03.144 ********
TASK [kubernetes-apps/network_plugin/weave : Weave | Wait for Weave to become available] ***************************************************************************************************************************************************
Sunday 13 October 2019 02:02:01 +0900 (0:00:00.113) 0:13:03.258 ********
TASK [kubernetes-apps/network_plugin/kube-router : kube-router | Start Resources] **********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:01 +0900 (0:00:00.098) 0:13:03.356 ********
TASK [kubernetes-apps/network_plugin/kube-router : kube-router | Wait for kube-router pods to be ready] ************************************************************************************************************************************
Sunday 13 October 2019 02:02:02 +0900 (0:00:00.089) 0:13:03.446 ********
TASK [kubernetes-apps/network_plugin/multus : Multus | Start resources] ********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:02 +0900 (0:00:00.089) 0:13:03.535 ********
TASK [policy_controller/calico : Set cert dir] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:02 +0900 (0:00:00.157) 0:13:03.692 ********
TASK [policy_controller/calico : Create calico-kube-controllers manifests] *****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:02 +0900 (0:00:00.121) 0:13:03.814 ********
changed: [kubemaster] => (item={'name': 'calico-kube-controllers', 'file': 'calico-kube-controllers.yml', 'type': 'deployment'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "59188dc2018b5663d24bd3c181404c3467ea29f5", "dest": "/etc/kubernetes/calico-kube-controllers.yml", "gid": 0, "group": "root", "item": {"file": "calico-kube-controllers.yml", "name": "calico-kube-controllers", "type": "deployment"}, "md5sum": "1f35be0fdfefa35328feb3bb4d0384bb", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 1950, "src": "/root/.ansible/tmp/ansible-tmp-1570899722.4726114-245799243004399/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'calico-kube-controllers', 'file': 'calico-kube-sa.yml', 'type': 'sa'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "9b599bf59cbdd85628410da40ac05a01f7985d1b", "dest": "/etc/kubernetes/calico-kube-sa.yml", "gid": 0, "group": "root", "item": {"file": "calico-kube-sa.yml", "name": "calico-kube-controllers", "type": "sa"}, "md5sum": "a14e9c090943c566dddb4a0f25e3e03b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 159, "src": "/root/.ansible/tmp/ansible-tmp-1570899723.5314178-175855388011188/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'calico-kube-controllers', 'file': 'calico-kube-cr.yml', 'type': 'clusterrole'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "328b854512a0ae28be6492554a7fa187e5e5cc14", "dest": "/etc/kubernetes/calico-kube-cr.yml", "gid": 0, "group": "root", "item": {"file": "calico-kube-cr.yml", "name": "calico-kube-controllers", "type": "clusterrole"}, "md5sum": "f8b36b96ee79b9f9ca16c2758dfa1ab1", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 446, "src": "/root/.ansible/tmp/ansible-tmp-1570899724.2415056-5930670177617/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'calico-kube-controllers', 'file': 'calico-kube-crb.yml', 'type': 'clusterrolebinding'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "a0ab18747c1c4701bf329d27e1442e006709c1fb", "dest": "/etc/kubernetes/calico-kube-crb.yml", "gid": 0, "group": "root", "item": {"file": "calico-kube-crb.yml", "name": "calico-kube-controllers", "type": "clusterrolebinding"}, "md5sum": "56279f57f711565a8a3e4411232f51d1", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 306, "src": "/root/.ansible/tmp/ansible-tmp-1570899724.845222-168418465748058/source", "state": "file", "uid": 0}
TASK [policy_controller/calico : Start of Calico kube controllers] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:05 +0900 (0:00:02.994) 0:13:06.809 ********
ok: [kubemaster] => (item=calico-kube-controllers.yml) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "59188dc2018b5663d24bd3c181404c3467ea29f5", "dest": "/etc/kubernetes/calico-kube-controllers.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "calico-kube-controllers.yml.j2", "attributes": null, "backup": false, "checksum": "59188dc2018b5663d24bd3c181404c3467ea29f5", "content": null, "delimiter": null, "dest": "/etc/kubernetes/calico-kube-controllers.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899722.4726114-245799243004399/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "calico-kube-controllers.yml", "name": "calico-kube-controllers", "type": "deployment"}, "md5sum": "1f35be0fdfefa35328feb3bb4d0384bb", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 1950, "src": "/root/.ansible/tmp/ansible-tmp-1570899722.4726114-245799243004399/source", "state": "file", "uid": 0}, "msg": "success: deployment.apps/calico-kube-controllers created"}
ok: [kubemaster] => (item=calico-kube-sa.yml) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "9b599bf59cbdd85628410da40ac05a01f7985d1b", "dest": "/etc/kubernetes/calico-kube-sa.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "calico-kube-sa.yml.j2", "attributes": null, "backup": false, "checksum": "9b599bf59cbdd85628410da40ac05a01f7985d1b", "content": null, "delimiter": null, "dest": "/etc/kubernetes/calico-kube-sa.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899723.5314178-175855388011188/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "calico-kube-sa.yml", "name": "calico-kube-controllers", "type": "sa"}, "md5sum": "a14e9c090943c566dddb4a0f25e3e03b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 159, "src": "/root/.ansible/tmp/ansible-tmp-1570899723.5314178-175855388011188/source", "state": "file", "uid": 0}, "msg": "success: serviceaccount/calico-kube-controllers created"}
ok: [kubemaster] => (item=calico-kube-cr.yml) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "328b854512a0ae28be6492554a7fa187e5e5cc14", "dest": "/etc/kubernetes/calico-kube-cr.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "calico-kube-cr.yml.j2", "attributes": null, "backup": false, "checksum": "328b854512a0ae28be6492554a7fa187e5e5cc14", "content": null, "delimiter": null, "dest": "/etc/kubernetes/calico-kube-cr.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899724.2415056-5930670177617/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "calico-kube-cr.yml", "name": "calico-kube-controllers", "type": "clusterrole"}, "md5sum": "f8b36b96ee79b9f9ca16c2758dfa1ab1", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 446, "src": "/root/.ansible/tmp/ansible-tmp-1570899724.2415056-5930670177617/source", "state": "file", "uid": 0}, "msg": "success: clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created"}
ok: [kubemaster] => (item=calico-kube-crb.yml) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "a0ab18747c1c4701bf329d27e1442e006709c1fb", "dest": "/etc/kubernetes/calico-kube-crb.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "calico-kube-crb.yml.j2", "attributes": null, "backup": false, "checksum": "a0ab18747c1c4701bf329d27e1442e006709c1fb", "content": null, "delimiter": null, "dest": "/etc/kubernetes/calico-kube-crb.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899724.845222-168418465748058/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "calico-kube-crb.yml", "name": "calico-kube-controllers", "type": "clusterrolebinding"}, "md5sum": "56279f57f711565a8a3e4411232f51d1", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 306, "src": "/root/.ansible/tmp/ansible-tmp-1570899724.845222-168418465748058/source", "state": "file", "uid": 0}, "msg": "success: clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created"}
TASK [policy_controller/calico : Set cert dir] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:07 +0900 (0:00:02.455) 0:13:09.264 ********
TASK [policy_controller/calico : Create calico-kube-controllers manifests] *****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:07 +0900 (0:00:00.127) 0:13:09.392 ********
TASK [policy_controller/calico : Start of Calico kube controllers] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:08 +0900 (0:00:00.140) 0:13:09.532 ********
TASK [kubernetes-apps/ingress_controller/ingress_nginx : NGINX Ingress Controller | Create addon dir] **************************************************************************************************************************************
Sunday 13 October 2019 02:02:08 +0900 (0:00:00.140) 0:13:09.673 ********
TASK [kubernetes-apps/ingress_controller/ingress_nginx : NGINX Ingress Controller | Templates list] ****************************************************************************************************************************************
Sunday 13 October 2019 02:02:08 +0900 (0:00:00.073) 0:13:09.747 ********
TASK [kubernetes-apps/ingress_controller/ingress_nginx : NGINX Ingress Controller | Append extra templates to NGINX Ingress Templates list for PodSecurityPolicy] **************************************************************************
Sunday 13 October 2019 02:02:08 +0900 (0:00:00.136) 0:13:09.884 ********
TASK [kubernetes-apps/ingress_controller/ingress_nginx : NGINX Ingress Controller | Create manifests] **************************************************************************************************************************************
Sunday 13 October 2019 02:02:08 +0900 (0:00:00.083) 0:13:09.967 ********
TASK [kubernetes-apps/ingress_controller/ingress_nginx : NGINX Ingress Controller | Apply manifests] ***************************************************************************************************************************************
Sunday 13 October 2019 02:02:08 +0900 (0:00:00.093) 0:13:10.060 ********
TASK [kubernetes-apps/ingress_controller/cert_manager : Cert Manager | Remove legacy addon dir and manifests] ******************************************************************************************************************************
Sunday 13 October 2019 02:02:08 +0900 (0:00:00.077) 0:13:10.138 ********
TASK [kubernetes-apps/ingress_controller/cert_manager : Cert Manager | Remove legacy namespace] ********************************************************************************************************************************************
Sunday 13 October 2019 02:02:08 +0900 (0:00:00.093) 0:13:10.232 ********
TASK [kubernetes-apps/ingress_controller/cert_manager : Cert Manager | Create addon dir] ***************************************************************************************************************************************************
Sunday 13 October 2019 02:02:08 +0900 (0:00:00.072) 0:13:10.305 ********
TASK [kubernetes-apps/ingress_controller/cert_manager : Cert Manager | Create manifests] ***************************************************************************************************************************************************
Sunday 13 October 2019 02:02:08 +0900 (0:00:00.083) 0:13:10.389 ********
TASK [kubernetes-apps/ingress_controller/cert_manager : Cert Manager | Apply manifests] ****************************************************************************************************************************************************
Sunday 13 October 2019 02:02:09 +0900 (0:00:00.207) 0:13:10.596 ********
TASK [kubernetes-apps/external_provisioner/local_volume_provisioner : Local Volume Provisioner | Ensure base dir is created on all hosts] **************************************************************************************************
Sunday 13 October 2019 02:02:09 +0900 (0:00:00.329) 0:13:10.926 ********
TASK [kubernetes-apps/external_provisioner/local_volume_provisioner : Local Volume Provisioner | Create addon dir] *************************************************************************************************************************
Sunday 13 October 2019 02:02:09 +0900 (0:00:00.115) 0:13:11.041 ********
TASK [kubernetes-apps/external_provisioner/local_volume_provisioner : Local Volume Provisioner | Templates list] ***************************************************************************************************************************
Sunday 13 October 2019 02:02:09 +0900 (0:00:00.099) 0:13:11.141 ********
TASK [kubernetes-apps/external_provisioner/local_volume_provisioner : Local Volume Provisioner | Insert extra templates to Local Volume Provisioner templates list for PodSecurityPolicy] **************************************************
Sunday 13 October 2019 02:02:09 +0900 (0:00:00.102) 0:13:11.244 ********
TASK [kubernetes-apps/external_provisioner/local_volume_provisioner : Local Volume Provisioner | Create manifests] *************************************************************************************************************************
Sunday 13 October 2019 02:02:09 +0900 (0:00:00.087) 0:13:11.331 ********
TASK [kubernetes-apps/external_provisioner/local_volume_provisioner : Local Volume Provisioner | Apply manifests] **************************************************************************************************************************
Sunday 13 October 2019 02:02:09 +0900 (0:00:00.090) 0:13:11.421 ********
TASK [kubernetes-apps/external_provisioner/cephfs_provisioner : CephFS Provisioner | Remove legacy addon dir and manifests] ****************************************************************************************************************
Sunday 13 October 2019 02:02:10 +0900 (0:00:00.095) 0:13:11.517 ********
TASK [kubernetes-apps/external_provisioner/cephfs_provisioner : CephFS Provisioner | Remove legacy namespace] ******************************************************************************************************************************
Sunday 13 October 2019 02:02:10 +0900 (0:00:00.086) 0:13:11.603 ********
TASK [kubernetes-apps/external_provisioner/cephfs_provisioner : CephFS Provisioner | Remove legacy storageclass] ***************************************************************************************************************************
Sunday 13 October 2019 02:02:10 +0900 (0:00:00.079) 0:13:11.683 ********
TASK [kubernetes-apps/external_provisioner/cephfs_provisioner : CephFS Provisioner | Create addon dir] *************************************************************************************************************************************
Sunday 13 October 2019 02:02:10 +0900 (0:00:00.074) 0:13:11.757 ********
TASK [kubernetes-apps/external_provisioner/cephfs_provisioner : CephFS Provisioner | Templates list] ***************************************************************************************************************************************
Sunday 13 October 2019 02:02:10 +0900 (0:00:00.077) 0:13:11.835 ********
TASK [kubernetes-apps/external_provisioner/cephfs_provisioner : CephFS Provisioner | Append extra templates to CephFS Provisioner Templates list for PodSecurityPolicy] ********************************************************************
Sunday 13 October 2019 02:02:10 +0900 (0:00:00.075) 0:13:11.910 ********
TASK [kubernetes-apps/external_provisioner/cephfs_provisioner : CephFS Provisioner | Create manifests] *************************************************************************************************************************************
Sunday 13 October 2019 02:02:10 +0900 (0:00:00.088) 0:13:11.999 ********
TASK [kubernetes-apps/external_provisioner/cephfs_provisioner : CephFS Provisioner | Apply manifests] **************************************************************************************************************************************
Sunday 13 October 2019 02:02:10 +0900 (0:00:00.069) 0:13:12.069 ********
TASK [kubernetes-apps/external_provisioner/rbd_provisioner : RBD Provisioner | Remove legacy addon dir and manifests] **********************************************************************************************************************
Sunday 13 October 2019 02:02:10 +0900 (0:00:00.072) 0:13:12.142 ********
TASK [kubernetes-apps/external_provisioner/rbd_provisioner : RBD Provisioner | Remove legacy namespace] ************************************************************************************************************************************
Sunday 13 October 2019 02:02:10 +0900 (0:00:00.080) 0:13:12.222 ********
TASK [kubernetes-apps/external_provisioner/rbd_provisioner : RBD Provisioner | Remove legacy storageclass] *********************************************************************************************************************************
Sunday 13 October 2019 02:02:10 +0900 (0:00:00.082) 0:13:12.305 ********
TASK [kubernetes-apps/external_provisioner/rbd_provisioner : RBD Provisioner | Create addon dir] *******************************************************************************************************************************************
Sunday 13 October 2019 02:02:10 +0900 (0:00:00.075) 0:13:12.380 ********
TASK [kubernetes-apps/external_provisioner/rbd_provisioner : RBD Provisioner | Templates list] *********************************************************************************************************************************************
Sunday 13 October 2019 02:02:11 +0900 (0:00:00.082) 0:13:12.463 ********
TASK [kubernetes-apps/external_provisioner/rbd_provisioner : RBD Provisioner | Append extra templates to RBD Provisioner Templates list for PodSecurityPolicy] *****************************************************************************
Sunday 13 October 2019 02:02:11 +0900 (0:00:00.079) 0:13:12.542 ********
TASK [kubernetes-apps/external_provisioner/rbd_provisioner : RBD Provisioner | Create manifests] *******************************************************************************************************************************************
Sunday 13 October 2019 02:02:11 +0900 (0:00:00.082) 0:13:12.624 ********
TASK [kubernetes-apps/external_provisioner/rbd_provisioner : RBD Provisioner | Apply manifests] ********************************************************************************************************************************************
Sunday 13 October 2019 02:02:11 +0900 (0:00:00.073) 0:13:12.698 ********
TASK [kubernetes-apps/external_provisioner/local_path_provisioner : Local Path Provisioner | Create addon dir] *****************************************************************************************************************************
Sunday 13 October 2019 02:02:11 +0900 (0:00:00.079) 0:13:12.777 ********
TASK [kubernetes-apps/external_provisioner/local_path_provisioner : Local Path Provisioner | Create claim root dir] ************************************************************************************************************************
Sunday 13 October 2019 02:02:11 +0900 (0:00:00.075) 0:13:12.853 ********
TASK [kubernetes-apps/external_provisioner/local_path_provisioner : Local Path Provisioner | Render Template] ******************************************************************************************************************************
Sunday 13 October 2019 02:02:11 +0900 (0:00:00.090) 0:13:12.943 ********
TASK [kubernetes-apps/external_provisioner/local_path_provisioner : Local Path Provisioner | Insert extra templates to Local Path Provisioner templates list for PodSecurityPolicy] ********************************************************
Sunday 13 October 2019 02:02:11 +0900 (0:00:00.087) 0:13:13.031 ********
TASK [kubernetes-apps/external_provisioner/local_path_provisioner : Local Path Provisioner | Create manifests] *****************************************************************************************************************************
Sunday 13 October 2019 02:02:11 +0900 (0:00:00.080) 0:13:13.112 ********
TASK [kubernetes-apps/external_provisioner/local_path_provisioner : Local Path Provisioner | Apply manifests] ******************************************************************************************************************************
Sunday 13 October 2019 02:02:11 +0900 (0:00:00.082) 0:13:13.195 ********
PLAY [kube-master] *************************************************************************************************************************************************************************************************************************
TASK [download : prep_download | Set a few facts] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:11 +0900 (0:00:00.145) 0:13:13.341 ********
TASK [download : Set image info command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:12 +0900 (0:00:00.093) 0:13:13.434 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:12 +0900 (0:00:00.079) 0:13:13.513 ********
TASK [download : prep_download | Create staging directory on remote node] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:12 +0900 (0:00:00.097) 0:13:13.611 ********
TASK [download : prep_download | Create local cache for files and images] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:12 +0900 (0:00:00.086) 0:13:13.698 ********
TASK [download : prep_download | On localhost, check if passwordless root is possible] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:02:12 +0900 (0:00:00.088) 0:13:13.786 ********
TASK [download : prep_download | On localhost, check if user has access to docker without using sudo] **************************************************************************************************************************************
Sunday 13 October 2019 02:02:12 +0900 (0:00:00.081) 0:13:13.867 ********
TASK [download : prep_download | Parse the outputs of the previous commands] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:12 +0900 (0:00:00.075) 0:13:13.942 ********
TASK [download : prep_download | Check that local user is in group or can become root] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:02:12 +0900 (0:00:00.083) 0:13:14.026 ********
TASK [download : include_tasks] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:12 +0900 (0:00:00.079) 0:13:14.105 ********
TASK [download : download | Get kubeadm binary and list of required images] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:12 +0900 (0:00:00.080) 0:13:14.186 ********
TASK [download : download | Download files / images] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:12 +0900 (0:00:00.085) 0:13:14.272 ********
TASK [download : download | Sync files / images from ansible host to nodes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:14 +0900 (0:00:01.533) 0:13:15.805 ********
TASK [kubespray-defaults : Configure defaults] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:15 +0900 (0:00:01.481) 0:13:17.287 ********
ok: [kubemaster] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
TASK [kubernetes-apps/ansible : Kubernetes Apps | Wait for kube-apiserver] *****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:15 +0900 (0:00:00.108) 0:13:17.396 ********
ok: [kubemaster] => {"attempts": 1, "cache_control": "no-cache, private", "changed": false, "connection": "close", "content_length": "2", "content_type": "text/plain; charset=utf-8", "cookies": {}, "cookies_string": "", "date": "Sat, 12 Oct 2019 17:02:16 GMT", "elapsed": 0, "msg": "OK (2 bytes)", "redirected": false, "status": 200, "url": "https://127.0.0.1:6443/healthz", "x_content_type_options": "nosniff"}
TASK [kubernetes-apps/ansible : Kubernetes Apps | Delete kubeadm CoreDNS] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:16 +0900 (0:00:00.644) 0:13:18.041 ********
ok: [kubemaster] => {"changed": false, "msg": "success: "}
TASK [kubernetes-apps/ansible : Kubernetes Apps | Delete kubeadm Kube-DNS service] *********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:17 +0900 (0:00:00.469) 0:13:18.510 ********
ok: [kubemaster] => {"changed": false, "msg": "success: "}
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay Down CoreDNS Template] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:17 +0900 (0:00:00.438) 0:13:18.948 ********
changed: [kubemaster] => (item={'name': 'coredns', 'module': 'template', 'file': 'coredns-config.yml', 'type': 'configmap'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "edcb58bcd9fb8ef401d141b2da7811101ed41c3c", "dest": "/etc/kubernetes/coredns-config.yml", "gid": 0, "group": "root", "item": {"file": "coredns-config.yml", "module": "template", "name": "coredns", "type": "configmap"}, "md5sum": "6751e2bc83217b62104483f5097b3b59", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 558, "src": "/root/.ansible/tmp/ansible-tmp-1570899737.634935-178584447553726/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'coredns', 'module': 'copy', 'file': 'coredns-sa.yml', 'type': 'sa'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "120a7b2f1f1e14e639ff7b5daa4ed21a980d80f7", "dest": "/etc/kubernetes/coredns-sa.yml", "gid": 0, "group": "root", "item": {"file": "coredns-sa.yml", "module": "copy", "name": "coredns", "type": "sa"}, "md5sum": "bfec6ec350bf47d850c62497fed4a879", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 190, "src": "/root/.ansible/tmp/ansible-tmp-1570899738.2068105-17375502318734/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'coredns', 'module': 'template', 'file': 'coredns-deployment.yml', 'type': 'deployment'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "d3f38955a4c8a5977c6319e2a7a1b89d4257e229", "dest": "/etc/kubernetes/coredns-deployment.yml", "gid": 0, "group": "root", "item": {"file": "coredns-deployment.yml", "module": "template", "name": "coredns", "type": "deployment"}, "md5sum": "213dc4247bcc222d93f95c51cad765e5", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 3154, "src": "/root/.ansible/tmp/ansible-tmp-1570899738.6788943-140271250613421/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'coredns', 'module': 'template', 'file': 'coredns-svc.yml', 'type': 'svc'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "8aa1dc035a9c73b1b50ef4f3227632df34917793", "dest": "/etc/kubernetes/coredns-svc.yml", "gid": 0, "group": "root", "item": {"file": "coredns-svc.yml", "module": "template", "name": "coredns", "type": "svc"}, "md5sum": "323b6c03f4a9989d5364850080160cee", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 554, "src": "/root/.ansible/tmp/ansible-tmp-1570899739.2275429-203535919776543/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'coredns', 'module': 'copy', 'file': 'coredns-clusterrole.yml', 'type': 'clusterrole'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "423ea49425b12e5470028a9813d9996523a656b0", "dest": "/etc/kubernetes/coredns-clusterrole.yml", "gid": 0, "group": "root", "item": {"file": "coredns-clusterrole.yml", "module": "copy", "name": "coredns", "type": "clusterrole"}, "md5sum": "7d445083884884bc8c459ea69329fab5", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 431, "src": "/root/.ansible/tmp/ansible-tmp-1570899739.7397895-208131272217475/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'coredns', 'module': 'copy', 'file': 'coredns-clusterrolebinding.yml', 'type': 'clusterrolebinding'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "27b881209cb41225b20ab14bc9bdca6e45c45750", "dest": "/etc/kubernetes/coredns-clusterrolebinding.yml", "gid": 0, "group": "root", "item": {"file": "coredns-clusterrolebinding.yml", "module": "copy", "name": "coredns", "type": "clusterrolebinding"}, "md5sum": "751c8b9b4b4de0d14eae5d3f4fc1f208", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 451, "src": "/root/.ansible/tmp/ansible-tmp-1570899740.217948-1344371784877/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'dns-autoscaler', 'module': 'copy', 'file': 'dns-autoscaler-sa.yml', 'type': 'sa'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d96dcf2256bb4968a45d09e9232666384e83f2e", "dest": "/etc/kubernetes/dns-autoscaler-sa.yml", "gid": 0, "group": "root", "item": {"file": "dns-autoscaler-sa.yml", "module": "copy", "name": "dns-autoscaler", "type": "sa"}, "md5sum": "a111f82d5400ea2f9af53e9d6e627b6c", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 763, "src": "/root/.ansible/tmp/ansible-tmp-1570899740.7045898-159087137665704/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'dns-autoscaler', 'module': 'copy', 'file': 'dns-autoscaler-clusterrole.yml', 'type': 'clusterrole'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "f5047c80df26d815fbc5873fc47b9d7c31a0aa0f", "dest": "/etc/kubernetes/dns-autoscaler-clusterrole.yml", "gid": 0, "group": "root", "item": {"file": "dns-autoscaler-clusterrole.yml", "module": "copy", "name": "dns-autoscaler", "type": "clusterrole"}, "md5sum": "859012d2b25325835b044a62cc0226bc", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 1150, "src": "/root/.ansible/tmp/ansible-tmp-1570899741.2100737-65773933006965/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'dns-autoscaler', 'module': 'copy', 'file': 'dns-autoscaler-clusterrolebinding.yml', 'type': 'clusterrolebinding'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "5c5b302e4cd73c2ec5ceb5262b2bd608173faccc", "dest": "/etc/kubernetes/dns-autoscaler-clusterrolebinding.yml", "gid": 0, "group": "root", "item": {"file": "dns-autoscaler-clusterrolebinding.yml", "module": "copy", "name": "dns-autoscaler", "type": "clusterrolebinding"}, "md5sum": "7f8769d1e1b5cd33313edea39550fb6a", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 959, "src": "/root/.ansible/tmp/ansible-tmp-1570899741.769751-4697334166194/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'dns-autoscaler', 'module': 'template', 'file': 'dns-autoscaler.yml', 'type': 'deployment'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "774b8dc44e10c8e50e9cadf1784d87841a916906", "dest": "/etc/kubernetes/dns-autoscaler.yml", "gid": 0, "group": "root", "item": {"file": "dns-autoscaler.yml", "module": "template", "name": "dns-autoscaler", "type": "deployment"}, "md5sum": "c518e95adc1d0e10b87d20b7b5cc2e92", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2604, "src": "/root/.ansible/tmp/ansible-tmp-1570899742.269357-70951349461665/source", "state": "file", "uid": 0}
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay Down Secondary CoreDNS Template] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:02:22 +0900 (0:00:05.262) 0:13:24.211 ********
TASK [kubernetes-apps/ansible : Kubernetes Apps | set up necessary nodelocaldns parameters] ************************************************************************************************************************************************
Sunday 13 October 2019 02:02:23 +0900 (0:00:00.241) 0:13:24.453 ********
ok: [kubemaster] => {"ansible_facts": {"clusterIP": "10.233.0.3", "secondaryclusterIP": "10.233.0.4"}, "changed": false}
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay Down nodelocaldns Template] **********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:23 +0900 (0:00:00.135) 0:13:24.588 ********
changed: [kubemaster] => (item={'name': 'nodelocaldns', 'file': 'nodelocaldns-config.yml', 'type': 'configmap'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "7f596d25085a985b19f569ce893494e4c25a6fdd", "dest": "/etc/kubernetes/nodelocaldns-config.yml", "gid": 0, "group": "root", "item": {"file": "nodelocaldns-config.yml", "name": "nodelocaldns", "type": "configmap"}, "md5sum": "e9fe4f4535a8d30ec7ffc40f8f3cf750", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 1035, "src": "/root/.ansible/tmp/ansible-tmp-1570899743.2545278-208748349334289/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'nodelocaldns', 'file': 'nodelocaldns-sa.yml', 'type': 'sa'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "f07d79d01f736ef0b03b521aed3fd6ecc00bf715", "dest": "/etc/kubernetes/nodelocaldns-sa.yml", "gid": 0, "group": "root", "item": {"file": "nodelocaldns-sa.yml", "name": "nodelocaldns", "type": "sa"}, "md5sum": "37aeba6c0e893c8b795c184857d5a722", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 191, "src": "/root/.ansible/tmp/ansible-tmp-1570899743.7558305-228761812990472/source", "state": "file", "uid": 0}
changed: [kubemaster] => (item={'name': 'nodelocaldns', 'file': 'nodelocaldns-daemonset.yml', 'type': 'daemonset'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "28b4e6aca829c71b004101c037d5fd680a8ab6f6", "dest": "/etc/kubernetes/nodelocaldns-daemonset.yml", "gid": 0, "group": "root", "item": {"file": "nodelocaldns-daemonset.yml", "name": "nodelocaldns", "type": "daemonset"}, "md5sum": "7a77254409e4705b7439cff26febf354", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2646, "src": "/root/.ansible/tmp/ansible-tmp-1570899744.2129679-53279716795488/source", "state": "file", "uid": 0}
TASK [kubernetes-apps/ansible : Kubernetes Apps | Start Resources] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:24 +0900 (0:00:01.555) 0:13:26.143 ********
ok: [kubemaster] => (item=coredns-config.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "edcb58bcd9fb8ef401d141b2da7811101ed41c3c", "dest": "/etc/kubernetes/coredns-config.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "coredns-config.yml.j2", "attributes": null, "backup": false, "checksum": "edcb58bcd9fb8ef401d141b2da7811101ed41c3c", "content": null, "delimiter": null, "dest": "/etc/kubernetes/coredns-config.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899737.634935-178584447553726/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "coredns-config.yml", "module": "template", "name": "coredns", "type": "configmap"}, "md5sum": "6751e2bc83217b62104483f5097b3b59", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 558, "src": "/root/.ansible/tmp/ansible-tmp-1570899737.634935-178584447553726/source", "state": "file", "uid": 0}, "msg": "success: configmap/coredns created"}
ok: [kubemaster] => (item=coredns-sa.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "120a7b2f1f1e14e639ff7b5daa4ed21a980d80f7", "dest": "/etc/kubernetes/coredns-sa.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "coredns-sa.yml", "attributes": null, "backup": false, "checksum": "120a7b2f1f1e14e639ff7b5daa4ed21a980d80f7", "content": null, "delimiter": null, "dest": "/etc/kubernetes/coredns-sa.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899738.2068105-17375502318734/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "coredns-sa.yml", "module": "copy", "name": "coredns", "type": "sa"}, "md5sum": "bfec6ec350bf47d850c62497fed4a879", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 190, "src": "/root/.ansible/tmp/ansible-tmp-1570899738.2068105-17375502318734/source", "state": "file", "uid": 0}, "msg": "success: serviceaccount/coredns created"}
ok: [kubemaster] => (item=coredns-deployment.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "d3f38955a4c8a5977c6319e2a7a1b89d4257e229", "dest": "/etc/kubernetes/coredns-deployment.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "coredns-deployment.yml.j2", "attributes": null, "backup": false, "checksum": "d3f38955a4c8a5977c6319e2a7a1b89d4257e229", "content": null, "delimiter": null, "dest": "/etc/kubernetes/coredns-deployment.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899738.6788943-140271250613421/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "coredns-deployment.yml", "module": "template", "name": "coredns", "type": "deployment"}, "md5sum": "213dc4247bcc222d93f95c51cad765e5", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 3154, "src": "/root/.ansible/tmp/ansible-tmp-1570899738.6788943-140271250613421/source", "state": "file", "uid": 0}, "msg": "success: deployment.apps/coredns created"}
ok: [kubemaster] => (item=coredns-svc.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "8aa1dc035a9c73b1b50ef4f3227632df34917793", "dest": "/etc/kubernetes/coredns-svc.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "coredns-svc.yml.j2", "attributes": null, "backup": false, "checksum": "8aa1dc035a9c73b1b50ef4f3227632df34917793", "content": null, "delimiter": null, "dest": "/etc/kubernetes/coredns-svc.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899739.2275429-203535919776543/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "coredns-svc.yml", "module": "template", "name": "coredns", "type": "svc"}, "md5sum": "323b6c03f4a9989d5364850080160cee", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 554, "src": "/root/.ansible/tmp/ansible-tmp-1570899739.2275429-203535919776543/source", "state": "file", "uid": 0}, "msg": "success: service/coredns created"}
ok: [kubemaster] => (item=coredns-clusterrole.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "423ea49425b12e5470028a9813d9996523a656b0", "dest": "/etc/kubernetes/coredns-clusterrole.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "coredns-clusterrole.yml", "attributes": null, "backup": false, "checksum": "423ea49425b12e5470028a9813d9996523a656b0", "content": null, "delimiter": null, "dest": "/etc/kubernetes/coredns-clusterrole.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899739.7397895-208131272217475/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "coredns-clusterrole.yml", "module": "copy", "name": "coredns", "type": "clusterrole"}, "md5sum": "7d445083884884bc8c459ea69329fab5", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 431, "src": "/root/.ansible/tmp/ansible-tmp-1570899739.7397895-208131272217475/source", "state": "file", "uid": 0}, "msg": "success: clusterrole.rbac.authorization.k8s.io/system:coredns created"}
ok: [kubemaster] => (item=coredns-clusterrolebinding.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "27b881209cb41225b20ab14bc9bdca6e45c45750", "dest": "/etc/kubernetes/coredns-clusterrolebinding.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "coredns-clusterrolebinding.yml", "attributes": null, "backup": false, "checksum": "27b881209cb41225b20ab14bc9bdca6e45c45750", "content": null, "delimiter": null, "dest": "/etc/kubernetes/coredns-clusterrolebinding.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899740.217948-1344371784877/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "coredns-clusterrolebinding.yml", "module": "copy", "name": "coredns", "type": "clusterrolebinding"}, "md5sum": "751c8b9b4b4de0d14eae5d3f4fc1f208", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 451, "src": "/root/.ansible/tmp/ansible-tmp-1570899740.217948-1344371784877/source", "state": "file", "uid": 0}, "msg": "success: clusterrolebinding.rbac.authorization.k8s.io/system:coredns created"}
ok: [kubemaster] => (item=dns-autoscaler-sa.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "9d96dcf2256bb4968a45d09e9232666384e83f2e", "dest": "/etc/kubernetes/dns-autoscaler-sa.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "dns-autoscaler-sa.yml", "attributes": null, "backup": false, "checksum": "9d96dcf2256bb4968a45d09e9232666384e83f2e", "content": null, "delimiter": null, "dest": "/etc/kubernetes/dns-autoscaler-sa.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899740.7045898-159087137665704/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "dns-autoscaler-sa.yml", "module": "copy", "name": "dns-autoscaler", "type": "sa"}, "md5sum": "a111f82d5400ea2f9af53e9d6e627b6c", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 763, "src": "/root/.ansible/tmp/ansible-tmp-1570899740.7045898-159087137665704/source", "state": "file", "uid": 0}, "msg": "success: serviceaccount/dns-autoscaler created"}
ok: [kubemaster] => (item=dns-autoscaler-clusterrole.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "f5047c80df26d815fbc5873fc47b9d7c31a0aa0f", "dest": "/etc/kubernetes/dns-autoscaler-clusterrole.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "dns-autoscaler-clusterrole.yml", "attributes": null, "backup": false, "checksum": "f5047c80df26d815fbc5873fc47b9d7c31a0aa0f", "content": null, "delimiter": null, "dest": "/etc/kubernetes/dns-autoscaler-clusterrole.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899741.2100737-65773933006965/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "dns-autoscaler-clusterrole.yml", "module": "copy", "name": "dns-autoscaler", "type": "clusterrole"}, "md5sum": "859012d2b25325835b044a62cc0226bc", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 1150, "src": "/root/.ansible/tmp/ansible-tmp-1570899741.2100737-65773933006965/source", "state": "file", "uid": 0}, "msg": "success: clusterrole.rbac.authorization.k8s.io/system:dns-autoscaler created"}
ok: [kubemaster] => (item=dns-autoscaler-clusterrolebinding.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "5c5b302e4cd73c2ec5ceb5262b2bd608173faccc", "dest": "/etc/kubernetes/dns-autoscaler-clusterrolebinding.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "dns-autoscaler-clusterrolebinding.yml", "attributes": null, "backup": false, "checksum": "5c5b302e4cd73c2ec5ceb5262b2bd608173faccc", "content": null, "delimiter": null, "dest": "/etc/kubernetes/dns-autoscaler-clusterrolebinding.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899741.769751-4697334166194/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "dns-autoscaler-clusterrolebinding.yml", "module": "copy", "name": "dns-autoscaler", "type": "clusterrolebinding"}, "md5sum": "7f8769d1e1b5cd33313edea39550fb6a", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 959, "src": "/root/.ansible/tmp/ansible-tmp-1570899741.769751-4697334166194/source", "state": "file", "uid": 0}, "msg": "success: clusterrolebinding.rbac.authorization.k8s.io/system:dns-autoscaler created"}
ok: [kubemaster] => (item=dns-autoscaler.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "774b8dc44e10c8e50e9cadf1784d87841a916906", "dest": "/etc/kubernetes/dns-autoscaler.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "dns-autoscaler.yml.j2", "attributes": null, "backup": false, "checksum": "774b8dc44e10c8e50e9cadf1784d87841a916906", "content": null, "delimiter": null, "dest": "/etc/kubernetes/dns-autoscaler.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899742.269357-70951349461665/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "dns-autoscaler.yml", "module": "template", "name": "dns-autoscaler", "type": "deployment"}, "md5sum": "c518e95adc1d0e10b87d20b7b5cc2e92", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2604, "src": "/root/.ansible/tmp/ansible-tmp-1570899742.269357-70951349461665/source", "state": "file", "uid": 0}, "msg": "success: deployment.apps/dns-autoscaler created"}
ok: [kubemaster] => (item=nodelocaldns-config.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "7f596d25085a985b19f569ce893494e4c25a6fdd", "dest": "/etc/kubernetes/nodelocaldns-config.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "nodelocaldns-config.yml.j2", "attributes": null, "backup": false, "checksum": "7f596d25085a985b19f569ce893494e4c25a6fdd", "content": null, "delimiter": null, "dest": "/etc/kubernetes/nodelocaldns-config.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899743.2545278-208748349334289/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "nodelocaldns-config.yml", "name": "nodelocaldns", "type": "configmap"}, "md5sum": "e9fe4f4535a8d30ec7ffc40f8f3cf750", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 1035, "src": "/root/.ansible/tmp/ansible-tmp-1570899743.2545278-208748349334289/source", "state": "file", "uid": 0}, "msg": "success: configmap/nodelocaldns created"}
ok: [kubemaster] => (item=nodelocaldns-sa.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "f07d79d01f736ef0b03b521aed3fd6ecc00bf715", "dest": "/etc/kubernetes/nodelocaldns-sa.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "nodelocaldns-sa.yml.j2", "attributes": null, "backup": false, "checksum": "f07d79d01f736ef0b03b521aed3fd6ecc00bf715", "content": null, "delimiter": null, "dest": "/etc/kubernetes/nodelocaldns-sa.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899743.7558305-228761812990472/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "nodelocaldns-sa.yml", "name": "nodelocaldns", "type": "sa"}, "md5sum": "37aeba6c0e893c8b795c184857d5a722", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 191, "src": "/root/.ansible/tmp/ansible-tmp-1570899743.7558305-228761812990472/source", "state": "file", "uid": 0}, "msg": "success: serviceaccount/nodelocaldns created"}
ok: [kubemaster] => (item=nodelocaldns-daemonset.yml) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "28b4e6aca829c71b004101c037d5fd680a8ab6f6", "dest": "/etc/kubernetes/nodelocaldns-daemonset.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "nodelocaldns-daemonset.yml.j2", "attributes": null, "backup": false, "checksum": "28b4e6aca829c71b004101c037d5fd680a8ab6f6", "content": null, "delimiter": null, "dest": "/etc/kubernetes/nodelocaldns-daemonset.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899744.2129679-53279716795488/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "nodelocaldns-daemonset.yml", "name": "nodelocaldns", "type": "daemonset"}, "md5sum": "7a77254409e4705b7439cff26febf354", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2646, "src": "/root/.ansible/tmp/ansible-tmp-1570899744.2129679-53279716795488/source", "state": "file", "uid": 0}, "msg": "success: daemonset.apps/nodelocaldns created"}
TASK [kubernetes-apps/ansible : Kubernetes Apps | Netchecker Templates list] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:30 +0900 (0:00:06.075) 0:13:32.219 ********
TASK [kubernetes-apps/ansible : Kubernetes Apps | Append extra templates to Netchecker Templates list for PodSecurityPolicy] ***************************************************************************************************************
Sunday 13 October 2019 02:02:30 +0900 (0:00:00.095) 0:13:32.315 ********
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay Down Netchecker Template] ************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:30 +0900 (0:00:00.079) 0:13:32.394 ********
TASK [kubernetes-apps/ansible : Kubernetes Apps | Start Netchecker Resources] **************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:31 +0900 (0:00:00.074) 0:13:32.469 ********
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay down dashboard template] *************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:31 +0900 (0:00:00.090) 0:13:32.559 ********
changed: [kubemaster] => (item={'file': 'dashboard.yml', 'type': 'deploy', 'name': 'kubernetes-dashboard'}) => {"ansible_loop_var": "item", "changed": true, "checksum": "cbfa08526d7bb6cad38ee0270445b526051d0a4e", "dest": "/etc/kubernetes/dashboard.yml", "gid": 0, "group": "root", "item": {"file": "dashboard.yml", "name": "kubernetes-dashboard", "type": "deploy"}, "md5sum": "3ce8d057f5e67850219585c113821d0b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 5847, "src": "/root/.ansible/tmp/ansible-tmp-1570899751.2160826-263369489615614/source", "state": "file", "uid": 0}
TASK [kubernetes-apps/ansible : Kubernetes Apps | Start dashboard] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:31 +0900 (0:00:00.594) 0:13:33.154 ********
ok: [kubemaster] => (item={'diff': [], 'src': '/root/.ansible/tmp/ansible-tmp-1570899751.2160826-263369489615614/source', 'changed': True, 'group': 'root', 'uid': 0, 'dest': '/etc/kubernetes/dashboard.yml', 'checksum': 'cbfa08526d7bb6cad38ee0270445b526051d0a4e', 'md5sum': '3ce8d057f5e67850219585c113821d0b', 'owner': 'root', 'state': 'file', 'gid': 0, 'secontext': 'system_u:object_r:etc_t:s0', 'mode': '0644', 'invocation': {'module_args': {'directory_mode': None, 'force': True, 'remote_src': None, '_original_basename': 'dashboard.yml.j2', 'owner': None, 'follow': False, 'local_follow': None, 'group': None, 'unsafe_writes': None, 'setype': None, 'content': None, 'serole': None, 'dest': '/etc/kubernetes/dashboard.yml', 'selevel': None, 'regexp': None, 'validate': None, 'src': '/root/.ansible/tmp/ansible-tmp-1570899751.2160826-263369489615614/source', 'checksum': 'cbfa08526d7bb6cad38ee0270445b526051d0a4e', 'seuser': None, 'delimiter': None, 'mode': None, 'attributes': None, 'backup': False}}, 'size': 5847, 'failed': False, 'item': {'file': 'dashboard.yml', 'type': 'deploy', 'name': 'kubernetes-dashboard'}, 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": true, "checksum": "cbfa08526d7bb6cad38ee0270445b526051d0a4e", "dest": "/etc/kubernetes/dashboard.yml", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "dashboard.yml.j2", "attributes": null, "backup": false, "checksum": "cbfa08526d7bb6cad38ee0270445b526051d0a4e", "content": null, "delimiter": null, "dest": "/etc/kubernetes/dashboard.yml", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1570899751.2160826-263369489615614/source", "unsafe_writes": null, "validate": null}}, "item": {"file": "dashboard.yml", "name": "kubernetes-dashboard", "type": "deploy"}, "md5sum": "3ce8d057f5e67850219585c113821d0b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 5847, "src": "/root/.ansible/tmp/ansible-tmp-1570899751.2160826-263369489615614/source", "state": "file", "uid": 0}, "msg": "success: secret/kubernetes-dashboard-certs created serviceaccount/kubernetes-dashboard created role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard-anonymous created clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-anonymous created deployment.apps/kubernetes-dashboard created service/kubernetes-dashboard created"}
TASK [kubernetes-apps/helm : Helm | Make sure HELM_HOME directory exists] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:32 +0900 (0:00:00.596) 0:13:33.751 ********
TASK [kubernetes-apps/helm : Helm | Set up helm launcher] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:32 +0900 (0:00:00.078) 0:13:33.830 ********
TASK [kubernetes-apps/helm : Helm | Lay Down Helm Manifests (RBAC)] ************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:32 +0900 (0:00:00.074) 0:13:33.904 ********
TASK [kubernetes-apps/helm : Helm | Apply Helm Manifests (RBAC)] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:32 +0900 (0:00:00.144) 0:13:34.048 ********
TASK [kubernetes-apps/helm : Helm | Set up TLS] ********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:32 +0900 (0:00:00.143) 0:13:34.192 ********
TASK [kubernetes-apps/helm : Helm | Install client on all masters] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:32 +0900 (0:00:00.073) 0:13:34.265 ********
TASK [kubernetes-apps/helm : Helm | Install/upgrade helm] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:32 +0900 (0:00:00.076) 0:13:34.341 ********
TASK [kubernetes-apps/helm : Helm | Force apply tiller overrides if necessary] *************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:32 +0900 (0:00:00.084) 0:13:34.426 ********
TASK [kubernetes-apps/helm : Make sure bash_completion.d folder exists] ********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:33 +0900 (0:00:00.076) 0:13:34.502 ********
TASK [kubernetes-apps/helm : Helm | Set up bash completion] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:33 +0900 (0:00:00.070) 0:13:34.573 ********
TASK [kubernetes-apps/registry : Registry | Create addon dir] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:33 +0900 (0:00:00.087) 0:13:34.661 ********
TASK [kubernetes-apps/registry : Registry | Templates list] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:33 +0900 (0:00:00.080) 0:13:34.741 ********
TASK [kubernetes-apps/registry : Registry | Append extra templates to Registry Templates list for PodSecurityPolicy] ***********************************************************************************************************************
Sunday 13 October 2019 02:02:33 +0900 (0:00:00.079) 0:13:34.820 ********
TASK [kubernetes-apps/registry : Registry | Create manifests] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:33 +0900 (0:00:00.091) 0:13:34.912 ********
TASK [kubernetes-apps/registry : Registry | Apply manifests] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:33 +0900 (0:00:00.072) 0:13:34.985 ********
TASK [kubernetes-apps/registry : Registry | Create PVC manifests] **************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:33 +0900 (0:00:00.069) 0:13:35.054 ********
TASK [kubernetes-apps/registry : Registry | Apply PVC manifests] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:33 +0900 (0:00:00.089) 0:13:35.143 ********
TASK [kubernetes-apps/metrics_server : Check all masters are node or not] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:33 +0900 (0:00:00.107) 0:13:35.251 ********
TASK [kubernetes-apps/metrics_server : Metrics Server | Delete addon dir] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:33 +0900 (0:00:00.089) 0:13:35.341 ********
TASK [kubernetes-apps/metrics_server : Metrics Server | Create addon dir] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:34 +0900 (0:00:00.102) 0:13:35.444 ********
TASK [kubernetes-apps/metrics_server : Metrics Server | Templates list] ********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:34 +0900 (0:00:00.084) 0:13:35.528 ********
TASK [kubernetes-apps/metrics_server : Metrics Server | Create manifests] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:34 +0900 (0:00:00.121) 0:13:35.649 ********
TASK [kubernetes-apps/metrics_server : Metrics Server | Apply manifests] *******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:34 +0900 (0:00:00.074) 0:13:35.724 ********
TASK [kubernetes-apps/persistent_volumes/openstack : Kubernetes Persistent Volumes | Lay down OpenStack Cinder Storage Class template] *****************************************************************************************************
Sunday 13 October 2019 02:02:34 +0900 (0:00:00.265) 0:13:35.990 ********
TASK [kubernetes-apps/persistent_volumes/openstack : Kubernetes Persistent Volumes | Add OpenStack Cinder Storage Class] *******************************************************************************************************************
Sunday 13 October 2019 02:02:34 +0900 (0:00:00.118) 0:13:36.109 ********
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU| gather os specific variables] ****************************************************************************************************
Sunday 13 October 2019 02:02:34 +0900 (0:00:00.105) 0:13:36.214 ********
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Set fact of download url Tesla] *************************************************************************************************
Sunday 13 October 2019 02:02:34 +0900 (0:00:00.150) 0:13:36.364 ********
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Set fact of download url GTX] ***************************************************************************************************
Sunday 13 October 2019 02:02:35 +0900 (0:00:00.115) 0:13:36.480 ********
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Create addon dir] ***************************************************************************************************************
Sunday 13 October 2019 02:02:35 +0900 (0:00:00.102) 0:13:36.582 ********
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Create manifests for nvidia accelerators] ***************************************************************************************
Sunday 13 October 2019 02:02:35 +0900 (0:00:00.098) 0:13:36.680 ********
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Apply manifests for nvidia accelerators] ****************************************************************************************
Sunday 13 October 2019 02:02:35 +0900 (0:00:00.120) 0:13:36.801 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_private_key] ***********************************************************************************************************************************
Sunday 13 October 2019 02:02:35 +0900 (0:00:00.110) 0:13:36.912 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_region_id] *************************************************************************************************************************************
Sunday 13 October 2019 02:02:35 +0900 (0:00:00.090) 0:13:37.002 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_tenancy_id] ************************************************************************************************************************************
Sunday 13 October 2019 02:02:35 +0900 (0:00:00.098) 0:13:37.100 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_user_id] ***************************************************************************************************************************************
Sunday 13 October 2019 02:02:35 +0900 (0:00:00.093) 0:13:37.194 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_user_fingerprint] ******************************************************************************************************************************
Sunday 13 October 2019 02:02:35 +0900 (0:00:00.105) 0:13:37.299 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_compartment_id] ********************************************************************************************************************************
Sunday 13 October 2019 02:02:35 +0900 (0:00:00.109) 0:13:37.409 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_vnc_id] ****************************************************************************************************************************************
Sunday 13 October 2019 02:02:36 +0900 (0:00:00.101) 0:13:37.511 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_subnet1_id] ************************************************************************************************************************************
Sunday 13 October 2019 02:02:36 +0900 (0:00:00.099) 0:13:37.610 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_subnet2_id] ************************************************************************************************************************************
Sunday 13 October 2019 02:02:36 +0900 (0:00:00.088) 0:13:37.699 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_security_list_management] **********************************************************************************************************************
Sunday 13 October 2019 02:02:36 +0900 (0:00:00.095) 0:13:37.794 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Generate Cloud Provider Configuration] *********************************************************************************************************************************
Sunday 13 October 2019 02:02:36 +0900 (0:00:00.089) 0:13:37.884 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Slurp Configuration] ***************************************************************************************************************************************************
Sunday 13 October 2019 02:02:36 +0900 (0:00:00.094) 0:13:37.979 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Encode Configuration] **************************************************************************************************************************************************
Sunday 13 October 2019 02:02:36 +0900 (0:00:00.103) 0:13:38.082 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Generate Manifests] ****************************************************************************************************************************************************
Sunday 13 October 2019 02:02:36 +0900 (0:00:00.100) 0:13:38.183 ********
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Apply Manifests] *******************************************************************************************************************************************************
Sunday 13 October 2019 02:02:36 +0900 (0:00:00.081) 0:13:38.265 ********
PLAY [k8s-cluster] *************************************************************************************************************************************************************************************************************************
TASK [download : prep_download | Set a few facts] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:37 +0900 (0:00:00.220) 0:13:38.485 ********
TASK [download : Set image info command for containerd] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:37 +0900 (0:00:00.133) 0:13:38.618 ********
TASK [download : Register docker images info] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:37 +0900 (0:00:00.144) 0:13:38.763 ********
TASK [download : prep_download | Create staging directory on remote node] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:37 +0900 (0:00:00.131) 0:13:38.895 ********
TASK [download : prep_download | Create local cache for files and images] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:37 +0900 (0:00:00.141) 0:13:39.037 ********
TASK [download : prep_download | On localhost, check if passwordless root is possible] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:02:37 +0900 (0:00:00.088) 0:13:39.126 ********
TASK [download : prep_download | On localhost, check if user has access to docker without using sudo] **************************************************************************************************************************************
Sunday 13 October 2019 02:02:37 +0900 (0:00:00.099) 0:13:39.225 ********
TASK [download : prep_download | Parse the outputs of the previous commands] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:37 +0900 (0:00:00.081) 0:13:39.306 ********
TASK [download : prep_download | Check that local user is in group or can become root] *****************************************************************************************************************************************************
Sunday 13 October 2019 02:02:38 +0900 (0:00:00.152) 0:13:39.459 ********
TASK [download : include_tasks] ************************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:38 +0900 (0:00:00.136) 0:13:39.595 ********
TASK [download : download | Get kubeadm binary and list of required images] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:38 +0900 (0:00:00.131) 0:13:39.726 ********
TASK [download : download | Download files / images] ***************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:38 +0900 (0:00:00.129) 0:13:39.856 ********
TASK [download : download | Sync files / images from ansible host to nodes] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:40 +0900 (0:00:02.051) 0:13:41.907 ********
TASK [kubespray-defaults : Configure defaults] *********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:42 +0900 (0:00:02.383) 0:13:44.292 ********
ok: [kubeworker] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
ok: [kubemaster] => {
"msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
TASK [adduser : User | Create User Group] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:43 +0900 (0:00:00.161) 0:13:44.454 ********
TASK [adduser : User | Create User] ********************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:43 +0900 (0:00:00.200) 0:13:44.654 ********
TASK [kubernetes/preinstall : Remove swapfile from /etc/fstab] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:43 +0900 (0:00:00.137) 0:13:44.791 ********
TASK [kubernetes/preinstall : check swap] **************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:43 +0900 (0:00:00.191) 0:13:44.983 ********
TASK [kubernetes/preinstall : Disable swap] ************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:43 +0900 (0:00:00.169) 0:13:45.153 ********
TASK [kubernetes/preinstall : Stop if either kube-master, kube-node or etcd is empty] ******************************************************************************************************************************************************
Sunday 13 October 2019 02:02:43 +0900 (0:00:00.151) 0:13:45.304 ********
TASK [kubernetes/preinstall : Stop if non systemd OS type] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:43 +0900 (0:00:00.126) 0:13:45.431 ********
TASK [kubernetes/preinstall : Stop if unknown OS] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:44 +0900 (0:00:00.157) 0:13:45.588 ********
TASK [kubernetes/preinstall : Stop if unknown network plugin] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:44 +0900 (0:00:00.151) 0:13:45.740 ********
TASK [kubernetes/preinstall : Stop if incompatible network plugin and cloudprovider] *******************************************************************************************************************************************************
Sunday 13 October 2019 02:02:44 +0900 (0:00:00.155) 0:13:45.896 ********
TASK [kubernetes/preinstall : Stop if unsupported version of Kubernetes] *******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:44 +0900 (0:00:00.137) 0:13:46.034 ********
TASK [kubernetes/preinstall : Stop if known booleans are set as strings (Use JSON format on CLI: -e "{'key': true }")] *********************************************************************************************************************
Sunday 13 October 2019 02:02:44 +0900 (0:00:00.145) 0:13:46.180 ********
TASK [kubernetes/preinstall : Stop if even number of etcd hosts] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:44 +0900 (0:00:00.180) 0:13:46.360 ********
TASK [kubernetes/preinstall : Stop if memory is too small for masters] *********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:45 +0900 (0:00:00.182) 0:13:46.542 ********
TASK [kubernetes/preinstall : Stop if memory is too small for nodes] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:45 +0900 (0:00:00.138) 0:13:46.680 ********
TASK [kubernetes/preinstall : Guarantee that enough network address space is available for all pods] ***************************************************************************************************************************************
Sunday 13 October 2019 02:02:45 +0900 (0:00:00.153) 0:13:46.834 ********
TASK [kubernetes/preinstall : Stop if ip var does not match local ips] *********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:45 +0900 (0:00:00.155) 0:13:46.989 ********
TASK [kubernetes/preinstall : Stop if access_ip is not pingable] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:45 +0900 (0:00:00.153) 0:13:47.143 ********
TASK [kubernetes/preinstall : Stop if RBAC is not enabled when dashboard is enabled] *******************************************************************************************************************************************************
Sunday 13 October 2019 02:02:45 +0900 (0:00:00.142) 0:13:47.285 ********
TASK [kubernetes/preinstall : Stop if RBAC is not enabled when OCI cloud controller is enabled] ********************************************************************************************************************************************
Sunday 13 October 2019 02:02:46 +0900 (0:00:00.173) 0:13:47.459 ********
TASK [kubernetes/preinstall : Stop if RBAC and anonymous-auth are not enabled when insecure port is disabled] ******************************************************************************************************************************
Sunday 13 October 2019 02:02:46 +0900 (0:00:00.190) 0:13:47.649 ********
TASK [kubernetes/preinstall : Stop if kernel version is too low] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:46 +0900 (0:00:00.174) 0:13:47.824 ********
TASK [kubernetes/preinstall : Stop if bad hostname] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:46 +0900 (0:00:00.214) 0:13:48.039 ********
TASK [kubernetes/preinstall : check cloud_provider value] **********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:46 +0900 (0:00:00.140) 0:13:48.179 ********
TASK [kubernetes/preinstall : Ensure minimum calico version] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:46 +0900 (0:00:00.171) 0:13:48.351 ********
TASK [kubernetes/preinstall : Get current version of calico cluster version] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:47 +0900 (0:00:00.124) 0:13:48.476 ********
TASK [kubernetes/preinstall : Check that calico version is enough for upgrade] *************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:47 +0900 (0:00:00.101) 0:13:48.577 ********
TASK [kubernetes/preinstall : Check that cluster_id is set if calico_rr enabled] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:47 +0900 (0:00:00.082) 0:13:48.660 ********
TASK [kubernetes/preinstall : Check that calico_rr nodes are in k8s-cluster group] *********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:47 +0900 (0:00:00.101) 0:13:48.761 ********
TASK [kubernetes/preinstall : Check that kube_service_addresses is a network range] ********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:47 +0900 (0:00:00.154) 0:13:48.916 ********
TASK [kubernetes/preinstall : Check that kube_pods_subnet is a network range] **************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:47 +0900 (0:00:00.083) 0:13:48.999 ********
TASK [kubernetes/preinstall : Check that kube_pods_subnet does not collide with kube_service_addresses] ************************************************************************************************************************************
Sunday 13 October 2019 02:02:47 +0900 (0:00:00.082) 0:13:49.081 ********
TASK [kubernetes/preinstall : Stop if unknown dns mode] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:47 +0900 (0:00:00.081) 0:13:49.162 ********
TASK [kubernetes/preinstall : Stop if unknown kube proxy mode] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:47 +0900 (0:00:00.089) 0:13:49.252 ********
TASK [kubernetes/preinstall : Stop if vault is chose] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:47 +0900 (0:00:00.086) 0:13:49.339 ********
TASK [kubernetes/preinstall : Stop if unknown cert_management] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:47 +0900 (0:00:00.084) 0:13:49.424 ********
TASK [kubernetes/preinstall : Stop if unknown resolvconf_mode] *****************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:48 +0900 (0:00:00.083) 0:13:49.507 ********
TASK [kubernetes/preinstall : Stop if kubeadm etcd mode is enabled but experimental control plane is not] **********************************************************************************************************************************
Sunday 13 October 2019 02:02:48 +0900 (0:00:00.079) 0:13:49.586 ********
TASK [kubernetes/preinstall : Pre-upgrade | check if old credential dir exists] ************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:48 +0900 (0:00:00.163) 0:13:49.749 ********
TASK [kubernetes/preinstall : Pre-upgrade | check if new credential dir exists] ************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:48 +0900 (0:00:00.099) 0:13:49.849 ********
TASK [kubernetes/preinstall : Pre-upgrade | move data from old credential dir to new] ******************************************************************************************************************************************************
Sunday 13 October 2019 02:02:48 +0900 (0:00:00.081) 0:13:49.930 ********
TASK [kubernetes/preinstall : set architecture_groups] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:48 +0900 (0:00:00.087) 0:13:50.018 ********
TASK [kubernetes/preinstall : ansible_architecture_rename] *********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:48 +0900 (0:00:00.135) 0:13:50.153 ********
TASK [kubernetes/preinstall : Force binaries directory for Container Linux by CoreOS] ******************************************************************************************************************************************************
Sunday 13 October 2019 02:02:48 +0900 (0:00:00.145) 0:13:50.299 ********
TASK [kubernetes/preinstall : check if atomic host] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:49 +0900 (0:00:00.139) 0:13:50.438 ********
TASK [kubernetes/preinstall : set is_atomic] ***********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:49 +0900 (0:00:00.135) 0:13:50.574 ********
TASK [kubernetes/preinstall : set kube_cert_group on atomic hosts] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:49 +0900 (0:00:00.126) 0:13:50.701 ********
TASK [kubernetes/preinstall : check resolvconf] ********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:49 +0900 (0:00:00.137) 0:13:50.838 ********
TASK [kubernetes/preinstall : set dns facts] ***********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:49 +0900 (0:00:00.146) 0:13:50.985 ********
TASK [kubernetes/preinstall : check if kubelet is configured] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:49 +0900 (0:00:00.140) 0:13:51.125 ********
TASK [kubernetes/preinstall : check if early DNS configuration stage] **********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:49 +0900 (0:00:00.130) 0:13:51.256 ********
TASK [kubernetes/preinstall : target resolv.conf files] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:49 +0900 (0:00:00.156) 0:13:51.413 ********
TASK [kubernetes/preinstall : target temporary resolvconf cloud init file (Container Linux by CoreOS)] *************************************************************************************************************************************
Sunday 13 October 2019 02:02:50 +0900 (0:00:00.135) 0:13:51.548 ********
TASK [kubernetes/preinstall : check if /etc/dhclient.conf exists] **************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:50 +0900 (0:00:00.146) 0:13:51.695 ********
TASK [kubernetes/preinstall : target dhclient conf file for /etc/dhclient.conf] ************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:50 +0900 (0:00:00.141) 0:13:51.836 ********
TASK [kubernetes/preinstall : check if /etc/dhcp/dhclient.conf exists] *********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:50 +0900 (0:00:00.152) 0:13:51.989 ********
TASK [kubernetes/preinstall : target dhclient conf file for /etc/dhcp/dhclient.conf] *******************************************************************************************************************************************************
Sunday 13 October 2019 02:02:50 +0900 (0:00:00.131) 0:13:52.121 ********
TASK [kubernetes/preinstall : target dhclient hook file for Red Hat family] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:50 +0900 (0:00:00.135) 0:13:52.257 ********
TASK [kubernetes/preinstall : target dhclient hook file for Debian family] *****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:50 +0900 (0:00:00.146) 0:13:52.404 ********
TASK [kubernetes/preinstall : generate search domains to resolvconf] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:51 +0900 (0:00:00.134) 0:13:52.538 ********
TASK [kubernetes/preinstall : pick coredns cluster IP or default resolver] *****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:51 +0900 (0:00:00.134) 0:13:52.673 ********
TASK [kubernetes/preinstall : generate nameservers to resolvconf] **************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:51 +0900 (0:00:00.136) 0:13:52.809 ********
TASK [kubernetes/preinstall : gather os specific variables] ********************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:51 +0900 (0:00:00.157) 0:13:52.967 ********
TASK [kubernetes/preinstall : override kube_proxy_mode to ipvs if kube_proxy_remove is set, as ipvs won't require kube-proxy cleanup when kube-proxy daemonset gets deleted] ***************************************************************
Sunday 13 October 2019 02:02:51 +0900 (0:00:00.184) 0:13:53.151 ********
TASK [kubernetes/preinstall : set etcd vars if using kubeadm mode] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:51 +0900 (0:00:00.160) 0:13:53.311 ********
TASK [kubernetes/preinstall : check /usr readonly] *****************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:52 +0900 (0:00:00.141) 0:13:53.453 ********
TASK [kubernetes/preinstall : set alternate flexvolume path] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:52 +0900 (0:00:00.137) 0:13:53.590 ********
TASK [kubernetes/preinstall : Create kubernetes directories] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:52 +0900 (0:00:00.151) 0:13:53.741 ********
TASK [kubernetes/preinstall : Check if kubernetes kubeadm compat cert dir exists] **********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:52 +0900 (0:00:00.243) 0:13:53.985 ********
TASK [kubernetes/preinstall : Create kubernetes kubeadm compat cert dir (kubernetes/kubeadm issue 1498)] ***********************************************************************************************************************************
Sunday 13 October 2019 02:02:52 +0900 (0:00:00.148) 0:13:54.133 ********
TASK [kubernetes/preinstall : Create cni directories] **************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:52 +0900 (0:00:00.138) 0:13:54.272 ********
TASK [kubernetes/preinstall : Create local volume provisioner directories] *****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:53 +0900 (0:00:00.184) 0:13:54.456 ********
TASK [kubernetes/preinstall : create temporary resolveconf cloud init file] ****************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:53 +0900 (0:00:00.154) 0:13:54.610 ********
TASK [kubernetes/preinstall : Add domain/search/nameservers/options to resolv.conf] ********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:53 +0900 (0:00:00.170) 0:13:54.780 ********
TASK [kubernetes/preinstall : Remove search/domain/nameserver options before block] ********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:53 +0900 (0:00:00.163) 0:13:54.944 ********
TASK [kubernetes/preinstall : Remove search/domain/nameserver options after block] *********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:53 +0900 (0:00:00.203) 0:13:55.147 ********
TASK [kubernetes/preinstall : get temporary resolveconf cloud init file content] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:53 +0900 (0:00:00.238) 0:13:55.385 ********
TASK [kubernetes/preinstall : persist resolvconf cloud init file] **************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:54 +0900 (0:00:00.145) 0:13:55.531 ********
TASK [kubernetes/preinstall : Update package management cache (zypper) - SUSE] *************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:54 +0900 (0:00:00.137) 0:13:55.669 ********
TASK [kubernetes/preinstall : Update package management cache (APT)] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:54 +0900 (0:00:00.152) 0:13:55.822 ********
TASK [kubernetes/preinstall : Install python-dnf for latest RedHat versions] ***************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:54 +0900 (0:00:00.133) 0:13:55.955 ********
TASK [kubernetes/preinstall : Install epel-release on RedHat/CentOS] ***********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:54 +0900 (0:00:00.137) 0:13:56.092 ********
TASK [kubernetes/preinstall : Update common_required_pkgs with ipvsadm when kube_proxy_mode is ipvs] ***************************************************************************************************************************************
Sunday 13 October 2019 02:02:54 +0900 (0:00:00.148) 0:13:56.240 ********
TASK [kubernetes/preinstall : Install packages requirements] *******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:54 +0900 (0:00:00.140) 0:13:56.381 ********
TASK [kubernetes/preinstall : Install ipvsadm for ClearLinux] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:55 +0900 (0:00:00.133) 0:13:56.515 ********
TASK [kubernetes/preinstall : Confirm selinux deployed] ************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:55 +0900 (0:00:00.147) 0:13:56.662 ********
TASK [kubernetes/preinstall : Set selinux policy] ******************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:55 +0900 (0:00:00.160) 0:13:56.823 ********
TASK [kubernetes/preinstall : Disable IPv6 DNS lookup] *************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:55 +0900 (0:00:00.130) 0:13:56.954 ********
TASK [kubernetes/preinstall : Stat sysctl file configuration] ******************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:55 +0900 (0:00:00.130) 0:13:57.084 ********
TASK [kubernetes/preinstall : Change sysctl file path to link source if linked] ************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:55 +0900 (0:00:00.135) 0:13:57.219 ********
TASK [kubernetes/preinstall : Make sure sysctl file path folder exists] ********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:55 +0900 (0:00:00.144) 0:13:57.364 ********
TASK [kubernetes/preinstall : Enable ip forwarding] ****************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:56 +0900 (0:00:00.136) 0:13:57.500 ********
TASK [kubernetes/preinstall : Hosts | populate inventory into hosts file] ******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:56 +0900 (0:00:00.167) 0:13:57.668 ********
TASK [kubernetes/preinstall : Hosts | populate kubernetes loadbalancer address into hosts file] ********************************************************************************************************************************************
Sunday 13 October 2019 02:02:56 +0900 (0:00:00.136) 0:13:57.804 ********
TASK [kubernetes/preinstall : Hosts | Retrieve hosts file content] *************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:56 +0900 (0:00:00.143) 0:13:57.948 ********
TASK [kubernetes/preinstall : Hosts | Extract existing entries for localhost from hosts file] **********************************************************************************************************************************************
Sunday 13 October 2019 02:02:56 +0900 (0:00:00.134) 0:13:58.083 ********
TASK [kubernetes/preinstall : Hosts | Update target hosts file entries dict with required entries] *****************************************************************************************************************************************
Sunday 13 October 2019 02:02:56 +0900 (0:00:00.165) 0:13:58.248 ********
TASK [kubernetes/preinstall : Hosts | Update (if necessary) hosts file] ********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:56 +0900 (0:00:00.178) 0:13:58.427 ********
TASK [kubernetes/preinstall : Configure dhclient to supersede search/domain/nameservers] ***************************************************************************************************************************************************
Sunday 13 October 2019 02:02:57 +0900 (0:00:00.164) 0:13:58.591 ********
TASK [kubernetes/preinstall : Configure dhclient hooks for resolv.conf (non-RH)] ***********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:57 +0900 (0:00:00.134) 0:13:58.726 ********
TASK [kubernetes/preinstall : Configure dhclient hooks for resolv.conf (RH-only)] **********************************************************************************************************************************************************
Sunday 13 October 2019 02:02:57 +0900 (0:00:00.140) 0:13:58.866 ********
TASK [kubernetes/preinstall : Remove kubespray specific config from dhclient config] *******************************************************************************************************************************************************
Sunday 13 October 2019 02:02:57 +0900 (0:00:00.158) 0:13:59.024 ********
TASK [kubernetes/preinstall : Remove kubespray specific dhclient hook] *********************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:57 +0900 (0:00:00.138) 0:13:59.163 ********
TASK [kubernetes/preinstall : Check if we are running inside a Azure VM] *******************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:57 +0900 (0:00:00.132) 0:13:59.296 ********
TASK [kubernetes/preinstall : install growpart] ********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:58 +0900 (0:00:00.160) 0:13:59.456 ********
TASK [kubernetes/preinstall : check if growpart needs to be run] ***************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:58 +0900 (0:00:00.134) 0:13:59.590 ********
TASK [kubernetes/preinstall : check fs type] ***********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:58 +0900 (0:00:00.141) 0:13:59.732 ********
TASK [kubernetes/preinstall : run growpart] ************************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:58 +0900 (0:00:00.153) 0:13:59.885 ********
TASK [kubernetes/preinstall : run xfs_growfs] **********************************************************************************************************************************************************************************************
Sunday 13 October 2019 02:02:58 +0900 (0:00:00.161) 0:14:00.047 ********
PLAY RECAP *********************************************************************************************************************************************************************************************************************************
kubemaster : ok=594 changed=97 unreachable=0 failed=0 skipped=831 rescued=0 ignored=1
kubeworker : ok=471 changed=51 unreachable=0 failed=0 skipped=602 rescued=0 ignored=0
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Sunday 13 October 2019 02:02:58 +0900 (0:00:00.094) 0:14:00.142 ********
===============================================================================
download : download_file | Download item ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 149.07s
download : download_file | Download item ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 114.51s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 28.73s
kubernetes/master : kubeadm | Initialize first master ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 22.91s
kubernetes/kubeadm : Join to cluster ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 22.56s
download : download_file | Download item ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 21.89s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 14.92s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 13.84s
etcd : Configure | Check if etcd cluster is healthy --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.68s
download : download_container | Download image if required -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.47s
download : download_file | Download item -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.27s
download : download_container | Download image if required -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.22s
kubernetes-apps/ansible : Kubernetes Apps | Start Resources ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.08s
download : download_container | Download image if required -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 5.96s
download : download_container | Download image if required -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 5.96s
kubernetes/preinstall : Install packages requirements ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 5.55s
etcd : wait for etcd up ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 5.28s
kubernetes-apps/ansible : Kubernetes Apps | Lay Down CoreDNS Template --------------------------------------------------------------------------------------------------------------------------------------------------------------- 5.26s
download : download_container | Download image if required -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 5.24s
download : download | Download files / images --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 4.28s
[root@kubemaster kubespray]#14분 정도의 시간이면, Kubemaster, Kubeworker에 대한 구성이 모두 완료됩니다.
위와 같이 구성이 완료되면 아래 같은 명령어로 상태 정보를 확인합니다.
[root@kubemaster kubespray]# kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-bbb69849d-rqxr2 1/1 Running 0 5m37s
kube-system calico-node-99zns 1/1 Running 0 6m11s
kube-system calico-node-dq6m7 1/1 Running 0 6m11s
kube-system coredns-58687784f9-p7gh9 1/1 Running 0 5m17s
kube-system coredns-58687784f9-qr9n9 1/1 Running 0 5m
kube-system dns-autoscaler-79599df498-76x8k 1/1 Running 0 5m14s
kube-system kube-apiserver-kubemaster 1/1 Running 0 7m2s
kube-system kube-controller-manager-kubemaster 1/1 Running 0 7m2s
kube-system kube-proxy-l6nzp 1/1 Running 0 6m28s
kube-system kube-proxy-rqmfc 1/1 Running 0 7m8s
kube-system kube-scheduler-kubemaster 1/1 Running 0 7m1s
kube-system kubernetes-dashboard-556b9ff8f8-p7fqh 1/1 Running 0 5m11s
kube-system nginx-proxy-kubeworker 1/1 Running 0 6m27s
kube-system nodelocaldns-nv4qc 1/1 Running 0 5m13s
kube-system nodelocaldns-t9xsb 1/1 Running 0 5m13s
[root@kubemaster kubespray]# kubectl get nodes --all-namespaces
NAME STATUS ROLES AGE VERSION
kubemaster Ready master 7m50s v1.15.3
kubeworker Ready <none> 6m49s v1.15.3
[root@kubemaster kubespray]# kubectl get deployment --all-namespaces
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system calico-kube-controllers 1/1 1 1 19m
kube-system coredns 2/2 2 2 19m
kube-system dns-autoscaler 1/1 1 1 19m
kube-system kubernetes-dashboard 1/1 1 1 19m
[root@kubemaster kubespray]#위와 같이 pods와 nodes의 상태를 확인할 수 있으며, 모두 정상 기동된 것을 확인할 수 있습니다.
Kubernetes Dashboard에 접속하는 방법은 지난 포스팅을 참고하시기 바랍니다.
[Container Management] Kubernetes Dashboard Install & Setting
대시보드 접속이 완료되면 다음과 같은 화면을 볼 수 있습니다.
(https://192.168.56.107:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/login)
상세한 설정 파일에 대한 커스터마이징은 이후 포스팅에서 진행하도록 하겠습니다.
728x90
반응형
'③ 클라우드 > ⓚ Kubernetes' 카테고리의 다른 글
| Kubernetes - 장애 발생 시 진단 Process 정의 (2) | 2019.10.27 |
|---|---|
| Kubespray - OpenStack 기반 Kubernetes installation (using ansible) (0) | 2019.10.25 |
| Kubernetes - Kuberspray Private Registry 적용하기 (7) | 2019.10.11 |
| Kubernetes - Err services "ingress-nginx" not found (0) | 2019.08.18 |
| Kubernetes - etcdmain: open /etc/kubernetes/pki/etcd/peer.crt: permission denied (0) | 2019.08.17 |
최근에 올라온 글
최근에 달린 댓글
- Total
- Today
- Yesterday
링크
TAG
- nodejs
- jeus
- openstack tenant
- 마이크로서비스 아키텍처
- Da
- MSA
- JBoss
- webtob
- 쿠버네티스
- openstack token issue
- JEUS7
- k8s
- wildfly
- 오픈스택
- TA
- SWA
- node.js
- git
- SA
- API Gateway
- aws
- Architecture
- apache
- 아키텍처
- JEUS6
- 마이크로서비스
- aa
- Docker
- OpenStack
- kubernetes
| 일 | 월 | 화 | 수 | 목 | 금 | 토 |
|---|---|---|---|---|---|---|
| 1 | 2 | |||||
| 3 | 4 | 5 | 6 | 7 | 8 | 9 |
| 10 | 11 | 12 | 13 | 14 | 15 | 16 |
| 17 | 18 | 19 | 20 | 21 | 22 | 23 |
| 24 | 25 | 26 | 27 | 28 | 29 | 30 |
글 보관함