OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter, all managed through a dashboard that gives administrators control while empowering their users to provision resources through a web interface.
Docker is the world's leading software containerization platform. Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries – anything that can be installed on a server.
what's kolla? Kolla's mission statement is: To provide production-ready containers and deployment tools for operating OpenStack clouds.
Some weeks ago, I deployed one private cloud for my company on three node. I ran into many problems, and resolve them all, it's not very easy for lack document. so I write a detail document to show my process of installation, maybe can help someone fix some issue.
This public doc is my note in develop environment, not production environment. I have ever redone some steps so maybe there are some id or output mistake.
Phsical Node:
192.168.1.61 oscontroller
192.168.1.91 osnova1
192.168.1.92 osnova2
firstly you must config the initialize environment on all node.
vi /etc/hostname
vi /etc/hosts
# controller
192.168.1.61 oscontroller
# compute
192.168.1.91 osnova1
192.168.1.92 osnova2
ping osnova1
ping osnova2
yum update
yum install yum-plugin-priorities -y
yum install epel-release -y
yum install wget -y
1.4 verify you cpu support VT
[root@oscontroller ~]# egrep -c '(vmx|svm)' /proc/cpuinfo
2
[root@osnova1 ~]# egrep -c '(vmx|svm)' /proc/cpuinfo
4
[root@osnova2 ~]# egrep -c '(vmx|svm)' /proc/cpuinfo
4
[root@oscontroller ~]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
[root@osnova1 ~]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
[root@osnova2 ~]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
[root@oscontroller .ssh]# scp root@osnova1:~/.ssh/id_rsa.pub id_rsa.nova1.pub
[root@oscontroller .ssh]# scp root@osnova2:~/.ssh/id_rsa.pub id_rsa.nova2.pub
[root@oscontroller .ssh]# cat id_rsa.pub >> ~/.ssh/authorized_keys
[root@oscontroller .ssh]# cat id_rsa.nova1.pub >> ~/.ssh/authorized_keys
[root@oscontroller .ssh]# cat id_rsa.nova2.pub >> ~/.ssh/authorized_keys
[root@oscontroller .ssh]# less authorized_keys
[root@oscontroller .ssh]# less authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvsqNEBby5QOlDwpTlL1CAtqnzgQJ96U1JlEXfJ11I4JUk617WriDfNtDsPewzBhf1NfyaXoHqN5Bl9S8Ap2Gov4LCuUsa35qTE9uGJSAfnuJc3zTp2ciX0SZwanoCORzzY4hPB38V1Tg6JFsvVFzJanWk6Lb2zPm5F6RmikY3O4PTvBJFsL1rTzpr9UH37qDEZJ97jonCHeFSmvVk0Y+uvgniU4ttVupJS6Yfr2ITWHmt0/WOTl9d7AW+hufkrMuPUUXLx3cT+wAMIGQ7+/aVWxonL7TFtFXFND/4BLyx1M3luQho8mHBENDt94bUQQZGVIrj1Bb1FfobKkb6eGWb root@oscontroller
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCUrNutfi4uhcThQL2pPbXmfZ2wSMv5W8tV5yzpeYLaCpSU9d+rCpcgBXPQA2v+on1JAFxFCC+J8A63OWiIoBtwn9FLzI4Takfr+HmOSetVJMshxolYZ+/S6Qjs+hxBLRL9Qwpwtu6c19qGhz14W8gaNeHEDFoD705kk/5PhdOsxYpL+v/GXIvUBlR4zF9OJGrxNJIA48bqHOGBqBtcZbM4Vbre3Q52aVFxI34Rs1N344YgSJSg4wnzlOIvW+Vq1CClwRT9QRms2wAPcrRb4pva7c30+6wCGncPM64rt4gw0vDc1cxR8Jp6Gs/KYWU5roqXdcl72lEgLdE8aM9qQLoZ root@osnova1
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeT8fy3K8XVc51JRvM/cF9DaQHL/e30JMebPBjUCf+/YdMDXkbIwVLGurMhDf8RVA7F5F0VxAe10kYeTv1BugkP34VVSDb80yHAyT8R/TOfi8z2WTREtT2eeuUVfuJI3dPmT+r+/7JowLyznJm2awRm9G2IOtUHZBn46/fTksdEtd7RLaWR7upkR37mwNJ8Qc7f3s9bYdNyn/LC9RKdNQWCbLZdnVbRRx5ixzt7snCClfE4djlO7cdPzsR+fggk9SIz9MJT12ePSgaqRBaTZgMuo+0NhB41NBK6Dx1IqdmxBhQlHufGG8sXhPDg3yyUONC4zK+GSjaLet1dCd4SAa1 root@osnova2
[root@oscontroller .ssh]# scp authorized_keys root@osnova1:~/.ssh/authorized_keys
[root@oscontroller .ssh]# scp authorized_keys root@osnova2:~/.ssh/authorized_keys
1.6 config network
[root@oscontroller ~]# cat /etc/sysconfig/network-scripts/ifcfg-em1
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=em1
UUID=74c4c813-646e-4de4-a45d-25770ae70d1e
DEVICE=em1
ONBOOT=yes
IPADDR=192.168.1.61
PREFIX=24
GATEWAY=192.168.1.1
DNS1=8.8.8.8
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_PRIVACY=no
[root@oscontroller ~]# cat /etc/sysconfig/network-scripts/ifcfg-em2 #this network can not be set ip, otherwise the instance can not be access.
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
NAME=em2
UUID=85095748-b497-455d-b94f-1ef1156ed9c3
DEVICE=em2
ONBOOT=yes
[root@osnova1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-em1
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=em1
UUID=ed52b5f3-1f6d-4fc6-a488-23ab19ab08e1
DEVICE=em1
ONBOOT=yes
IPADDR=192.168.1.91
PREFIX=24
GATEWAY=192.168.1.1
DNS1=8.8.8.8
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_PRIVACY=no
[root@osnova1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-em2 #this network can not be set ip
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
NAME=em2
UUID=8e3f62a3-b878-4e96-ac80-c1456de7fa49
DEVICE=em2
ONBOOT=yes
[root@osnova2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-em1
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=em1
UUID=ac949101-9e4d-47e8-971f-b0e67b1b4f17
DEVICE=em1
ONBOOT=yes
IPADDR=192.168.1.92
PREFIX=24
GATEWAY=192.168.1.1
DNS1=8.8.8.8
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_PRIVACY=no
[root@osnova2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-em2 #this network can not be set ip
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
NAME=em2
UUID=0920f7c0-a406-4170-9d70-706b65d485d6
DEVICE=em2
ONBOOT=yes
1.7 install dependency of kolla
yum install python-pip -y
pip install -U pip
pip -V
[root@oscontroller ~]# yum install ansible1.9.noarch
[root@oscontroller ~]# ansible --version
ansible 1.9.6
configured module search path = None
yum install gcc gcc-c++ -y
1.8 config NTP
yum install ntp -y
systemctl enable ntpd.service
systemctl start ntpd.service
yum install chrony -y
[root@oscontroller ~]# vi /etc/chrony.conf
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst
# Ignore stratum in source selection.
stratumweight 0
# Record the rate at which the system clock gains/losses time.
driftfile /var/lib/chrony/drift
# Enable kernel RTC synchronization.
rtcsync
# In first three updates step the system clock instead of slew
# if the adjustment is larger than 10 seconds.
makestep 10 3
# Allow NTP client access from local network.
#allow 192.168/16
allow 192.168.1.0/24
[root@osnova1 osnova2 ~]# vi /etc/chrony.conf
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst
server oscontroller iburst
systemctl enable chronyd.service
systemctl start chronyd.service
chronyc sources
1.9 config firwall
@oscontroller
firewall-cmd --permanent --zone=public --add-port=80/tcp
firewall-cmd --permanent --zone=public --add-port=443/tcp
firewall-cmd --permanent --zone=public --add-port=3306/tcp
firewall-cmd --permanent --zone=public --add-port=5672/tcp
firewall-cmd --permanent --zone=public --add-port=5000/tcp
firewall-cmd --permanent --zone=public --add-port=8082/tcp
firewall-cmd --permanent --zone=public --add-port=8773-8775/tcp
firewall-cmd --permanent --zone=public --add-port=8386/tcp
firewall-cmd --permanent --zone=public --add-port=9696/tcp
firewall-cmd --permanent --zone=public --add-port=35357/tcp
firewall-cmd --permanent --zone=public --add-port=6080-6082/tcp
firewall-cmd --permanent --zone=public --add-port=5900-5999/tcp
firewall-cmd --permanent --zone=public --add-port=9191/tcp
firewall-cmd --permanent --zone=public --add-port=9292/tcp
firewall-cmd --permanent --zone=public --add-port=8776/tcp
firewall-cmd --permanent --zone=public --add-port=8777/tcp
firewall-cmd --permanent --zone=public --add-port=8000/tcp
firewall-cmd --permanent --zone=public --add-port=8003/tcp
firewall-cmd --permanent --zone=public --add-port=8004/tcp
firewall-cmd --permanent --zone=public --add-port=6000-6002/tcp
firewall-cmd --permanent --zone=public --add-port=873/tcp
firewall-cmd --permanent --zone=public --add-port=3260/tcp
firewall-cmd --permanent --zone=public --add-port=8080/tcp
firewall-cmd --permanent --zone=public --add-port=90/tcp #nginx for images
firewall-cmd --reload
firewall-cmd --list-all
systemctl restart firewalld
@osnova1 & osnova2
firewall-cmd --permanent --zone=public --add-port=5900-5999/tcp #Compute ports for access to virtual machine consoles
firewall-cmd --permanent --zone=public --add-port=8022/tcp
firewall-cmd --reload
firewall-cmd --list-all
systemctl restart firewalld
1.10 config selinux
getenforce #set selinux=disable, otherwise the vm instances can not be access from external network
vi /etc/selinux/config
SELINUX=disabled
2. install kolla with source type
[root@oscontroller ~]# yum install git -y
[root@oscontroller ~]# cd /data/Iaas/OpenStack/
[root@oscontroller OpenStack]# git clone https://git.openstack.org/openstack/kolla
[root@oscontroller OpenStack]# cd kolla/
[root@oscontroller kolla]# git checkout stable/mitaka # change branch, for now, I choose stable/mitaka for stable and less bug.
Branch stable/mitaka set up to track remote branch stable/mitaka from origin.
Switched to a new branch 'stable/mitaka'
[root@oscontroller kolla]# cd ..
[root@oscontroller OpenStack]# pip install -r kolla/requirements.txt -r kolla/test-requirements.txt
[root@oscontroller OpenStack]# pip install kolla/
[root@oscontroller OpenStack]# cd kolla
[root@oscontroller kolla]# cp -r etc/kolla /etc/
[root@all ~]# yum install python-devel libffi-devel openssl-devel gcc -y
[root@oscontroller kolla]# pip install -U python-openstackclient python-neutronclient
[root@oscontroller kolla]# pip install tox
[root@oscontroller kolla]# tox -e genconfig
genconfig create: /data/Iaas/OpenStack/kolla/.tox/genconfig
genconfig installdeps: -r/data/Iaas/OpenStack/kolla/requirements.txt, -r/data/Iaas/OpenStack/kolla/test-requirements.txt
genconfig develop-inst: /data/Iaas/OpenStack/kolla
genconfig installed: appdirs==1.4.0,Babel==2.3.4,backports.ssl-match-hostname==3.5.0.1,bandit==1.1.0,bashate==0.5.1,beautifulsoup4==4.5.1,cliff==2.2.0,cmd2==0.6.8,debtcollector==1.8.0,docker-py==1.10.3,docker-pycreds==0.2.1,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.5.5,funcsigs==1.0.2,functools32==3.2.3.post2,futures==3.0.5,gitdb==0.6.4,GitPython==2.0.8,graphviz==0.5.1,hacking==0.11.0,ipaddress==1.0.17,iso8601==0.1.11,Jinja2==2.8,jsonpatch==1.14,jsonpointer==1.10,jsonschema==2.5.1,kazoo==2.2.1,keystoneauth1==2.12.1,-e git+https://git.openstack.org/openstack/kolla@99683643653f4859e47a157647463b6eed074b04#egg=kolla,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,monotonic==1.2,mox3==0.18.0,msgpack-python==0.4.8,netaddr==0.7.18,netifaces==0.10.5,os-client-config==1.21.1,osc-lib==1.1.0,oslo.config==3.17.0,oslo.context==2.9.0,oslo.i18n==3.9.0,oslo.log==3.16.0,oslo.serialization==2.13.0,oslo.utils==3.16.0,oslosphinx==4.7.0,oslotest==2.10.0,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,prettytable==0.7.2,pycrypto==2.6.1,pyflakes==0.8.1,Pygments==2.1.3,pyinotify==0.9.6,pyparsing==2.1.9,python-barbicanclient==4.1.0,python-ceilometerclient==2.6.1,python-cinderclient==1.9.0,python-dateutil==2.5.3,python-glanceclient==2.5.0,python-heatclient==1.4.0,python-keystoneclient==2.3.1,python-mimeparse==1.5.2,python-neutronclient==6.0.0,python-novaclient==6.0.0,python-subunit==1.2.0,python-swiftclient==3.1.0,pytz==2016.6.1,PyYAML==3.12,reno==1.8.0,requests==2.10.0,requestsexceptions==1.1.3,rfc3986==0.4.1,simplejson==3.8.2,six==1.10.0,smmap==0.9.0,Sphinx==1.2.3,stevedore==1.17.1,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unicodecsv==0.14.1,unittest2==1.1.0,warlock==1.2.0,websocket-client==0.37.0,wrapt==1.10.8,zake==0.2.2
genconfig runtests: PYTHONHASHSEED='1214607384'
genconfig runtests: commands[0] | oslo-config-generator --config-file etc/oslo-config-generator/kolla-build.conf
WARNING:stevedore.named:Could not load kolla
__________________________________________________________________________________________________________________ summary __________________________________________________________________________________________________________________
genconfig: commands succeeded
congratulations :)
3.1 install docker on all nodes
[root@all ~]# curl -sSL https://get.docker.com/ | sh
[root@all ~]# docker --version
[root@all ~]# usermod -aG docker root
[root@all ~]# tee /etc/sysconfig/docker <<-'EOF'
DOCKER_OPTS="--insecure-registry 192.168.1.61:4000"
EOF
[root@all ~]# mkdir -p /etc/systemd/system/docker.service.d # Create the drop-in unit directory for docker.service
[root@all ~]# vi /etc/systemd/system/docker.service.d/kolla.conf
[Service]
MountFlags=shared
EnvironmentFile=-/etc/sysconfig/docker
#EnvironmentFile=-/etc/sysconfig/docker-storage
#EnvironmentFile=-/etc/sysconfig/docker-network
ExecStart=
ExecStart=/usr/bin/dockerd --insecure-registry 192.168.1.61:4000
#ExecStart=/usr/bin/dockerd $DOCKER_OPTS
[root@all ~]# systemctl daemon-reload
[root@all ~]# systemctl restart docker # systemctl daemon-reload & systemctl restart docker
[root@all ~]# systemctl status docker.service
[root@all ~]# systemctl enable docker.service
[root@all ~]# yum install python-docker-py -y # or pip install -U docker-py
3.2 start up private registry on oscontroller
[root@oscontroller ~]# docker run -d -p 4000:5000 --restart=always -v /data/Iaas/Docker/Volumes/Registry:/var/lib/registry --name registry registry:2 # set -v param for storage images in path /data/Iaas/Docker/Volumes/Registry
[root@oscontroller docker]# docker stop registry
[root@oscontroller ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
72f90844b386 registry:2 "/entrypoint.sh /etc/" 53 seconds ago Up 52 seconds 0.0.0.0:4000->5000/tcp registry
4. modify the configuration files of kolla
[root@oscontroller ~]# cp /usr/share/kolla/ansible/inventory/multinode /usr/share/kolla/ansible/inventory/multinode.bak
[root@oscontroller ~]# vi /usr/share/kolla/ansible/inventory/multinode
# These initial groups are the only groups required to be modified. The
# additional groups are for more control of the environment.
[control]
# These hostname must be resolvable from your deployment host
oscontroller
# The above can also be specified as follows:
#control[01:03] ansible_ssh_user=kolla
# The network nodes are where your l3-agent and loadbalancers will run
# This can be the same as a host in the control group
[network]
oscontroller
[compute]
osnova1
osnova2
# When compute nodes and control nodes use different interfaces,
# you can specify "api_interface" and another interfaces like below:
#compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em1
[storage]
oscontroller
[root@oscontroller ~]# cp /usr/share/kolla/etc_examples/kolla/* /etc/kolla
[root@oscontroller ~]# kolla-genpwd #The tool will populate all empty fields in the /etc/kolla/passwords.yml file using randomly generated values to secure the deployment.
[root@oscontroller ~]# vi /etc/kolla/globals.yml
kolla_internal_vip_address: "192.168.1.251"
network_interface: "em1"
neutron_external_interface: "em2"
docker_registry: "192.168.1.61:4000"
kolla_install_type: "source"
#kolla_internal_vip_address: "10.10.10.254"
#network_interface: "eth0"
#neutron_external_interface: "eth1"
[root@oscontroller ~]# cat /etc/kolla/passwords.yml | grep database
cinder_database_password: sWluY2iHtHviFZdfx9CFA3rdeghaqow1fwuf14QR
database_password: XI8EeKsOyhQaDYM2P6PK9nlyjbCjGEJwM1UuU9xy # use this password you can login into mysql
glance_database_password: JubdRouJOTtNv0mIhSzgT9wRc3PuVEZFt9hkGCmX
heat_database_password: JGZaVXnlLtA0QWjpOLM1qEmu13sFBF5b8vSbtFK0
ironic_database_password: SumunjZPdSpZKcJYhZXjHHtMkR0Kp5Rg9aqD1ppv
keystone_database_password: 0LZuaRargoiSiGKi5JamKL5qgOGSYptf7s5vjx9J
magnum_database_password: O1RGvmCgnaYWnzcWxuSGBLLaRZyeF6FsKs7CRrga
manila_database_password: 3oyHotlkVIFW8caIfWL5ciJt7yTgEk6GpxdWOCMf
mistral_database_password: vnvT7HHjbOFFg6EaybWlslQA1AuVwDlss52kXGKY
murano_database_password: VX7Nmmn5RVyAkl5gdaUOI8fO5u5L0VrP26XoptYA
neutron_database_password: sFqiRFBENofc50leb9jWAPLkrKFn2fnfYeeQ2T8T
nova_api_database_password: t2jkrGzeIZ5s3UCxZIeTvZJeIeHsBjaSIhFADDqa
nova_database_password: ZZ5J8LjxoRVD0aRH1xyOd07E9d6jTo792SN1DDKX
5. Building Container Images
[root@oscontroller ~]# kolla-build --base centos --type source --registry 192.168.1.61:4000 --push # this may spend one or two hour
INFO:kolla.cmd.build:Found the docker image folder at /usr/share/kolla/docker
INFO:kolla.cmd.build:base:Building
INFO:kolla.cmd.build:base:Step 1 : FROM centos:latest
INFO:kolla.cmd.build:base: ---> 980e0e4c79ec
INFO:kolla.cmd.build:base:Step 2 : MAINTAINER Kolla Project (https://launchpad.net/kolla)
INFO:kolla.cmd.build:base: ---> Using cache
INFO:kolla.cmd.build:base: ---> 7da41b625814
INFO:kolla.cmd.build:base:Step 3 : LABEL kolla_version "2.0.3"
INFO:kolla.cmd.build:base: ---> Using cache
INFO:kolla.cmd.build:base: ---> 53059b0adec1
INFO:kolla.cmd.build:base:Step 4 : ENV KOLLA_BASE_DISTRO centos
INFO:kolla.cmd.build:base: ---> Using cache
INFO:kolla.cmd.build:base: ---> 214bda1795ab
INFO:kolla.cmd.build:base:Step 5 : ENV KOLLA_INSTALL_TYPE source
INFO:kolla.cmd.build:base: ---> Using cache
INFO:kolla.cmd.build:base: ---> fa008b8b4460
INFO:kolla.cmd.build:base:Step 6 : ENV KOLLA_INSTALL_METATYPE mixed
INFO:kolla.cmd.build:base: ---> Using cache
INFO:kolla.cmd.build:base: ---> 6ccb756d9fa3
INFO:kolla.cmd.build:base:Step 7 : COPY kolla_bashrc /tmp/
INFO:kolla.cmd.build:base: ---> Using cache
INFO:kolla.cmd.build:base: ---> d7aead56fdd8
.........................
.........................
INFO:kolla.cmd.build:mongodb:Total download size: 54 M
INFO:kolla.cmd.build:mongodb:Installed size: 165 M
INFO:kolla.cmd.build:mongodb:Downloading packages:
INFO:kolla.cmd.build:mongodb:http://mirror.centos.org/centos/7/cloud/x86_64/openstack-mitaka/common/mongodb-2.6.11-1.el7.x86_64.rpm: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=centos-openstack-mitaka clean metadata
INFO:kolla.cmd.build:mongodb:Trying other mirror.
INFO:kolla.cmd.build:mongodb:
INFO:kolla.cmd.build:mongodb:http://mirror.centos.org/centos/7/cloud/x86_64/openstack-mitaka/common/mongodb-server-2.6.11-1.el7.x86_64.rpm: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=centos-openstack-mitaka clean metadata
INFO:kolla.cmd.build:mongodb:Trying other mirror.
INFO:kolla.cmd.build:mongodb:
INFO:kolla.cmd.build:mongodb:
INFO:kolla.cmd.build:mongodb:Error downloading packages:
INFO:kolla.cmd.build:mongodb: mongodb-server-2.6.11-1.el7.x86_64: [Errno 256] No more mirrors to try.
INFO:kolla.cmd.build:mongodb: mongodb-2.6.11-1.el7.x86_64: [Errno 256] No more mirrors to try.
INFO:kolla.cmd.build:mongodb:
.........................
.........................
INFO:kolla.cmd.build:nova-consoleauth
INFO:kolla.cmd.build:designate-base
INFO:kolla.cmd.build:neutron-dhcp-agent
INFO:kolla.cmd.build:neutron-base
INFO:kolla.cmd.build:designate-mdns
INFO:kolla.cmd.build:ceilometer-notification
INFO:kolla.cmd.build:ironic-inspector
INFO:kolla.cmd.build:ironic-conductor
INFO:kolla.cmd.build:nova-scheduler
INFO:kolla.cmd.build:manila-base
INFO:kolla.cmd.build:manila-share
INFO:kolla.cmd.build:ceph-osd
INFO:kolla.cmd.build:mariadb
INFO:kolla.cmd.build:swift-account
INFO:kolla.cmd.build:nova-compute
INFO:kolla.cmd.build:neutron-metadata-agent
INFO:kolla.cmd.build:nova-compute-ironic
INFO:kolla.cmd.build:glance-base
INFO:kolla.cmd.build:ironic-pxe
INFO:kolla.cmd.build:gnocchi-statsd
INFO:kolla.cmd.build:trove-conductor
({}, {'aodh-expirer': 'built', 'nova-api': 'built', 'cinder-base': 'built', 'swift-container': 'built', 'designate-api': 'built', 'kolla-toolbox': 'built', 'heat-engine': 'built', 'gnocchi-base': 'built', 'openvswitch-vswitchd': 'built', 'nova-network': 'built', 'trove-base': 'built', 'heat-api-cfn': 'built', 'aodh-base': 'built', 'murano-engine': 'built', 'neutron-server': 'built', 'magnum-base': 'built', 'magnum-conductor': 'built', 'murano-api': 'built', 'mistral-executor': 'built', 'mesos-dns': 'built', 'nova-spicehtml5proxy': 'built', 'aodh-api': 'built', 'nova-libvirt': 'built', 'cinder-api': 'built', 'rabbitmq': 'built', 'manila-api': 'built', 'nova-base': 'built', 'zookeeper': 'built', 'kibana': 'built', 'cron': 'built', 'swift-proxy-server': 'built', 'aodh-evaluator': 'built', 'tempest': 'built', 'trove-guestagent': 'built', 'neutron-linuxbridge-agent': 'built', 'nova-ssh': 'built', 'magnum-api': 'built', 'keepalived': 'built', 'openstack-base': 'built', 'keystone': 'built', 'mongodb': 'built', 'ceph-mon': 'built', 'memcached': 'built', 'mesos-slave': 'built', 'heka': 'built', 'base': 'built', 'haproxy': 'built', 'ceilometer-collector': 'built', 'glance-api': 'built', 'swift-base': 'built', 'elasticsearch': 'built', 'horizon': 'built', 'cinder-scheduler': 'built', 'designate-sink': 'built', 'murano-base': 'built', 'dind': 'built', 'trove-api': 'built', 'designate-poolmanager': 'built', 'neutron-openvswitch-agent': 'built', 'ceilometer-compute': 'built', 'manila-scheduler': 'built', 'mesos-base': 'built', 'ceilometer-central': 'built', 'nova-novncproxy': 'built', 'mesos-master': 'built', 'designate-backend-bind9': 'built', 'gnocchi-api': 'built', 'ceilometer-base': 'built', 'heat-api': 'built', 'aodh-notifier': 'built', 'cinder-backup': 'built', 'nova-conductor': 'built', 'heat-base': 'built', 'designate-central': 'built', 'neutron-l3-agent': 'built', 'ironic-api': 'built', 'mistral-base': 'built', 'mistral-api': 'built', 'cinder-rpcbind': 'built', 'swift-object': 'built', 'openvswitch-base': 'built', 'ceph-base': 'built', 'cinder-volume': 'built', 'ceilometer-api': 'built', 'trove-taskmanager': 'built', 'swift-rsyncd': 'built', 'aodh-listener': 'built', 'glance-registry': 'built', 'mesosphere-base': 'built', 'openvswitch-db-server': 'built', 'ceph-rgw': 'built', 'chronos': 'built', 'marathon': 'built', 'ironic-base': 'built', 'mistral-engine': 'built', 'zaqar': 'built', 'nova-consoleauth': 'built', 'designate-base': 'built', 'neutron-dhcp-agent': 'built', 'neutron-base': 'built', 'designate-mdns': 'built', 'ceilometer-notification': 'built', 'ironic-inspector': 'built', 'ironic-conductor': 'built', 'nova-scheduler': 'built', 'manila-base': 'built', 'manila-share': 'built', 'ceph-osd': 'built', 'mariadb': 'built', 'swift-account': 'built', 'nova-compute': 'built', 'neutron-metadata-agent': 'built', 'nova-compute-ironic': 'built', 'glance-base': 'built', 'ironic-pxe': 'built', 'gnocchi-statsd': 'built', 'trove-conductor': 'built'}, {})
if you are lucky, you'll see the state of all images is built, if you see some images is error or parent_error， you must rebuild that one.
forexample:
[root@oscontroller ~]# kolla-build --base centos --type source --registry 192.168.1.61:4000 --push ceph-base
if it timeout, you can modify the Dockerfile.j2, eg:
[root@oscontroller docker]# vi /usr/share/kolla/docker/ceph/ceph-base/Dockerfile.j2
FROM {{ namespace }}/{{ image_prefix }}base:{{ tag }}
MAINTAINER {{ maintainer }}
{% if base_distro in ['centos', 'fedora', 'oraclelinux', 'rhel'] %}
RUN yum install centos-release-openstack-mitaka.noarch -y
RUN sed -i 's/mirror.centos.org/mirrors.aliyun.com/g' /etc/yum.repos.d/CentOS-OpenStack-mitaka.repo # in china, maybe you need this line
RUN yum -y install ceph
RUN yum -y install ceph-radosgw
RUN yum -y install parted \
hdparm \
btrfs-progs \
&& yum clean all
#RUN rpm -ivh http://ftp.riken.jp/Linux/centos/7/storage/x86_64/ceph-hammer/ceph-0.94.5-1.el7.x86_64.rpm
6. Deploy Kolla
6.1 precheck and verify
[root@oscontroller ~]# kolla-ansible prechecks -i /usr/share/kolla/ansible/inventory/multinode
Pre-deployment checking : ansible-playbook -i /usr/share/kolla/ansible/inventory/multinode -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla /usr/share/kolla/ansible/prechecks.yml
PLAY [all] ********************************************************************
GATHERING FACTS ***************************************************************
ok: [oscontroller]
ok: [osnova2]
ok: [osnova1]
...........
...........
TASK: [prechecks | Checking that libvirt is not running] **********************
skipping: [oscontroller]
ok: [osnova1]
ok: [osnova2]
TASK: [prechecks | Checking Docker version] ***********************************
ok: [oscontroller]
ok: [osnova2]
ok: [osnova1]
TASK: [prechecks | Checking if 'MountFlags' for docker service is set to 'shared'] ***
skipping: [osnova1]
skipping: [osnova2]
ok: [oscontroller]
TASK: [prechecks | Checking if '/run' mount flag is set to 'shared'] **********
skipping: [osnova1]
skipping: [osnova2]
skipping: [oscontroller]
TASK: [prechecks | Checking empty passwords in passwords.yml. Run kolla-genpwd if this task fails] ***
ok: [osnova2 -> 127.0.0.1]
ok: [oscontroller -> 127.0.0.1]
ok: [osnova1 -> 127.0.0.1]
TASK: [prechecks | Checking docker-py version] ********************************
ok: [oscontroller]
ok: [osnova1]
ok: [osnova2]
TASK: [prechecks | Checking Ansible version] **********************************
ok: [osnova1 -> 127.0.0.1]
ok: [osnova2 -> 127.0.0.1]
ok: [oscontroller -> 127.0.0.1]
PLAY RECAP ********************************************************************
osnova1 : ok=8 changed=0 unreachable=0 failed=0
osnova2 : ok=8 changed=0 unreachable=0 failed=0
oscontroller : ok=64 changed=0 unreachable=0 failed=0
you should see all nodes is ok, none nodes is failed
6.2 Verify that all required images with appropriate tags are available
[root@oscontroller ~]# kolla-ansible pull -i /usr/share/kolla/ansible/inventory/multinode
Pulling Docker images : ansible-playbook -i /usr/share/kolla/ansible/inventory/multinode -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e action=pull /usr/share/kolla/ansible/site.yml
PLAY [ceph-mon;ceph-osd;ceph-rgw] *********************************************
GATHERING FACTS ***************************************************************
ok: [oscontroller]
TASK: [common | Pulling kolla-toolbox image] **********************************
skipping: [oscontroller]
TASK: [common | Pulling heka image] *******************************************
skipping: [oscontroller]
TASK: [common | Pulling cron image] *******************************************
skipping: [oscontroller]
TASK: [common | Registering common role has run] ******************************
skipping: [oscontroller]
TASK: [ceph | Pulling ceph-mon image] *****************************************
skipping: [oscontroller]
TASK: [ceph | Pulling ceph-osd image] *****************************************
skipping: [oscontroller]
TASK: [ceph | Pulling ceph-rgw image] *****************************************
skipping: [oscontroller]
PLAY [elasticsearch] **********************************************************
GATHERING FACTS ***************************************************************
ok: [oscontroller]
.....................
.....................
.....................
PLAY [mongodb] ****************************************************************
GATHERING FACTS ***************************************************************
ok: [oscontroller]
TASK: [common | Pulling kolla-toolbox image] **********************************
skipping: [oscontroller]
TASK: [common | Pulling heka image] *******************************************
skipping: [oscontroller]
TASK: [common | Pulling cron image] *******************************************
skipping: [oscontroller]
TASK: [common | Registering common role has run] ******************************
skipping: [oscontroller]
TASK: [mongodb | Pulling mongodb image] ***************************************
skipping: [oscontroller]
PLAY [manila-api;manila-share;manila-scheduler;rabbitmq;memcached] ************
GATHERING FACTS ***************************************************************
ok: [oscontroller]
TASK: [common | Pulling kolla-toolbox image] **********************************
skipping: [oscontroller]
TASK: [common | Pulling heka image] *******************************************
skipping: [oscontroller]
TASK: [common | Pulling cron image] *******************************************
skipping: [oscontroller]
TASK: [common | Registering common role has run] ******************************
skipping: [oscontroller]
TASK: [manila | Pulling manila-api image] *************************************
skipping: [oscontroller]
TASK: [manila | Pulling manila-scheduler image] *******************************
skipping: [oscontroller]
TASK: [manila | Pulling manila-share image] ***********************************
skipping: [oscontroller]
PLAY RECAP ********************************************************************
osnova1 : ok=12 changed=9 unreachable=0 failed=0
osnova2 : ok=12 changed=9 unreachable=0 failed=0
oscontroller : ok=49 changed=0 unreachable=0 failed=0
you should see all nodes is ok, none nodes is failed
6.3 begin to deploy
[root@oscontroller ~]# kolla-ansible deploy -i /usr/share/kolla/ansible/inventory/multinode # this may spend one hour
Deploying Playbooks : ansible-playbook -i /usr/share/kolla/ansible/inventory/multinode -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e action=deploy /usr/share/kolla/ansible/site.yml
PLAY [ceph-mon;ceph-osd;ceph-rgw] *********************************************
GATHERING FACTS ***************************************************************
ok: [oscontroller]
TASK: [common | Ensuring config directories exist] ****************************
skipping: [oscontroller] => (item=heka)
skipping: [oscontroller] => (item=cron)
skipping: [oscontroller] => (item=cron/logrotate)
TASK: [common | Copying over config.json files for services] ******************
skipping: [oscontroller] => (item=heka)
skipping: [oscontroller] => (item=cron)
TASK: [common | Copying over heka config files] *******************************
skipping: [oscontroller] => (item={'enabled': u'no', 'name': 'elasticsearch'})
skipping: [oscontroller] => (item={'enabled': 'yes', 'name': 'global'})
skipping: [oscontroller] => (item={'enabled': u'yes', 'name': 'haproxy'})
skipping: [oscontroller] => (item={'enabled': u'yes', 'name': 'horizon'})
skipping: [oscontroller] => (item={'enabled': u'yes', 'name': 'keepalived'})
skipping: [oscontroller] => (item={'enabled': u'yes', 'name': 'keystone'})
skipping: [oscontroller] => (item={'enabled': u'yes', 'name': 'mariadb'})
skipping: [oscontroller] => (item={'enabled': 'yes', 'name': 'openstack'})
skipping: [oscontroller] => (item={'enabled': u'yes', 'name': 'rabbitmq'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-account-auditor'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-account-reaper'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-account-replicator'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-account-server'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-container-auditor'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-container-replicator'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-container-server'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-container-updater'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-object-auditor'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-object-expirer'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-object-replicator'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-object-server'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-object-updater'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-proxy-server'})
skipping: [oscontroller] => (item={'src': 'swift', 'enabled': u'no', 'name': 'swift-rsyncd'})
TASK: [common | Copying over cron logrotate config files] *********************
skipping: [oscontroller] => (item=ansible)
skipping: [oscontroller] => (item=cinder)
skipping: [oscontroller] => (item=glance)
skipping: [oscontroller] => (item=global)
skipping: [oscontroller] => (item=haproxy)
skipping: [oscontroller] => (item=heat)
skipping: [oscontroller] => (item=keepalived)
skipping: [oscontroller] => (item=keystone)
skipping: [oscontroller] => (item=magnum)
skipping: [oscontroller] => (item=manila)
skipping: [oscontroller] => (item=mariadb)
skipping: [oscontroller] => (item=mistral)
skipping: [oscontroller] => (item=murano)
skipping: [oscontroller] => (item=neutron)
skipping: [oscontroller] => (item=nova)
skipping: [oscontroller] => (item=rabbitmq)
skipping: [oscontroller] => (item=swift)
....................
....................
....................
TASK: [common | Starting kolla-toolbox container] *****************************
skipping: [oscontroller]
TASK: [common | Starting cron container] **************************************
skipping: [oscontroller]
TASK: [common | Registering common role has run] ******************************
skipping: [oscontroller]
TASK: [manila | Creating the Manila service and endpoint] *********************
skipping: [oscontroller] => (item={'interface': 'admin', 'url': u'http://192.168.1.251:8786/v1/%(tenant_id)s', 'service_type': 'share', 'service_name': 'manila'})
skipping: [oscontroller] => (item={'interface': 'internal', 'url': u'http://192.168.1.251:8786/v1/%(tenant_id)s', 'service_type': 'share', 'service_name': 'manila'})
skipping: [oscontroller] => (item={'interface': 'public', 'url': u'http://192.168.1.251:8786/v1/%(tenant_id)s', 'service_type': 'share', 'service_name': 'manila'})
skipping: [oscontroller] => (item={'interface': 'admin', 'url': u'http://192.168.1.251:8786/v2/%(tenant_id)s', 'service_type': 'sharev2', 'service_name': 'manilav2'})
skipping: [oscontroller] => (item={'interface': 'internal', 'url': u'http://192.168.1.251:8786/v2/%(tenant_id)s', 'service_type': 'sharev2', 'service_name': 'manilav2'})
skipping: [oscontroller] => (item={'interface': 'public', 'url': u'http://192.168.1.251:8786/v2/%(tenant_id)s', 'service_type': 'sharev2', 'service_name': 'manilav2'})
TASK: [manila | Creating the Manila project, user and role] *******************
skipping: [oscontroller]
TASK: [manila | Ensuring config directories exist] ****************************
skipping: [oscontroller] => (item=manila-api)
skipping: [oscontroller] => (item=manila-scheduler)
skipping: [oscontroller] => (item=manila-share)
TASK: [manila | Copying over config.json files for services] ******************
skipping: [oscontroller] => (item=manila-api)
skipping: [oscontroller] => (item=manila-scheduler)
skipping: [oscontroller] => (item=manila-share)
TASK: [manila | Copying over manila.conf] *************************************
skipping: [oscontroller] => (item=manila-api)
skipping: [oscontroller] => (item=manila-scheduler)
skipping: [oscontroller] => (item=manila-share)
TASK: [manila | Creating Manila database] *************************************
skipping: [oscontroller]
TASK: [manila | Reading json from variable] ***********************************
skipping: [oscontroller]
TASK: [manila | Creating Manila database user and setting permissions] ********
skipping: [oscontroller]
TASK: [manila | Running Manila bootstrap container] ***************************
skipping: [oscontroller]
TASK: [manila | Starting manila-api container] ********************************
skipping: [oscontroller]
TASK: [manila | Starting manila-scheduler container] **************************
skipping: [oscontroller]
TASK: [manila | Starting manila-share container] ******************************
skipping: [oscontroller]
PLAY RECAP ********************************************************************
osnova1 : ok=65 changed=43 unreachable=0 failed=0
osnova2 : ok=65 changed=43 unreachable=0 failed=0
oscontroller : ok=299 changed=110 unreachable=0 failed=0
all nodes should be ok.
if you see some node is failed, you must deploy again, but before do that, you should execute:
[root@all ~]# cd /data/kolla/tools/
[root@all tools]# ./cleanup-containers
[root@all tools]# ./cleanup-host
7. update
sometime you neeed add or remove a computer node or controller node, you should modify /usr/share/kolla/ansible/inventory/multinode, then execute:
kolla-ansible upgrade -i /usr/share/kolla/ansible/inventory/multinode
8.1 visit horizon
[root@oscontroller ~]# cat /etc/kolla/passwords.yml | grep keystone_admin_password
keystone_admin_password: ynRZeY85GzGd4Fa5VK7UBB0lNbpNUXrXawQBmJCO
now you can open http://192.168.1.61
login with:
domain: default username:admin pwd: ynRZeY85GzGd4Fa5VK7UBB0lNbpNUXrXawQBmJCO
8.2 create an openrc file
[root@oscontroller ~]# kolla-ansible post-deploy
[root@oscontroller ~]# cat /etc/kolla/admin-openrc.sh
export OS_PROJECT_DOMAIN_ID=default
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_NAME=admin
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=ynRZeY85GzGd4Fa5VK7UBB0lNbpNUXrXawQBmJCO
export OS_AUTH_URL=http://192.168.1.253:35357/v3
export OS_IDENTITY_API_VERSION=3
[root@oscontroller ~]# source /etc/kolla/admin-openrc.sh
8.3 initialize network and a glance
[root@oscontroller ~]# /data/Iaas/OpenStack/kolla/tools/init-runonce
All finished. you can see the initalize network from http://192.168.1.61.
Sometime, after you recreated your network, if the gateway of your network can not ping, you may need reboot some node. or assign interface to vm instances.
Enjoy yourself!