(1).实验环境
| 主机名 | IP地址 | 角色 | 内存 | 网卡 | CPU | 磁盘 |
| OpenStack-con | 192.168.128.110 | controller（控制） | 8G | 桥接网卡ens32和ens33 | 4核 | 200G |
OpenStack-com | 192.168.128.111 | compute（计算） | 4G | 桥接网卡ens32 | 4核 | 20G |
| OpenStack-sto | 192.168.128.112 | storage（存储） | 4G | 桥接网卡ens32 | 4核 | 2个20G |
注意：OpenStack-con主机充当两个角色：kolla-ansible部署节点和controller节点。
(2).准备工作
三台主机都安装<Tab>键补全软件包和vim软件包
yum -y install bash-completion.noarch vim
三台主机都关闭firewalld和SELinux
# vim /etc/selinux/config
SELINUX=disabled
# setenforce 0
# getenforce
Permissive
# systemctl disable firewalld && systemctl stop firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@OpenStack-con ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:firewalld(1)
12月 27 10:43:55 OpenStack-con systemd[1]: Starting firewalld - dynamic firewall daemon...
12月 27 10:44:01 OpenStack-con systemd[1]: Started firewalld - dynamic firewall daemon.
12月 27 11:03:53 OpenStack-con systemd[1]: Stopping firewalld - dynamic firewall daemon...
12月 27 11:03:54 OpenStack-con systemd[1]: Stopped firewalld - dynamic firewall daemon.在OpenStack-con（110）上配置hosts文件，并复制到另两台主机上
[root@OpenStack-con ~]# vim /etc/hosts 192.168.128.110 OpenStack-con 192.168.128.111 OpenStack-com 192.168.128.112 OpenStack-sto [root@OpenStack-con ~]# scp /etc/hosts 192.168.128.111:/etc/ [root@OpenStack-con ~]# scp /etc/hosts 192.168.128.112:/etc/
修改OpenStack-con（110）的ens33配置文件
[root@OpenStack-con ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33 //只放以下内容 TYPE=Ethernet BOOTPROTO=none //静态获取IP ONBOOT=yes //启用网卡 PROXY_METHOD=none NAME=ens33 DEVICE=ens33 [root@OpenStack-con ~]# systemctl restart network
(3).安装ansible
1）在OpenStack-con（110）主机上安装并更新pip工具
//安装epel源 [root@OpenStack-con ~]# yum -y install epel-release [root@OpenStack-con ~]# yum -y install python-pip [root@OpenStack-con ~]# mkdir .pip //配置pip软件包源 [root@OpenStack-con ~]# tee /root/.pip/pip.conf << 'EOF' [global] index-url = http://mirrors.aliyun.com/pypi/simple/ [install] trusted-host=mirrors.aliyun.com EOF //更新pip [root@OpenStack-con ~]# pip install -U pip
2）配置OpenStack-com（111）和OpenStack-sto（112）pip软件包源
注意：这一步是为了后期安装
//OpenStack-com（111）执行一次 [root@OpenStack-com ~]# mkdir .pip [root@OpenStack-com ~]# tee /root/.pip/pip.conf << 'EOF' [global] index-url = http://mirrors.aliyun.com/pypi/simple/ [install] trusted-host=mirrors.aliyun.com EOF //OpenStack-sto（112）执行一次 [root@OpenStack-sto ~]# mkdir .pip [root@OpenStack-sto ~]# tee /root/.pip/pip.conf << 'EOF' [global] index-url = http://mirrors.aliyun.com/pypi/simple/ [install] trusted-host=mirrors.aliyun.com EOF
3）在OpenStack-con（110）上安装ansible
//安装依赖包 [root@OpenStack-con ~]# yum -y install python-devel libffi-devel gcc openssl-devel libselinux-python [root@OpenStack-con ~]# pip install ansible
4）在OpenStack-con（110）上配置ansible参数
注意：pip安装的ansible是没有配置文件的，此时需要前往github将默认配置文件拷贝下来。网址：https://github.com/ansible/ansible/blob/devel/examples/ansible.cfg
[root@OpenStack-con ~]# ansible --version //可以看到此时是没有配置文件的 ansible 2.9.2 configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible executable location = /usr/bin/ansible python version = 2.7.5 (default, Aug 7 2019, 00:51:29) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] //生成配置文件 [root@OpenStack-con ~]# mkdir /etc/ansible [root@OpenStack-con ~]# vim /etc/ansible/ansible.cfg [root@OpenStack-con ~]# ansible --version //已经自动加载了配置文件 ansible 2.9.2 config file = /etc/ansible/ansible.cfg configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible executable location = /usr/bin/ansible python version = 2.7.5 (default, Aug 7 2019, 00:51:29) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
然后开始优化
[root@OpenStack-con ~]# vim /etc/ansible/ansible.cfg forks = 10 //第19行，设置并行进程数。如果要管理的主机很多，可以优先尝试增加该值 host_key_checking = False //第67行，跳过ssh首次连接提示验证部分 pipelining = True //第403行，开启管道输送。ansible在执行一个模块需要ssh到目的主机多次，开启该模式减少ssh连接次数，缩短ansible执行时间。 //在部署大规模服务器或引用模块非常多时，开启pipelining会给ansible带来显著的性能提升
(4).OpenStack-sto（112）配置cinder（块存储）信息
[root@OpenStack-sto ~]# yum -y install yum-utils device-mapper-persistent-data lvm2 //安装相关软件包
[root@OpenStack-sto ~]# pvs //查看已经存在的pv
PV VG Fmt Attr PSize PFree
/dev/sda2 centos lvm2 a-- <19.00g 0
[root@OpenStack-sto ~]# pvcreate /dev/sdb
Physical volume "/dev/sdb" successfully created.
[root@OpenStack-sto ~]# vgcreate cinder /dev/sdb
Volume group "cinder" successfully created
[root@OpenStack-sto ~]# systemctl status lvm2-lvmetad.service //保证开机自启
● lvm2-lvmetad.service - LVM2 metadata daemon
Loaded: loaded (/usr/lib/systemd/system/lvm2-lvmetad.service; static; vendor preset: enabled)
Active: active (running) since 二 2019-12-17 15:26:11 CST; 20min ago
Docs: man:lvmetad(8)
Main PID: 11954 (lvmetad)
CGroup: /system.slice/lvm2-lvmetad.service
└─11954 /usr/sbin/lvmetad -f
12月 17 15:26:11 OpenStack-sto systemd[1]: Started LVM2 metadata daemon.
12月 17 15:26:11 OpenStack-sto systemd[1]: Starting LVM2 metadata daemon...
(5).在OpenStack-con（110）上安装kolla-ansible并自定义OpenStack的相关配置文件
1）安装kolla-ansible
[root@OpenStack-con ~]# pip install kolla-ansible
2）复制kolla-ansible的相关配置文件
[root@OpenStack-con ~]# cp -r /usr/share/kolla-ansible/etc_examples/kolla /etc/ [root@OpenStack-con ~]# cp /usr/share/kolla-ansible/ansible/inventory/* /etc/kolla/ [root@OpenStack-con ~]# ls /etc/kolla/ all-in-one globals.yml multinode passwords.yml
文件说明：all-in-one是安装单节点OpenStack的ansible自动安装配置文件；multinode是安装多节点OpenStack的ansible自动安装配置文件；globals.yml是OpenStack部署的自定义配置文件；passwords.yml是OpenStack中各个服务的密码文件。
3）生成密钥，并授权三台主机
[root@OpenStack-con ~]# ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: SHA256:MlEHvdjHadF+ydFC80Gg0u/sKcP+hvC8gDpvHTOGuL4 root@OpenStack-con The key's randomart image is: +---[RSA 2048]----+ | oo. +=o.| | . .o o o+o| | . + = +..+| | .. + * .o.| | + S o . . | | . +.* o | | ..o.O .o | | o.. ..B... | | .E=. .o*+ | +----[SHA256]-----+ [root@OpenStack-con ~]# ssh-copy-id -i .ssh/id_rsa.pub root@OpenStack-con [root@OpenStack-con ~]# ssh-copy-id -i .ssh/id_rsa.pub root@OpenStack-com [root@OpenStack-con ~]# ssh-copy-id -i .ssh/id_rsa.pub root@OpenStack-sto
4）配置multinode多节点主机清单文件
[root@OpenStack-con ~]# vim /etc/kolla/multinode //修改模块名不带:chilldren的部分 # These initial groups are the only groups required to be modified. The # additional groups are for more control of the environment. [control] #控制模块 # These hostname must be resolvable from your deployment host OpenStack-con #给110主机 # The above can also be specified as follows: #control[01:03] ansible_user=kolla # The network nodes are where your l3-agent and loadbalancers will run # This can be the same as a host in the control group [network] #网络模块 OpenStack-con #给110主机 [compute] #计算模块 OpenStack-com #给111主机 [monitoring] #监控模块 OpenStack-con #给110主机 # When compute nodes and control nodes use different interfaces, # you need to comment out "api_interface" and other interfaces from the globals.yml # and specify like below: #compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em1 [storage] #存储模块 OpenStack-sto #给112主机 [deployment] #部署模块 OpenStack-con #给110主机
5）检测所有主机是否正常通信
[root@OpenStack-con ~]# ansible -i /etc/kolla/multinode all -m ping
[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to
allow bad characters in group names by default, this will change, but still be
user configurable on deprecation. This feature will be removed in version 2.10.
Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
[WARNING]: Invalid characters were found in group names but not replaced, use
-vvvv to see details
OpenStack-con | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
OpenStack-com | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
OpenStack-sto | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
6）自动生成OpenStack各服务的密码文件
[root@OpenStack-con ~]# kolla-genpwd [root@OpenStack-con ~]# vim /etc/kolla/passwords.yml keystone_admin_password: 123456 //第164行，修改网页登录密码
7）编辑/etc/kolla/global.yml自定义OpenStack中的部署事项
[root@OpenStack-con ~]# vim /etc/kolla/globals.yml
//第14行和第15行，选择下载的基础镜像，5选1
# Valid options are ['centos', 'debian', 'oraclelinux', 'rhel', 'ubuntu']
kolla_base_distro: "centos"
//第17行和第18行，选择的安装方法，2选1。binary二进制安装，source源码安装
# Valid options are [ binary, source ]
kolla_install_type: "source"
//第20行和第21行，选择OpenStack的版本标签，详细请看：https://releases.openstack.org/
# Valid option is Docker repository tag
openstack_release: "stein" //注意版本必须小写，后期下载的OpenStack相关的docker镜像标签也为stein。我是train版本失败，才换成stein
//第23行和第24行，存放配置文件的位置
# Location of configuration overrides
#node_custom_config: "/etc/kolla/config" //默认存放地址
//第31行，OpenStack内部管理网络地址，通过该IP访问OpenStack Web页面进行管理。如果启用了高可用，需要设置为VIP（漂移IP）
kolla_internal_vip_address: "192.168.128.110"
//第88行，OpenStack内部管理网络地址的网卡接口
network_interface: "ens32"
//第93~95、98~99行去除注释，使内部通信网络都走ens32
api_interface: "{{ network_interface }}"
storage_interface: "{{ network_interface }}"
cluster_interface: "{{ network_interface }}"
tunnel_interface: "{{ network_interface }}"
dns_interface: "{{ network_interface }}"
//第120行，OpenStack外部（或公共）网络的网卡接口，可以是vlan模式或flat模式。
//此网卡应该在没有IP地址的情况下处于活动，如果不是，那么OpenStack云平台中的云主机实例将无法访问外部网络。（存在IP时br-ex桥接就不成功）
neutron_external_interface: "ens33"
//第208行，关闭高可用
enable_haproxy: "no"
//第234行，启用cinder（块存储）
enable_cinder: "yes"
//第238行，cinder（块存储）后端启用lvm
enable_cinder_backend_lvm: "yes"
//第462行，cinder（块存储）的卷组名称，需要和OpenStack-sto主机上的一致
cinder_volume_group: "cinder"
//第497行和第498行,指定nova-compute守护进程使用的虚拟化技术。（kvm好像有点问题，大家可以试试，看看你们能不能过nova下载）
//nova-compute是一个非常重要的守护进程，负责创建和终止虚拟机实例，即管理虚拟机实例的生命周期
# Valid options are [ qemu, kvm, vmware, xenapi ]
nova_compute_virt_type: "qemu"
8）通过kolla-ansible安装OpenStack所需依赖包
注意：此时会对三台主机都进行操作，请保持网络畅通
[root@OpenStack-con ~]# kolla-ansible -i /etc/kolla/multinode bootstrap-servers PLAY RECAP ********************************************************************* OpenStack-com : ok=40 changed=23 unreachable=0 failed=0 skipped=32 rescued=0 ignored=0 OpenStack-con : ok=40 changed=21 unreachable=0 failed=0 skipped=32 rescued=0 ignored=0 OpenStack-sto : ok=40 changed=23 unreachable=0 failed=0 skipped=32 rescued=0 ignored=0
9）对主机进行预部署检查
[root@OpenStack-con ~]# kolla-ansible -i /etc/kolla/multinode prechecks PLAY RECAP ********************************************************************* OpenStack-com : ok=30 changed=0 unreachable=0 failed=0 skipped=22 rescued=0 ignored=0 OpenStack-con : ok=70 changed=0 unreachable=0 failed=0 skipped=77 rescued=0 ignored=0 OpenStack-sto : ok=23 changed=0 unreachable=0 failed=0 skipped=11 rescued=0 ignored=0
10）编辑docker volume卷挂载方式，并指定docker加速器
注意：三台主机都需要进行设置，设置方法一样
//由于不存在docker卷挂载配置文件，所以需要手动生成
# mkdir -p /etc/systemd/system/docker.service.d/
# vim /etc/systemd/system/docker.service.d/kolla.conf
[Service]
MountFlags=shared
//指定加速器，这里使用阿里云的加速器
# tee /etc/docker/daemon.json << 'EOF'
{
"registry-mirrors": ["https://8mkqrctt.mirror.aliyuncs.com"]
}
EOF
# systemctl daemon-reload
# systemctl restart docker && systemctl enable docker
11）拉取OpenStack的镜像
[root@OpenStack-con ~]# kolla-ansible -i /etc/kolla/multinode pull PLAY RECAP ********************************************************************* OpenStack-com : ok=18 changed=6 unreachable=0 failed=0 skipped=11 rescued=0 ignored=0 OpenStack-con : ok=37 changed=15 unreachable=0 failed=0 skipped=52 rescued=0 ignored=0 OpenStack-sto : ok=14 changed=4 unreachable=0 failed=0 skipped=4 rescued=0 ignored=0 [root@OpenStack-con ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE kolla/centos-source-horizon stein e1251effe3cb 2 hours ago 1.04GB kolla/centos-source-heat-engine stein 5c71a1370cb6 2 hours ago 895MB kolla/centos-source-heat-api-cfn stein af03c22f8f12 2 hours ago 895MB kolla/centos-source-heat-api stein 413ecad34b48 2 hours ago 895MB kolla/centos-source-neutron-server stein fcee7f0b408d 2 hours ago 1.03GB kolla/centos-source-neutron-l3-agent stein 6067e9aaefe7 2 hours ago 1.04GB kolla/centos-source-neutron-openvswitch-agent stein af9a9abe078f 2 hours ago 1GB kolla/centos-source-neutron-dhcp-agent stein 0cdb73c0c93a 2 hours ago 1GB kolla/centos-source-neutron-metadata-agent stein 81917e943f01 2 hours ago 1GB kolla/centos-source-nova-api stein c68ebcfb98aa 2 days ago 1.09GB kolla/centos-source-nova-novncproxy stein a026c0d1309e 2 days ago 1.06GB kolla/centos-source-glance-api stein f8b487bd7770 2 days ago 911MB kolla/centos-source-nova-scheduler stein 5e98a6ff422d 2 days ago 1.03GB kolla/centos-source-nova-conductor stein da65aeb3c40f 2 days ago 1.03GB kolla/centos-source-placement-api stein f15badd1d2fc 2 days ago 920MB kolla/centos-source-cinder-api stein 993d5361e1b6 2 days ago 1.09GB kolla/centos-source-keystone-fernet stein 83ba4f35ee57 2 days ago 921MB kolla/centos-source-keystone-ssh stein d616e4544f1d 2 days ago 922MB kolla/centos-source-cinder-scheduler stein 3f06dc4ddb70 2 days ago 1.02GB kolla/centos-source-keystone stein bb43ae11542a 2 days ago 921MB kolla/centos-source-openvswitch-db-server stein 893e4aa4c7fa 2 days ago 424MB kolla/centos-source-openvswitch-vswitchd stein c4fc759fbd81 2 days ago 424MB kolla/centos-source-memcached stein eff3e9a80876 2 days ago 407MB kolla/centos-source-rabbitmq stein 7163a6f28c55 2 days ago 486MB kolla/centos-source-kolla-toolbox stein 6dd5d80b8505 2 days ago 688MB kolla/centos-source-mariadb stein cf42fce94264 2 days ago 594MB kolla/centos-source-fluentd stein de5a46efdcff 2 days ago 540MB kolla/centos-source-chrony stein 6a81938d2b3b 2 days ago 407MB kolla/centos-source-cron stein 5b83db419abb 2 days ago 406MB [root@OpenStack-com ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE kolla/centos-source-nova-compute stein aac534e5f463 2 hours ago 1.85GB kolla/centos-source-nova-ssh stein 095013bc0a50 2 hours ago 1.06GB kolla/centos-source-neutron-openvswitch-agent stein af9a9abe078f 2 hours ago 1GB kolla/centos-source-openvswitch-db-server stein 893e4aa4c7fa 2 days ago 424MB kolla/centos-source-openvswitch-vswitchd stein c4fc759fbd81 2 days ago 424MB kolla/centos-source-nova-libvirt stein a8e165c353f5 2 days ago 1.2GB kolla/centos-source-kolla-toolbox stein 6dd5d80b8505 2 days ago 688MB kolla/centos-source-fluentd stein de5a46efdcff 2 days ago 540MB kolla/centos-source-iscsid stein 7e9884caedc9 2 days ago 411MB kolla/centos-source-chrony stein 6a81938d2b3b 2 days ago 407MB kolla/centos-source-cron stein 5b83db419abb 2 days ago 406MB [root@OpenStack-sto ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE kolla/centos-source-cinder-volume stein a4bd8f7c0633 2 days ago 1.1GB kolla/centos-source-cinder-backup stein 6c69c3f19602 2 days ago 1.09GB kolla/centos-source-kolla-toolbox stein 6dd5d80b8505 2 days ago 688MB kolla/centos-source-fluentd stein de5a46efdcff 2 days ago 540MB kolla/centos-source-iscsid stein 7e9884caedc9 2 days ago 411MB kolla/centos-source-tgtd stein d9c831e2b3a7 2 days ago 406MB kolla/centos-source-chrony stein 6a81938d2b3b 2 days ago 407MB kolla/centos-source-cron stein 5b83db419abb 2 days ago 406MB
拉取时，如果报错，可以尝试重新拉取。检查配置文件是否有问题，没有问题的前提下可以尝试换版本进行操作。
12）部署OpenStack
部署OpenStack时遇到了报错，暂时不知道怎么处理
[root@OpenStack-con ~]# kolla-ansible -i /etc/kolla/multinode deploy
TASK [mariadb : Creating haproxy mysql user] ********************************************************************
fatal: [OpenStack-con]: FAILED! => {
"changed": false,
"msg": "Can not parse the inner module output: localhost | SUCCESS => {\n \"changed\": false, \n \"user\": \"haproxy\"\n}\n"
}
[root@OpenStack-con ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dbf2e410563a kolla/centos-source-mariadb:stein "dumb-init -- kolla_…" 4 days ago Up About an hour mariadb
a4b2668bff0d kolla/centos-source-chrony:stein "dumb-init --single-…" 4 days ago Up About an hour chrony
0e072ba208f9 kolla/centos-source-cron:stein "dumb-init --single-…" 4 days ago Up About an hour cron
36cd35ad03fb kolla/centos-source-kolla-toolbox:stein "dumb-init --single-…" 4 days ago Up About an hour kolla_toolbox
0463ab847e1c kolla/centos-source-fluentd:stein "dumb-init --single-…" 4 days ago Up About an hour fluentd
[root@OpenStack-con ~]# docker stop mariadb
mariadb
[root@OpenStack-con ~]# docker rm mariadb
mariadb
[root@OpenStack-con ~]# rm -rf /var/lib/docker/volumes/mariadb/_data/*13）验证部署
[root@OpenStack-con ~]# kolla-ansible -i /etc/kolla/multinode post-deploy
PLAY RECAP *********************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
[root@OpenStack-con ~]# cat /etc/kolla/admin-openrc.sh //会生成该文件，查看一下
# Clear any old environment that may conflict.
for key in $( set | awk '{FS="="} /^OS_/ {print $1}' ); do unset $key ; done
export OS_PROJECT_DOMAIN_NAME=Default
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_NAME=admin
export OS_TENANT_NAME=admin
export OS_USERNAME=admin //网页的账号
export OS_PASSWORD=123456 //网页的密码
export OS_AUTH_URL=http://192.168.128.110:35357/v3
export OS_INTERFACE=internal
export OS_ENDPOINT_TYPE=internalURL
export OS_IDENTITY_API_VERSION=3
export OS_REGION_NAME=RegionOne
export OS_AUTH_PLUGIN=password
理论上这样就可以使用了，但是卡住了，暂时不知道怎么解决。