02 Devstack部署

配置pip源

mkdir .pip
cat > ~/.pip/pip.conf <<"EOF"
[global]
trusted-host =  mirrors.aliyun.com
index-url = https://mirrors.aliyun.com/pypi/simple
EOF

配置apt源

cp -rf /etc/apt/sources.list{,.bak}
cat >/etc/apt/sources.list<<"EOF"
deb http://repo.huaweicloud.com/ubuntu/ bionic main restricted
deb http://repo.huaweicloud.com/ubuntu/ bionic-updates main restricted
deb http://repo.huaweicloud.com/ubuntu/ bionic universe
deb http://repo.huaweicloud.com/ubuntu/ bionic-updates universe
deb http://repo.huaweicloud.com/ubuntu/ bionic multiverse
deb http://repo.huaweicloud.com/ubuntu/ bionic-updates multiverse
deb http://repo.huaweicloud.com/ubuntu/ bionic-backports main restricted universe multiverse
deb http://repo.huaweicloud.com/ubuntu bionic-security main restricted
deb http://repo.huaweicloud.com/ubuntu bionic-security universe
deb http://repo.huaweicloud.com/ubuntu bionic-security multiverse
deb http://archive.ubuntu.com/ubuntu bionic universe
EOF

下载代码

git clone http://git.trystack.cn/openstack/devstack -b stable/victoria
#git clone https://opendev.org/openstack/devstack -b stable/victoria

配置 stack 用户

由于当前DevStack脚本已不支持直接使用root用户，所以需要自己创建stack用户。

groupadd stack ;useradd -g stack -s /bin/bash -d /opt/stack -m stack
echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
#或 ./devstack/tools/create-stack-user.sh


mv devstack/ /opt/stack/
chown -R stack: /opt/stack/devstack/

创建local.conf

在 /opt/stack/devstack 目录下，创建 local.conf。

su - stack
cd devstack/

如果没有指定disable_service或enable_service，默认安装全部的service。
1. key
2. n-api,n-cpu,n-cond,n-sch,n-novnc,n-api-meta
3. placement-api,placement-client
4. g-api
5. c-sch,c-api,c-vol
6. q-svc,q-dhcp,q-meta,q-agt,q-l3
7. horizon,rabbit
8. tempest,mysql,etcd3,dstat
公共组件，通过指定HOST进行判断是否安装。

情形1：all-in-one+单网卡eth0配置

[[local|localrc]]
HOST_IP=10.211.55.101

ADMIN_PASSWORD=111111
DATABASE_PASSWORD=111111
RABBIT_PASSWORD=111111
SERVICE_PASSWORD=$ADMIN_PASSWORD

LOGFILE=$DEST/logs/stack.sh.log
LOGDAYS=2

DOWNLOAD_DEFAULT_IMAGES=False
IMAGE_URLS="http://oss.cn-north-3.inspurcloudoss.com/tmp/cirros-no_cloud-0.3.0-x86_64-disk.img"

# use TryStack git mirror (使用国内的 devstack 镜像站点)
GIT_BASE=http://git.trystack.cn
NOVNC_REPO=http://git.trystack.cn/kanaka/noVNC.git
SPICE_REPO=http://git.trystack.cn/git/spice/spice-html5.git

#INSTALL_TEMPEST=false
#disable_service tempest

#NEUTRON_CREATE_INITIAL_NETWORKS=False

此时服务的状态：

stack@ubuntu1:~/devstack$ egrep -v '^#|^$' /etc/neutron/plugins/ml2/ml2_conf.ini
[DEFAULT]
[ml2]
tenant_network_types = vxlan
extension_drivers = port_security
mechanism_drivers = openvswitch,linuxbridge
[ml2_type_flat]
flat_networks = public,
[ml2_type_geneve]
vni_ranges = 1:1000
[ml2_type_gre]
tunnel_id_ranges = 1:1000
[ml2_type_vlan]
network_vlan_ranges = public
[ml2_type_vxlan]
vni_ranges = 1:1000
[ovs_driver]
[securitygroup]
firewall_driver = openvswitch
[sriov_driver]
[agent]
tunnel_types = vxlan
root_helper_daemon = sudo /usr/local/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
[ovs]
datapath_type = system
bridge_mappings = public:br-ex
tunnel_bridge = br-tun
local_ip = 10.211.55.101

stack@ubuntu1:~/devstack$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:1c:42:03:47:fa brd ff:ff:ff:ff:ff:ff
    inet 10.211.55.101/24 brd 10.211.55.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:0:21c:42ff:fe03:47fa/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 2591616sec preferred_lft 604416sec
    inet6 fe80::21c:42ff:fe03:47fa/64 scope link
       valid_lft forever preferred_lft forever

5: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:75:82:51 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
6: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN group default qlen 1000
    link/ether 52:54:00:75:82:51 brd ff:ff:ff:ff:ff:ff
24: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether da:36:65:c7:9f:11 brd ff:ff:ff:ff:ff:ff
25: br-int: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN group default qlen 1000
    link/ether 92:0e:85:10:31:4e brd ff:ff:ff:ff:ff:ff
26: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether ae:99:9f:1d:cb:4d brd ff:ff:ff:ff:ff:ff
    inet 172.24.4.1/24 scope global br-ex
       valid_lft forever preferred_lft forever
    inet6 2001:db8::2/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::ac99:9fff:fe1d:cb4d/64 scope link
       valid_lft forever preferred_lft forever
27: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 92:31:ee:cb:8a:4b brd ff:ff:ff:ff:ff:ff
33: tapf6b5b3d3-95: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc fq_codel master ovs-system state UNKNOWN group default qlen 1000
    link/ether fe:16:3e:9d:e3:dd brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc16:3eff:fe9d:e3dd/64 scope link
       valid_lft forever preferred_lft forever
       
stack@ubuntu1:~/devstack$ ip r
default via 10.211.55.1 dev eth0 proto static
10.211.55.0/24 dev eth0 proto kernel scope link src 10.211.55.101
172.24.4.0/24 dev br-ex proto kernel scope link src 172.24.4.1
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown

办公网络可以与宿主机互通，而且宿主机上有floating ip的网关。所以可以通过如下方式打通办公网络访问虚机。

#宿主机开启内核转发
sysctl -w net.ipv4.ip_forward=1
echo 'net.ipv4.ip_forward=1' >>/etc/sysctl.conf
sysctl -p

#办公机器添加路由：
ip r add 172.24.4.0/24 via 10.211.55.101 #到达floating网络的报文，下一跳为宿主机IP

情形2：多节点部署

devstack-controller

[[local|localrc]]
MULTI_HOST=true
HOST_IP=192.168.104.10 # management & api network
LOGFILE=/opt/stack/logs/stack.sh.log


# Credentials
ADMIN_PASSWORD=admin
MYSQL_PASSWORD=secret
RABBIT_PASSWORD=secret
SERVICE_PASSWORD=secret
SERVICE_TOKEN=abcdefghijklmnopqrstuvwxyz

# enable neutron-ml2-vlan
disable_service n-net
enable_service q-svc,q-agt,q-dhcp,q-l3,q-meta,neutron,q-lbaas,q-fwaas,q-vpn
Q_AGENT=linuxbridge
ENABLE_TENANT_VLANS=True
TENANT_VLAN_RANGE=3001:4000
PHYSICAL_NETWORK=default

LOG_COLOR=False
LOGDIR=$DEST/logs
SCREEN_LOGDIR=$LOGDIR/screen

# use TryStack git mirror
GIT_BASE=http://git.trystack.cn
NOVNC_REPO=http://git.trystack.cn/kanaka/noVNC.git
SPICE_REPO=http://git.trystack.cn/git/spice/spice-html5.git

本次采用的是网卡1设置为网络地址转换（NAT），网卡2设置为桥接网卡。

[[local][localrc]]
HOST_IP=10.170.3.219

PUBLIC_INTERFACE=eth1 #public network
FLOATING_RANGE=10.0.2.0/29

FLAT_INTERFACE=eth1
FIXED_RANGE=10.170.3.0/28

PUBLIC_NETWORK_GATEWAY=10.0.2.2

#MYSQL_HOST=10.170.3.3
#RABBIT_HOST=10.170.3.3
#GLANCE_HOSTPORT=10.170.3.3:9292
#KEYSTONE_AUTH_HOST=10.170.3.3
#KEYSTONE_SERVICE_HOST=10.170.3.3
#FIXED_NETWORK_SIZE=15

MULTI_HOST=1
LOGFILE=/opt/stack/logs/stack.sh.log

ADMIN_PASSWORD=admin
DATABASE_PASSWORD=secret
RABBIT_PASSWORD=secret
SERVICE_PASSWORD=secret
SERVICE_TOKEN=abcdefghijklmnopqrstuvwxyz

#enable neutron-ml2-vxlan
disable_service n-net
enable_service q-svc,q-agt,q-dhcp,q-13,q-meta,q-metering,q-lbaas,neutron,tempest,heat

#OFFLINE=True

devstack-compute

[[local|localrc]]
MULTI_HOST=true
HOST_IP=192.168.104.11 # management & api network

# Credentials
ADMIN_PASSWORD=admin
MYSQL_PASSWORD=secret
RABBIT_PASSWORD=secret
SERVICE_PASSWORD=secret
SERVICE_TOKEN=abcdefghijklmnopqrstuvwxyz

# Service information
SERVICE_HOST=192.168.104.10
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
Q_HOST=$SERVICE_HOST
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST
CEILOMETER_BACKEND=mongodb
DATABASE_TYPE=mysql

ENABLED_SERVICES=n-cpu,q-agt,neutron
Q_AGENT=linuxbridge
ENABLE_TENANT_VLANS=True
TENANT_VLAN_RANGE=3001:4000
PHYSICAL_NETWORK=default

# vnc config
NOVA_VNC_ENABLED=True
NOVNCPROXY_URL="http://$SERVICE_HOST:6080/vnc_auto.html"
VNCSERVER_LISTEN=$HOST_IP
VNCSERVER_PROXYCLIENT_ADDRESS=$VNCSERVER_LISTEN


LOG_COLOR=False
LOGDIR=$DEST/logs
SCREEN_LOGDIR=$LOGDIR/screen

# use TryStack git mirror (使用国内的 devstack 镜像站点)
GIT_BASE=http://git.trystack.cn
NOVNC_REPO=http://git.trystack.cn/kanaka/noVNC.git
SPICE_REPO=http://git.trystack.cn/git/spice/spice-html5.git

 [[local|localrc]]
MULTI_HOST=true

HOST_IP=10.170.3.220 # management IP
FLAT_INTERFACE=eth1
FIXED_RANGE=10.170.3.0/24
#FIXED_NETWORK_SIZE=15
FLOATING_RANGE=10.170.3.0/28

PUBLIC_NETWORK_GATEWAY=10.0.2.2
TUNNEL_ENDPOINT_IP=10.170.3.220

#MULTI_HOST=1
LOGFILE=/opt/stack/logs/stack.sh.log

# Credentials
ADMIN_PASSWORD=admin
MYSQL_PASSWORD=secret
RABBIT_PASSWORD=secret
SERVICE_PASSWORD=secret
SERVICE_TOKEN=abcdefghijklmnopqrstuvwxyz

#Use mirror
GIT_BASE=http://git.trystack.cn
NOVNC_REPO=http://git.trystack.cn/kanaka/noVNC.git
SPICE_REPO=http://git.trystack.cn/git/spice/sipce-htm15.git

# Service information
SERVICE_HOST=10.170.3.219
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE-HOST
GLANCE_HOSTPORT=10.170.3.219:9292
Q_HOST=$SERVICE_HOST
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CEILOMETER_BACKEND=mongodb

DATABASE_TYPE=mysql
ENABLED_SERVICES=n-cpu,n-net,n-api-meta,c-vol

# vnc config
NOVA_VNC_ENABLED=True
NOVNCPROXY_URL="http://10.170.3.219:6080/vnc_auto.html"
VNCSERVER_LISTEN=$HOST_IP
VNCSERVER_PROXYCLIENT_ADDRESS=$VNCSERVER_LISTEN

LOG_COLOR=Flase
LOGDIR=$DEST/logs
SCREEN_LOGDIR=$LOGDIR/screen

# OFFLINE=True

开始部署

分别在 devstack-controller 和 devstak-compute 上执行命令

./stack.sh

会输出各项操作的结果。日志会写到 stack.sh.log 文件。整个过程需要连接Internet，网速慢可能会花较长时间，成功后最后会打印出相关信息。

部署完之后的清理工作

现在关闭openstack相应的服务已经变得非常简单了，只需要运行 unstack.sh脚本。

./unstack.sh

更加深度的清除工作我们可以使用clean.sh脚本。这种模式清除可以删除掉一些明确有问题的包，并且修改掉原有部署的数据库和消息队列管理器。可以理解为深度清理。

./clean.sh

有些时候正在运行的实例可能无法被清理，Devstack会尝试清除正在运行的实例但有时候仍然需要手动来完成清除操作。

sudo rm -rf /etc/libvirt/qemu/inst*
sudo virsh list | grep inst | awk '{print $1}' | xargs -n1 virsh destroy

参考文档

all_in_one：https://docs.openstack.org/devstack/latest/configuration.html#

多节点部署：https://docs.openstack.org/devstack/latest/guides/multinode-lab.html

Using DevStack with neutron Networking

网络说明：https://docs.openstack.org/devstack/latest/networking.html

https://blog.csdn.net/minxihou/article/details/51930202