Install OpenStack from source - part 2

In the first article of this series, we started installing OpenStack from source. We installed keystone and populated it with some basic information including a Services project and an admin user for our new OpenStack install. Additionally, in an initial script we setup users and directories for the upcoming installs of the Image service (glance), Networking service (neutron), Compute service (nova) and Volume service (cinder). Now, let’s continue and install and start the glance process on the controller node.

In the first article, we added some information into our bashrc file to set a couple of shell variables to use in the remaining sets.. Verify that at least one of these variables are set:

echo $MY_IP

To start installing the Image service (glance), we need to make a few additional glance sub-directories and ensure that they have the proper permissions (the basic glance directories were created by a script in the previous article):

mkdir -p /var/lib/glance/images
mkdir -p /var/lib/glance/scrubber
mkdir -p /var/lib/glance/image-cache

Next, download (clone the repo) and install glance:

git clone -b stable/kilo

Note: Remember that we can change to install from trunk by eliminating the -b stable/kilo from the above command.

cp -R glance/etc/* /etc/glance/
cd glance
python install

Configure glance by setting some basic information like database connection information, message queue info and keystone connection information:

sed -i "s|#connection = <None>|connection = mysql://glance:glance@$MY_PRIVATE_IP/glance|g" /etc/glance/glance-api.conf
sed -i "s/rabbit_host = localhost/rabbit_host = $MY_PRIVATE_IP/g" /etc/glance/glance-api.conf
sed -i "s/auth_host = = $MY_PRIVATE_IP/g" /etc/glance/glance-api.conf
sed -i 's/%SERVICE_TENANT_NAME%/service/g' /etc/glance/glance-api.conf
sed -i 's/%SERVICE_USER%/glance/g' /etc/glance/glance-api.conf
sed -i 's/%SERVICE_PASSWORD%/glance/g' /etc/glance/glance-api.conf
sed -i 's/#flavor=/flavor = keystone/g' /etc/glance/glance-api.conf
sed -i 's/#show_image_direct_url = False/show_image_direct_url = True/g' /etc/glance/glance-api.conf

Configure the Glance-Registry with the same information as above:

sed -i "s|#connection = <None>|connection = mysql://glance:glance@$MY_PRIVATE_IP/glance|g" /etc/glance/glance-registry.conf
sed -i "s/auth_host = = $MY_PRIVATE_IP/g" /etc/glance/glance-registry.conf
sed -i 's/%SERVICE_TENANT_NAME%/service/g' /etc/glance/glance-registry.conf
sed -i 's/%SERVICE_USER%/glance/g' /etc/glance/glance-registry.conf
sed -i 's/%SERVICE_PASSWORD%/glance/g' /etc/glance/glance-registry.conf
sed -i 's/#flavor=/flavor = keystone/g' /etc/glance/glance-registry.conf
sed -i 's|#connection = <None>|connection = mysql://glance:glance@$MY_IP|g' /etc/glance/glance-api.conf
sed -i 's|#connection = <None>|connection = mysql://glance:glance@$MY_IP|g' /etc/glance/glance-registry.conf

Create the MySQL database for glance and populate the newly created database with the glance tables:

mysql -u root -pmysql -e 'CREATE DATABASE glance;'
mysql -u root -pmysql -e "GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'localhost' IDENTIFIED BY 'glance';"
mysql -u root -pmysql -e "GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'%' IDENTIFIED BY 'glance';"
glance-manage db_sync

Although not needed in this environment, we will install glance-image-sync and set it to run on a regular basis. If we had multiple glance API nodes this would sync the glance images between the nodes:

git clone
pushd glance-image-sync
mv /usr/local/bin/glance-image-sync
chmod +x /usr/local/bin/glance-image-sync

cat >> /etc/glance/glance-image-sync << EOF
api_nodes = $MY_IP
rsync_user = glance
lock_file = /var/run/glance-image-sync

cat >>  /etc/cron.d/glance-image-syn << EOF
*/5 * * * * glance /usr/local/bin/glance-image-sync

We must remember to rotate the glance log files:

cat >> /etc/logrotate.d/glance << EOF
/var/log/glance/*.log {
        rotate 7

If you do not get the Identity service signing certificates for the Image service, the Image API requests fail. Prevent this by getting the keystone signing certs for glance:

cp /etc/keystone/ssl/certs/ca.pem /var/lib/glance/cacert.pem
cp /etc/keystone/ssl/certs/signing_cert.pem /var/lib/glance/
chown glance:glance /var/lib/glance/*
chmod 600 /var/lib/glance/cacert.pem
chmod 600 /var/lib/glance/signing_cert.pem

We are almost ready to start glance, but we must first create the glance upstart scripts:

Note: These scripts are just copied from the scripts that the Ubuntu operating system packaged version of glance installs.

cat >> /etc/init/glance-api.conf << EOF
description "Glance API server"
author "Soren Hansen <>"

start on runlevel [2345]
stop on runlevel [!2345]


exec start-stop-daemon --start --chuid glance --exec /usr/local/bin/glance-api -- --config-file=/etc/glance/glance-api.conf --config-file=/etc/glance/glance-api-paste.ini

cat >> /etc/init/glance-registry.conf << EOF
description "Glance registry server"
author "Soren Hansen <>"

start on runlevel [2345]
stop on runlevel [!2345]


exec start-stop-daemon --start --chuid glance --exec /usr/local/bin/glance-registry -- --config-file=/etc/glance/glance-registry.conf --config-file=/etc/glance/glance-registry-paste.ini


Start glance and verify that is is running:

start glance
start glance-registry

Wait about 15 seconds and run the following to verify that glance is running:

ps aux|grep glance

If glance started, we should see a line from the last command showing information about the running process. If that didn’t happen and glance didn’t start or stay running use the following to manually start glance for troubleshooting:

sudo -u glance glance-api --config-file=/etc/glance/glance-api.conf --config-file=/etc/glance/glance-api-paste.ini > /dev/null 2>&1 &

sudo -u glance glance-registry --config-file=/etc/glance/glance-registry.conf --config-file=/etc/glance/glance-registry-paste.ini > /dev/null 2>&1 &

Note: Setting the value debug = True in the file /etc/glance/glance.conf will increade the logging output, for help in debugging failures.

Next, install an image into glance for use once we have OpenStack fully installed as a basis for VMs. There is a small cloud image based on the Ubuntu operating system that is available, named cirros. With glance running download the cirros image and load it into glance:


glance image-create --name=cirros-qcow2 \
                    --disk-format=qcow2 \
                    --container-format=bare \
                    --is-public=true < cirros-0.3.3-x86_64-disk.img

Our second process is now running and we can now turn our attention to installing the Networking service (neutron). This is the first of three neutron installs that we will have to do, the neutron server on the controller node and the neutron agents on the network and compute nodes.

Since the nova, neutron and glance processes need root or sudo privileges to accomplish various tasks, we need to create special sudo permissions for these process users. OpenStack also limits the extent of these privileges through use of a rootwrap configuration. Run the following script to set up the sudo permissions:

for SERVICE in neutron nova cinder
cat > '/etc/sudoers.d/'$SERVICE'_sudoers' << EOF
Defaults:$SERVICE !requiretty

$SERVICE ALL = (root) NOPASSWD: /usr/local/bin/$SERVICE-rootwrap  /etc/$SERVICE/rootwrap.conf *

chmod 440 /etc/sudoers.d/$SERVICE_sudoers
chmod 750 /etc/sudoers.d

Some of the pip prerequisites have a couple of the package prerequisites, that aren’t automatically installed. Install these packages now:

apt-get -y install git mysql-client python-dev libxml2-dev libffi-dev

Clone the neutron repon:

git clone -b stable/kilo

Copy the provided configuration files:

cp neutron/etc/* /etc/neutron/

Copy some needed configuration subdirectories:

cp -R neutron/etc/neutron/plugins/ml2/* /etc/neutron/plugins/ml2
cp -R neutron/etc/neutron/rootwrap.d/* /etc/neutron/rootwrap.d

Now install neutron:

cd neutron
python install
cd ~

Create the MySQL database for neutron:

mysql -u root -pmysql -e 'CREATE DATABASE neutron;'
mysql -u root -pmysql -e "GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' IDENTIFIED BY 'neutron';"
mysql -u root -pmysql -e "GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'%' IDENTIFIED BY 'neutron';"

The sample data script that was run in the previous article does not insert data for the neutron service. Create Neutron service user in keystone:

keystone user-create --tenant service --name neutron --pass neutron

Grant the ‘admin’ role to the newly created Neutron service user:

keystone user-role-add --user neutron --tenant service --role admin

And verify that the user creation worked. List the service users and roles and verify that the neutron user exists and has the admin role:

keystone user-list --tenant Services
keystone user-role-list --tenant service --user neutron

You should see in the output of the two previous commands that the neutron user was created and has the proper permissions. Now populate a neutron service into the keystone service catalog:

keystone service-create --name=neutron --type=network --description="Neutron Network Service"

Place the neutron endpoint information into the keystone service catalog:

keystone endpoint-create --region RegionOne --service neutron --publicurl=https://$MY_PUBLIC_IP:9696 --internalurl=https://$MY_PRIVATE_IP:9696 --adminurl=https://$MY_PRIVATE_IP:9696

SERVICE_TENANT_ID=`keystone tenant-get service | awk '/ id / { print $4 }'`

Finally, we must configure neutron. If the basic configuration file is supplied in the package close process, remove it, because it is very long and difficult to edit. Next, the following sequence will replace it with a simpler one (the original is still in the cloned neutron directory if we ever need to look at it in the future):

rm /etc/neutron/neutron.conf

cat > /etc/neutron/neutron.conf << EOF
verbose = True
debug = True
core_plugin = ml2
service_plugins = router
auth_strategy = keystone
allow_overlapping_ips = True
dhcp_agents_per_network = 1
notify_nova_on_port_status_changes = True
notify_nova_on_port_data_changes = True
nova_url = https://$MY_PRIVATE_IP:8774/v2
nova_admin_username = nova
nova_admin_tenant_id = $SERVICE_TENANT_ID
#nova_admin_tenant_name = service
nova_admin_password = nova
nova_admin_auth_url = https://$MY_PRIVATE_IP:35357/v2.0

root_helper=sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
auth_uri = https://$MY_PRIVATE_IP:35357/v2.0/
identity_uri = https://$MY_PRIVATE_IP:5000
admin_tenant_name = service
admin_user = neutron
admin_password = neutron
connection = mysql://neutron:neutron@$MY_PRIVATE_IP/neutron

lock_path = /var/lock/neutron

rabbit_host = $MY_PRIVATE_IP

Rotate the neutron log files:

cat >> /etc/logrotate.d/neutron << EOF
/var/log/neutron/*.log {
        rotate 7

Neutron supports multiple networking layer 2 technologies through the ML2 plugin. Configure the neutron ML2 plugin agent to use GRE tunnels for project network isolation. Although this agent doesn’t run on the controller node, the neutron server needs to know about the plugin:

rm /etc/neutron/plugins/ml2/ml2_conf.ini
cat > /etc/neutron/plugins/ml2/ml2_conf.ini  << EOF

type_drivers = gre
tenant_network_types = gre
mechanism_drivers = openvswitch

tunnel_id_ranges = 1:1000

enable_security_group = True
enable_ipset = True
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver

chown neutron:neutron /etc/neutron/*.{conf,json,ini}
chown -R neutron:neutron /etc/neutron/plugins

cat > /etc/default/neutron << EOF

As we did for glance, build the neutron database tables:

neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head

Before we start the neutron process, we need to give the startup script some options information so it will read the ml2 configuration file:

cat > /etc/default/neutron-server << EOF

And, finally, create the upstart scripts that are used to start the neutron server process:

cat > /etc/init/neutron-server.conf << EOF
# vim:set ft=upstart ts=2 et:

start on runlevel [2345]
stop on runlevel [!2345]

  [ -r /etc/default/neutron-server ] && . /etc/default/neutron-server
  exec start-stop-daemon --start --chuid neutron --exec /usr/local/bin/neutron-server -- \
    --config-file /etc/neutron/neutron.conf \
    --log-file /var/log/neutron/server.log \$CONF_ARG
end script


Start neutron server running and verify that it stays running. The second line gives information about the running neutron process (again wait for about 15 seconds before running the second command):

start neutron
ps aux|grep neutron

If neutron will not start, use the following line to manually start neutron. If the neutron process has errors in starting, it gives you output to aid in debugging the problem.

sudo -u neutron neutron-server --config-file=/etc/neutron/neutron.conf --config-file=/etc/neutron/plugins/ml2/ml2_conf.ini --log-file /var/log/neutron/server.log

In the next article of this series, we install several nova processes onto the controller node.

post avatar
Phil Hopkins

Share this information: