Chapter 2. Install and configure

These instructions tell you how to install and configure:

  • raxmon, which is the Cloud Monitoring command-line interface (CLI)

  • The Cloud Monitoring Agent

 2.1. Install and configure raxmon

You need to use the Cloud Monitoring command-line interface (CLI), raxmon, for some preliminary agent configuration. raxmon is also helpful if you use the tutorial in Chapter 3, Create your first monitor. raxmon runs on your local workstation (not the servers you plan to monitor).

[Note]Note

The $ symbol represents a command line prompt, do not type it.

 2.1.1. Requirements

Before you install raxmon, meet these requirements on your local workstation:

  • Install Python 2.5, 2.6, or 2.7 on your local workstation.

  • Make sure you install raxmon on your local workstation (not the server you plan to monitor). This allows you to install raxmon once to monitor many remote servers.

 2.1.2. Install raxmon

Follow these steps to install raxmon on your local workstation if you don't already have a version installed.

Upgrade raxmon if you have an earlier version installed. Version 0.4.6 or higher is required. See Section 2.1.3, “Upgrade raxmon” for details.

 

To install raxmon:

  1. Log into a bash shell on your local workstation.

    If you don't use a virtual environment, log into a shell as the user who has write permissions to /usr/local/bin on your local machine.

  2. Install pip.

    To install the rackspace-monitoring-cli package, you need the python pip command.  If you need access to that tool, enter these commands:

    $ sudo yum install -y python-setuptools  
    $ sudo easy_install pip  
  3. Install raxmon.

    Enter this command to install raxmon:

    $ sudo pip install rackspace-monitoring-cli  

 2.1.3. Upgrade raxmon

Upgrade raxmon on your local workstation if you have an earlier version installed. Version 0.4.6 or higher is required.

 

To upgrade to a new version of raxmon:

  1. Remove raxmon.

    Remove the previous version of raxmon before running the upgrade:

    $ sudo pip uninstall rackspace-monitoring-cli  
  2. Upgrade to the latest version.

    Enter this command:

    $ sudo pip install --upgrade rackspace-monitoring-cli  

 2.1.4. Configure raxmon

Configure raxmon with your credentials and URLs for Cloud Monitoring and Rackspace Cloud Identity Service.

 

To configure raxmon:

Although you can specify your Rackspace credentials and URLs as command-line options, it saves time to add them to the raxmon configuration file.

  1. Locate your API key.

    Follow these instructions to find your API key if you do not already have it: Section 1.7.2, “Obtaining an API key”.

  2. Edit the .raxrc file.

    Use vi or your favorite text editor to edit the .raxrc file in your home directory. You need to supply both your Rackspace username and API key.

    $  vi ~/.raxrc  

    Be sure to specify the correct identity endpoint for your account location. For details, see Section 1.7.1, “Identity endpoints”. Here is an example of the .raxrc file with the Rackspace credentials and URLs:

    [credentials]
    username=MyRackspaceAcct
    api_key=0000000000000000000
    
    [api]
    url=https://monitoring.api.rackspacecloud.com/v1.0
    
    [auth_api]
    url=https://identity.api.rackspacecloud.com/v2.0
    
    [ssl]
    verify=true

 2.2. Install and configure the monitoring agent

Use of the agent is optional. If you choose to use the agent, you must install it on each server that you want to monitor.

Package Manager installation is supported for these environments:

  • Ubuntu or Debian

  • Red Hat, Fedora, or Centos

For more information about the agent, see Section 1.5, “How the monitoring agent works”. For information about the agent endpoints, including attributes and code samples, see Agents in the Rackspace Cloud Monitoring Developer Guide. The Cloud Monitoring agent is distributed under the Apache license.

For information about installing the agent from source, see https://github.com/racker/virgo.

 2.2.1. Install on Ubuntu or Debian

This section tells you how to install the agent on a server running Ubuntu or Debian.

These steps have been tested on Ubuntu Lucid 10.04, but should work on most recent versions of Ubuntu or Debian.

 

To install the agent on the Ubuntu environment:

  1. Add the repository for the agent.

    Create a text file named rackspace-monitoring-agent.list in the /etc/apt/sources.list.d directory.

    Add the following content to the rackspace-monitoring-agent.list file:

    deb <repo-name> cloudmonitoring main

    where repo-name is one of the following available packages:

    • http://stable.packages.cloudmonitoring.rackspace.com/ubuntu-10.04-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/ubuntu-11.04-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/ubuntu-11.10-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/ubuntu-12.04-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/ubuntu-12.10-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/ubuntu-13.04-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/debian-squeeze-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/debian-wheezy-x86_64

    For example, to install Ubuntu, version 10.04, your file would contain this content:

    deb http://stable.packages.cloudmonitoring.rackspace.com/ubuntu-10.04-x86_64 cloudmonitoring main
  2. Add a signing key for the apt repository.

    $ curl https://monitoring.api.rackspacecloud.com/pki/agent/linux.asc | sudo apt-key add -
  3. Update your apt-get program to recognize the new repository.

    $ sudo apt-get update
  4. Install the agent.

    $ sudo apt-get install rackspace-monitoring-agent

 2.2.2. Install on Red Hat/Fedora/Centos

This section tells you how to install the agent on a server running a Red Hat, Fedora or Centos environment.

 

To install the agent on Red Hat, Fedora, or Centos:

  1. Enable the signing key.

    $ curl https://monitoring.api.rackspacecloud.com/pki/agent/<signing-key> > /tmp/signing-key.asc 
    $ sudo rpm --import /tmp/signing-key.asc 

    where <signing-key> is the correct signing key for your operating system:

    Operating System Signing Key
    redhat-5-x86_64 redhat-5.asc
    redhat-6-x86_64 linux.asc
    fedora-16-x86_6 linux.asc
    fedora-17-x86_64 linux.asc
    fedora-18-x86_64 linux.asc
    fedora-19-x86_64 linux.asc
    centos-5-x86_64 centos-5.asc
    centos-6-x86_64 linux.asc

    For example, on Redhat version 5, you enter:

    $ curl https://monitoring.api.rackspacecloud.com/pki/agent/redhat-5.asc > /tmp/signing-key.asc 
    $ sudo rpm --import /tmp/signing-key.asc 
  2. Set up the yum repository.

    Create a text file named rackspace-cloud-monitoring.repo in the /etc/yum.repos.d directory.

    Add the following content to the rackspace-cloud-monitoring file:

    [rackspace]
    name=Rackspace Monitoring
    baseurl=<repo-name>
    enabled=1

    where <repo-name> is one of the following available packages:

    • http://stable.packages.cloudmonitoring.rackspace.com/redhat-5-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/redhat-6-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/fedora-16-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/fedora-17-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/fedora-18-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/fedora-19-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/centos-5-x86_64

    • http://stable.packages.cloudmonitoring.rackspace.com/centos-6-x86_64

    For example, to install Redhat, version 6, your file would contain this content:

    [rackspace]
    name=Rackspace Monitoring
    baseurl=http://stable.packages.cloudmonitoring.rackspace.com/redhat-6-x86_64/
    enabled=1
  3. Install the agent packages.

    $ sudo yum update
    $ sudo yum install rackspace-monitoring-agent

 2.2.3. Install on Windows

This section tells you how to install the agent on a server running a Windows environment.

 

To install the agent on Windows:

  1. Download the latest stable agent installer.

    Operating System URL
    64-bit systems: Windows 2008 and Windows 2012 http://stable.packages.cloudmonitoring.rackspace.com/rackspace-monitoring-agent-x64.msi
    32-bit legacy systems: Windows 2008 http://stable.packages.cloudmonitoring.rackspace.com/rackspace-monitoring-agent.msi
  2. Run the installer.

    The installer will automatically exit when it is complete.

Next, run the agent setup program to generate a configuration file. Without a configuration file, the agent will exit and be restarted by the Windows service manager. To prevent this continuous restarting, run the setup program immediately after installing the agent.

 2.2.4. Set up the agent

The agent Setup program provides the easiest way to get started with the agent. Setup takes care of these configuration tasks for you:

  • Configures an agent token that the agent uses to authenticate with Cloud Monitoring.

  • Creates an agent configuration file for you.

  • Verifies connectivity to the Rackspace datacenters.

  • Associates the agent with a monitoring entity.

You can also manually edit the agent configuration file. See Section 2.3, “Manual agent configuration” for details.

 

To run the agent setup program:

  1. Before you start, make sure you have your Rackspace user name and API key. If not, see: Section 1.7.2, “Obtaining an API key”.

  2. Log in as the root user on the server where you installed the agent package.

  3. Enter this command to run Setup:

    $ sudo rackspace-monitoring-agent --setup

    A list of Setup Settings appears, which includes the agent ID. The agent ID matches the hostname of the server.

  4. Respond to the prompts and supply your Rackspace username and API key.

  5. Specify an agent token to use when authenticating the agent with Cloud Monitoring. You have these options:

    • Select an existing agent token from the list — Setup lists the tokens that have been created under your account. You can use the same token for multiple agents, or select the option to create a new one for use on this server.

    • Create a New Token — Setup creates a token and uses it to authenticate the agent with the Cloud Monitoring endpoints.

    You should see the message: Agent successfully connected!

  6. Associate the agent with the monitoring entity for this server. You have these options:

    1. <hostname> — Setup creates an entity for the server where you installed the agent. Select this option unless you have other requirements.

    2. Create a new entity for this server — Select this option to create a new entity with an alternate label.

    3. Do not associate the agent with an entity— Select this option to associate the agent with an entity at a later time.

    Make note of the entity ID, which you'll use when you create checks to monitor the health of your server. See Section 2.2.6, “First steps with the agent”.

You're now ready to run the agent.

 2.2.5. Run the agent

After you perform the configuration and set-up tasks, you're ready to start the agent.

 

To start the agent:

  • Enter this command:

    $ sudo service rackspace-monitoring-agent start 

The rackspace-monitoring-agent command lets you manage the agent. Enter the following command to see the available options:

$ rackspace-monitoring-agent --help 

 2.2.6. First steps with the agent

After you start the agent, you can schedule some agent checks.

 

To create an entity and agent checks:

  1. Install raxmon on your local workstation.

    If you have not installed raxmon yet, do that now. See Section 2.1, “Install and configure raxmon” .

    If you prefer to use the API, instead of the raxmon CLI, see Entities and Checks in the Rackspace Cloud Monitoring Developer Guide.

  2. Create some monitoring checks.

    Now you can create some monitoring checks for the agent to run.

    $ raxmon-checks-create --entity-id=<entityId> --type=agent.memory --period=30 --label=mem 
    $ raxmon-checks-create --entity-id=<entityId> --type=agent.cpu --period=30 --label=cpu  
    $ raxmon-checks-create --entity-id=<entityId> --type=agent.filesystem --period=30 --label=root --details="target=/"

    Where <entityId> is the ID for the entity you associated with the agent.

    These commands create three monitoring checks. The --type values of agent.memory, agent.cpu, and agent.filesystem are agent check types, which means the checks will run local to the system being monitored.

 2.3. Manual agent configuration

[Note]Note

Using the Setup program is the preferred way to set up the agent. This section is provided as an alternate method of setting up the agent. If you used the Setup program to complete the agent configuration, skip this section.

 

To manually set up the agent:

  1. Install raxmon.

    If you have not installed raxmon yet, do that now. See Section 2.1, “Install and configure raxmon” .

    If you prefer to use the API, instead of the raxmon CLI, see Entities and Agent Tokens in the Rackspace Cloud Monitoring Developer Guide.

  2. Create an entity in the monitoring service:

    $ raxmon-entities-create --label=<entityLabel>   

    Where <entityLabel> is a name for the new entity. This entity represents the server where you're installing the agent. Suppose the server is named employee-news, you might use that hostname as the <entityLabel>.

    This command returns an entity ID for the new entity. For example, ent12345. You need to supply this entity ID for <entityId> in the next several steps.

  3. Assign an ID to your agent.

    You need to assign an ID to your agent and associate it with the server entity you just created. Cloud Monitoring uses this ID for two-way communication between the agent and the Cloud Monitoring endpoint.

    $ raxmon-entities-update --id=<entityId> --agent-id=<agentId>

    Where:

    <entityId>

    is the entity ID returned in the previous step.

    <agentId>

    is the ID, or name, you want to assign to the agent installed on the server. For the ID value, use any label that makes sense for the system that you plan to monitor. For example, you can use the server hostname, although the ID does not need to match or contain any part of the entity label or the entityId.

  4. Create an agent token.

    The agent uses an agent token to authenticate with the Cloud Monitoring endpoint. The token ensures that no one masquerades as your server. Enter this command to create the token:

    $ raxmon-agent-tokens-create --label="<agent-token-label>"   

    where <agent-token-label> is a name for your agent token. You can use any name you like.

  5. List the tokens.

    Enter this command to see a list of tokens, including the one you just created.

    $ raxmon-agent-tokens-list

    Make a note of the agent token value to use in the next few steps.

  6. Log in as the root user on the server where you installed the agent.

  7. Use vi or your favorite text editor to edit the file or create it if it does not exist.

    $ sudo vi /etc/rackspace-monitoring-agent.cfg  
  8. Add this content to the rackspace-monitoring-agent.cfg file:

    monitoring_id <agentId>
    monitoring_token <agentToken>
    

    Where:

    <agentId>

    is the agent ID you assigned to the agent in Step 3.

    <agentToken>

    is the token value returned for your agent token by the raxmon-agent-tokens-list command.

You're now ready to start the agent. See Section 2.2.5, “Run the agent”.

 2.4. Server-side monitoring configuration with YAML files

The server-side monitoring configuration is a new method that allows you to easily configure cloud monitoring on the server that you wish to monitor. It is especially useful in conjunction with tools such as Chef or when dealing with duplicate servers. Automation tools like Chef, Puppet, and Ansible can maintain a repository of configuration files to automatically create monitoring checks and alarms for a given server or servers. Server-side monitoring configuration helps you set up monitoring more quickly.

[Note]Note

You must update your Agent to take advantage of this new feature. The required agent version is 1.0.0-68 or later.

Server-side monitoring configuration files are written in YAML ( YAML Ain’t Markup Language), a text file with a column-based syntax. Each YAML configuration file can contain configurations for one check and its associated alarms. You create a series of YAML files, one for each check you want. A single file can be used repeatedly on many servers to configure its check and its alarms. The YAML configuration files are read every time you start the agent.

The top-level fields in the YAML file represent the check's parameters. The alarms are configured below a top-level field named alarms. Each alarm must be given a unique “handle” under the alarms field; “handle” is a new term referring to a unique name. The system uses the alarm “handle” to uniquely identify that alarm within the file. The system uses the filename to uniquely identify checks and their alarms.

[Note]Note

The handle is not the same as the alarm label in the API (or alarm name, in the Cloud Control panel) or the alarmId. Rather, it is a name that, along with the file name, uniquely identifies that alarm. It is important that the alarm handles and filenames be unique because the system uses them to identify server-side configured checks and alarms. Once an alarm or check has been created with server-side configuration from a YAML file, the system uses the filename and the handle, for an alarm, or just the filename for a check, to detect changes to those alarms or checks (i.e. an updated YAML file). When you use the API to list checks or alarms, you can look to the confd_name field to determine if that check or alarm was created by server-side configuration; if the field is non-null, then this object was created by server-side configuration.

The alarm's configuration options are identical to the configuration options used by the API, but there are two new configuration parameters specifically for use with the server-side configuration file, confd_name and confd_hash. Both are generated from the YAML file, but confd_hash is generated each time the YAML file is updated and uploaded to the agent endpoint whereas confd_name is generated at the time of the initial file deployment. API write-requests (PUT or POST) to the confd_name and confd_hash fields are ignored. When using the API, a user can tell if a change was made to a server-side configuration object without using server-side configuration, if the confd_name field is non-null and the confd_hash field is zero (0). For details on agent configuration options, see the Manual Agent Configuration.

After authentication, the agent reads the YAML configuration files and sends them to the monitoring server. The monitoring service parses the files and creates, or updates, the checks and alarms according to the content of the files.

 2.4.1. Example server-side configuration YAML files

This section offers two YAML configuration file examples.

 2.4.1.1. File system check

The following is an example server-side configuration file that sets up an agent check for a file system at the target "/". The file name is my_fs.yaml. It configures one check to alert on disk usage that exceeds 90% of free space, two alarms, and other agent check configuration options. The alarms have been given the handles of techs and its. These handles uniquely identify these alarms within the context of this check in the same manner that the filename uniquely identifies this check amongst other server-side created checks for this entity.

[Note]Note

You can find existing notification_plan_id values and criteria values through the API or the Cloud Control panel.

type        : agent.filesystem
label       : Check for Main Disk
disabled    : false
period      : 60
timeout     : 30
details     :
    target  : /
alarms      :
    techs   :
        label                 : disk used alarm
        notification_plan_id  : npTechnicalContactsEmail
        criteria              : |
            if (percentage(metric['used'], metric['total']) > 90) {
                return new AlarmStatus(CRITICAL, 'Less than 10% free space left.');
            }
            if (percentage(metric['used'], metric['total']) > 80) {
                return new AlarmStatus(WARNING, 'Less than 20% free space left.');
            }
    its     :
        label                 : disk used alarm
        notification_plan_id  : npInformationTechEmail
        criteria              : |
            if (percentage(metric['used'], metric['total']) > 95) {
                return new AlarmStatus(CRITICAL, 'Less than 5% free space left.');
            }
            if (percentage(metric['used'], metric['total']) > 90) {
                return new AlarmStatus(WARNING, 'Less than 10% free space left.');
            }            
            

 2.4.1.2. HTTP check

This example agent configuration file sets up an agent check for HTTP traffic at the target "/". The file name is my_http.yaml, it configures one check to alert on a non-responsive web server, one alarm, and other agent configuration options. The alarm has the handle of alarm1.

type           : remote.http
label          : Website check 1
timeout        : 30
period         : 90
target_alias   : default
details        : 
    url        : http://www.foo.com
    method     : GET
monitoring_zones_poll: 
               - mzord
alarms         :
    alarm1     :
        label                 : http connect alarm
        notification_plan_id  : npTechnicalContactsEmail
            

For additional server-side agent configuration file examples, see Appendix A, Server-Side agent configuration YAML file examples

 2.4.2. How to use a server-side configuration YAML file

This section describes how to create, update, and delete, a server-side YAML configuration file.

 2.4.2.1. Create a server-side configuration YAML file

 

To create a server-side configuration YAML file:

  1. Using any text editor, create a YAML file, specifying the extension as ".yaml". YAML files are column-based, the columns being created with whitespace indentation. Use spaces to add the indents, tabs are ignored. The number of spaces used to create the indents is unimportant as long as parallel elements have the same left justification.

  2. Save the YAML file. Put the YAML file in the rackspace-monitoring-agent.conf.d subdirectory under the config directory:

    On a UNIX style platform this directory is "/etc/rackspace-monitoring-agent.conf.d".

    On Windows it is "C:\ProgramData\Rackspace Monitoring\config\rackspace-monitoring-agent.conf.d".

When you start the agent, it creates the checks and alarms

[Note]Note

Ensure that your agent has been set up via the agent Setup program, or has a valid monitoring_token in the /etc/rackspace-monitoring-agent.cfg file as described in the Manual Agent Configuration section of the Cloud Monitoring Developer’s guide.

 2.4.2.2. Update a server-side configuration file and its checks and alarms

If you change parameters within the configuration files, the agent updates the corresponding check and alarms once the newly saved file is read when you start, or re-start, the agent. For information on starting the agent, see Section 2.2.5, “Run the agent”.

 2.4.2.3. Delete a server-side configuration file and its checks and alarms

If a server-side YAML configuration file is removed from a server, the agent deletes the check and corresponding alarms configured in the file when the file is next read. The YAML files are read every time you start the agent. For information on starting the agent, see Section 2.2.5, “Run the agent”.

 2.5. Troubleshooting

Q: How do I know if my server-side configuration was successful on a particular server or on a group of servers?

A: On a single server you can look to the agent log file:

        Wed Apr 23 03:47:49 2014 INF: Confd -> config_file post overall success
Wed Apr 23 03:47:49 2014 INF: Confd -> config_file post operation result: success for file handle: mem.yaml at parsing
Wed Apr 23 03:47:49 2014 INF: Confd -> config_file post operation result: success for file handle: main_disk.yaml at parsing
Wed Apr 23 03:47:49 2014 INF: Confd -> config_file post operation result: success for check handle: {"check":"default","filename":"mem.yaml"} at unchanged
Wed Apr 23 03:47:49 2014 INF: Confd -> config_file post operation result: success for check handle: {"check":"default","filename":"main_disk.yaml"} at unchanged
Wed Apr 23 03:47:49 2014 INF: Confd -> config_file post operation result: success for alarm handle: {"alarm":"alarm1","filename":"mem.yaml"} at unchanged
Wed Apr 23 03:47:49 2014 INF: Confd -> config_file post operation result: success for alarm handle: {"alarm":"alarm1","filename":"main_disk.yaml"} at unchanged

A: For a group of servers, or even one server, you can use the list checks and/or list alarms API endpoints to verify that the agent created a check or alarm. Look for the confd_name field to have a string value and the confd_hash to be a valid SHA1 hash.

        {
            "id": "chlLIGmg4X",
            "label": "Check for Main Disk",
            "type": "agent.filesystem",
            "details": {
                "target": "/"
            },
            "monitoring_zones_poll": null,
            "timeout": 30,
            "period": 60,
            "target_alias": null,
            "target_hostname": null,
            "target_resolver": null,
            "disabled": false,
            "metadata": null,
            "confd_name": "{\"check\":\"default\",\"filename\":\"my_fs2.yaml\"}",
            "confd_hash": "cf4174eef962cc27f7f9a410a39d83e82049803a",
            "created_at": 1398217064602,
            "updated_at": 1398217064602
        }

Q: Can I deploy server-side configuration to an existing server or only to newly created servers?

A: Server-side configuration can be used on any server inside or outside of Rackspace, newly created or existing. The YAML configuration files are read every time you start the agent. The agent is commonly started at server boot or you can manually restart it.

Q: What happens if I use the API or UI to make a change to a server that was configured with server-side configuration? Which takes precedence, the API or the configuration file?

A: The change from API or UI will take effect immediately but the file is the source of authority. What this means is that if the API modifies a check or alarm that was created by server-side configuration, the object’s confd_hash will be invalidated with a zero (0) value. When you next start the agent, the object will be updated or recreated to match what is in the server-side configuration file. Note that if a check or alarm is created through the API or control panel, it cannot be modified through server-side monitoring configuration.



loading table of contents...