Cloud Files directory listing

My last holiday took me to Morocco, where me and a couple of friends used motorbikes to travel through the country. To have something to show off back home, we used small cameras mounted on the bikes to film everything. At the end of the trip, our group had filmed about 10 gigabytes of footage and I was put in charge to gather and distribute these files amongst ourselves.

I uploaded all the footage - roughly 55 videos - to a Cloud Files container, b ut obviously I didn’t want to give my portal username and password to my friends. Also, there is no option to generate an index file within Cloud Files, so the only option left to me was to send everyone the 55 video URLs separately. Clearly, a different solution was needed. And so I built one myself.

Read More

Neutron networking: Neutron routers and the L3 agent

In this multi-part blog series I intend to dive into the various components of the OpenStack Neutron project and provide working examples of networking configurations for clouds built with Rackspace Private Cloud powered by OpenStack on Ubuntu 12.04 LTS.

In the previous installment, Neutron Networking: VLAN Provider Networks, I provided guidance on configuring networks in Neutron using VLAN tagging. In this fourth installment, I’ll describe how to combine flat or VLAN provider networks with GRE-based tenant networks using the L3 agent and Neutron routers.

Read More

PostgreSQL + WAL-E + Cloudfiles = Awesome

If you’re a big PostgreSQL fan like I am, you may have heard of a tool called WAL-E. Originally developed by Heroku, WAL-E is a tool for efficiently sending PostgreSQL’s WAL (Write Ahead Log) to the cloud. In addition to Heroku, WAL-E is now used by many companies with large PostgreSQL deployments, including Instagram.

Let’s unpack what that means. If you’ve ever set up replication with PostgreSQL you’re probably familiar with the WAL. Essentially there are two parts to replication and backup in PostgreSQL, the “base backup” and the WAL. Base backups are a copy of your database files that can be taken while the database is running. You might create base backups every night, for example. The WAL is where PostgreSQL writes each and every transaction, as they happen. When you run normal replication, the leader will send its log file to the followers as it writes it.

Instead of just using a simple socket to communicate, WAL-E sends these base backups and WAL files across the internet with the help of a cloud object store, like Cloudfiles (or any OpenStack Swift deployment). This gives you the advantage that, in addition to just being replication, you have a durable backup of your database for disaster recovery. Further, you have effectively infinite read scalability from the archives, you can keep adding more followers without putting more stress on the leader.

With the help of WAL-E’s primary author, Daniel Farina, we recently added support for OpenStack Swift to it. It’s not yet in a final release, but if you’re interested in checking it out, read on!

Read More

Chef server backups

There are few ways to backup a Chef server. Opscode has some documenation on their wiki Backing up Chef Server. Some of this is outdated now because chef no longer uses Couch DB.

However there is this little gem (pun intended) called knife-backup.

Read More

Neutron networking: VLAN provider networks

In this multi-part blog series I intend to dive into the various components of the OpenStack Neutron project, and to also provide working examples of networking configurations for clouds built with Rackspace Private Cloud powered by OpenStack on Ubuntu 12.04 LTS.

In the previous installment, Neutron Networking: Simple Flat Network, I demonstrated an easy method of providing connectivity to instances using an untagged flat network. In this third installment, I’ll describe how to build multiple provider networks using 802.1q vlan tagging.

Read More

Upload files directly to Rackspace Cloud Files from the browser

We needed to let users take files (possibly really big files) from their hard drive and push them straight to the Rackspace Cloud Files storage service http://www.rackspace.com/cloud/files/.

We don’t want the upload traffic anywhere near our servers.

This is available on github: https://github.com/216software/ajax-put-rackspace

Read More

Step-by-step walkthrough to using Chef to bootstrap Windows nodes on the Rackspace Cloud

If you are a frequent reader of this blog you will have seen Hart’s posts about “Cooking with Chef”:

Further to these there are also a lot of tutorials on the internet. Most of them seem to focus on using chef to deploy/manage Linux servers but you will have a hard time to find a lot for doing the same on Windows Servers (Yes, Windows as in Microsoft Windows).

I have therefore sat down and put together a detailed step-by-step walkthrough that will guide you through installing your own Open Source Chef Server on a Rackspace Cloud Server running CentOS 6.4, installing the knife-windows plugin and then spinning up, bootstrapping and installing IIS on a Windows Server 2012 Rackspace Cloud Server without logging on to it once. Read on, if you dare…

Read More

Welcome to Performance Cloud Servers; have some benchmarks!

Today, all of us at Rackspace are proud to roll out our new Performance Cloud Server offering. These are new flavors of Cloud Servers available to all customers as of today, providing higher performance from RAM to Disk I/O, and more. We’ve also rolled out a significant update to our cloud control panel to greatly improve the user experience of selecting flavors, operating systems and more.

Our Cloud Servers have been completely re-engineered from the ground up to deliver performance and reliability. They are built entirely with RAID 10 protected SSDs and powerful Intel Xeon processors. 40 gigabits per second of highly available network throughput is delivered to every host, enabling high bandwidth applications and blazing fast Cloud Block Storage performance. All hosts have dual power feeds with redundant power supplies and are deployed in Rackspace’s world-class data centers.

But you, as developers, probably want to know more. A lot more. I’ve spent several weeks taking many of the new flavors for a spin, putting them through a variety of benchmarks and stress tests. I’m a developer though and pretty much any time a vendor – even a well meaning one – publishes benchmarks, I want to see the raw data, how to run them and most of all, how I can recreate them, so that’s what I am doing - all raw results, plots, run scripts, etc are on GitHub.

Read More

Neutron Networking: Simple flat network

In this multi-part walkthrough series, I intend to dive into the various components of the OpenStack Neutron project, and to also provide working examples of multiple networking configurations for clouds built with Rackspace Private Cloud powered by OpenStack on Ubuntu 12.04 LTS. When possible, I’ll provide configuration file examples for those following along on an install from source.

In the previous installment, Neutron Networking: The Building Blocks of an OpenStack Cloud, I laid the foundation of the Neutron networking model that included terminology, concepts, and a brief description of services and capabilities. In this second installment, I’ll describe how to build a simple flat network consisting of a few servers and limited networking gear. Future installments will include VLAN-based provider and tenant networks, GRE-based tenant networks, Open vSwitch troubleshooting, and more.

Read More

Neutron networking: The building blocks of an OpenStack Cloud

In this multi-part walkthrough series, I intend to dive into the various components of the OpenStack Neutron project, and to also provide working examples of multiple networking configurations for clouds built with Rackspace Private Cloud powered by OpenStack on Ubuntu 12.04 LTS. When possible, I’ll provide configuration file examples for those following along on an install from source.

In this first installment, I’ll briefly highlight Neutron features and terminology that will be useful for later installments. Future installments will include VLAN-based provider/tenant networks, GRE-based tenant networks, Open vSwitch troubleshooting, and more.

Read More