There is a clear benefit to being able to aggregate logs from various servers and services into one place and be able to search them for any sort of arbitrary event. Traditional syslog can aggregate logs, but aggregating events from them sometimes involves grep and convoluted regular expressions. Logging structured data to a database makes a lot of sense. rsyslog and ElasticSearch can do that, but figuring out how to get it to work from the rsyslog documentation can be difficult. Let’s start from the beginning.
This is a guest post written by Michael DeHaan, CTO at AnsibleWorks. AnsibleWorks provides IT orchestration solutions that simplify the way IT manages systems, applications, and infrastructure.
When developers and systems administrators work to automate the rollout of application updates, a common problem is automating web and SaaS architectures that span more than a single machine and, more importantly, managing those systems in a way that preserves uptime. This is especially critical in high-traffic web sites and services.
When looking at the automation modelling itself, it is insufficient to model the actions that happen on one machine at a time, or even all classes of machines at a time, because simultaneous updates can introduce outages.
Agile. Scrum. Extreme programming. RAD. Lean. These terms all represent a departure from the traditional Waterfall development process in favor of a more rapid, iterative approach to application development. For companies with large-scale web applications, there are significant benefits to the agile methodology, but it also presents significant challenges.
When every minute of downtime represents significant lost revenue and increased support costs, it stands to reason that application support should be just as agile as application development. Among other things, this means tearing down l ong-standing walls between teams, and getting all business units working together toward common goals.
In the first article we configured salt-master and created a Cloud Server. In this article we will start building up the Marconi environment and while doing so shape what our salt configuration will look like.
We have two goals in mind. First, we have to be capable of creating several Marconi environments with little effort. As an example, we should have servers under dev, test and production environments managed under one configuration. Taking it a step further, we may have these in different locations. So having the ability to managing multiple environments is essential. Second, we will try to build generic configurations (SLS Formulas) that we can use for different projects. For example, we could have a generic firewall formula that will set proper iptables rules on Linux servers based on the role and environment they are in.
At Rackspace, we're working very hard to support the ever-growing platform of mobile. We're working hard to design a cloud-based mobile platform for developers and on the next generation of our mobile applications. Over the years we have developed a number mobile applications to interact with our services, but we recently made a conscious decision to improve them to more "fanatical" standards.
Possibly one of the hardest things about mobile testing is the infrastructure needed to support it. There are various vendors that provide some of this infrastructure, but there are few established best practices for how to build things in-house if you want to do more than run your tests on a local simulator. While we plan to rely on some of the work our friends at Sauce Labs have been cooking up, we also built a sizable chunk of testing infrastructure in-house.
Because of the lack of shared knowledge, we spent a good deal of time figuring things out the hard way. We now want to share with you some of the nasty details that we have overcome in this battle.
If you need to run MySQL on the Rackspace Cloud, you have two fundamental choices: run MySQL on a Cloud Server, or run MySQL as a Cloud Database instance. This naturally raises a few questions: What are the features and benefits of each? Which performs better? Which will be more cost effective? As with every application, the answer is ”it depends;” however, the information below should help you make the right choice based on your needs.
One of the cool things we do on the [Cloud Databases](http://www.rackspace.com/cloud/databases/) operations side of the house is come up with statistics that can help us gain insight to hardware performance to identify issues with systems. We use some really cool tools, but one of the most versatile tools we work with is [logstash](http://www.logstash.net/). The goal of this article is to get you started pushing metrics with logstash that you may already collect to [Graphite](http://graphite.wikidot.com/). Along the way, I'll be showing you how to get started with logstash, test your configuration locally and then start pushing your first metrics to Graphite with some different examples along the way.
As of July 16th 2015, this client has been updated to use JSON requests ONLY. All XML references have been removed at this time. Several new updates have been introduced:
Note that on July 20, 2015, Rackspace (in following with OpenStack developments) will disable XML support within the Cloud Servers API. All PowerClient users should upgrade to the new release. See the following for more information:
Adding Redis to your application stack is a fantastic way to gain speed with existing applications. Many of our customers aren’t running the latest and greatest new hotness NoSQL-using cloud thing. A lot of them port over a full stack of an existing applications that once only existed on bare metal servers, or use a hybrid environment with a big MySQL configuration on bare metal with web/app servers in the cloud.
In any case, we advise that customers use caching… EVERYWHERE. Adding Redis to your application stack can greatly improve site speeds when used as a cache.
Getting started with a distributed system like Hadoop can be a daunting task for developers.From installing and configuring Hadoop to learning the basics of MapReduce and other add-on tools, the learning curve is pretty high.