Python: Automate Everyday Tasks!

Excel is one of the most popular tools used worldwide. Irrespective of the company size, be it corporate bigwigs or local grocery shops, or even maintaining a personal vacation budget, excel is a common tool used by all. Since every basic tabulation of data is preferred in an excel format, it’s only logical to keep it that way. However, automating recurring Excel tasks can be challenging, but we need not give up such a useful and widely accepted format altogether. One would argue the use of Power Query to automate a few steps or maybe VBA as an alternate, however, these methods are not very portable.

Read More

Scantron - A distributed nmap scanning framework

scantron robot

Background

The Threat and Vulnerability Analysis team at Rackspace is charged with providing internal vulnerability scanning, penetration testing, and red/purple teaming capabilities to reduce cyber-based threats, risk, and exposure for the company. One of our tasks, as part of meeting certain compliance objectives, is to ensure systems are not exposed from various networking “perspectives” without going through a bastion first.

Read More

InsightVM (Nexpose) Slackbot

A few months back, I decided to find a way to lighten the load of ad-hoc vulnerability scanning requests by our system owners. Our most frequent requests used to go something like this: “Can you scan this”…“Ok I fixed it, scan again.”

Given the prevalence of Slack and associated bots, I thought it would be a good idea to try to write my own for scanning. Enter the InsightVM Slack Bot!

Read More

A Tutorial on Application Development Using Vagrant with the PyCharm IDE

I have spent the majority of my career as a Java developer. As a result, I learned to be more productive using an IDE instead of an editor like Vi. Even though Vi is still my editor of choice when I’m in a Linux shell, I don’t believe it’s practical when managing large Java projects.

Read More

Data science with Pandas; Baltimore City salary analysis

Wes McKinney started working on Pandas in 2008. Since then, Pandas has become one of the most popular and useful software components for the data scientist. For good reason; using Python, Pandas and iPython/Jupyter notebooks makes it simple and quick to perform analysis on various datasets.

In this post, we perform some basic analysis on the City of Baltimore employee salary data from data.gov, but this technique can be used on a wide variety of data sets very easily.

Pandas and Jupyter notebooks make this work quick. It may be surprising to see where the money goes!

Read More

Deploying JupyterHub for Education

As a PhD student at UC Berkeley, my duties involve some amount of teaching; so, this semester (Spring 2015), as well as last spring, I have been a teaching assistant for a class taught by my advisor, Tom Griffiths. The class, called Computational Models of Cognition (COGSCI 131), aims to introduce students to computational models of human behavior. The problem sets are a mixture of simple programming assignments—usually requiring students to implement pieces of different models—and written answers, in which students report and interpret the results of their code.

In the past, the problem sets were written in MATLAB. This year, however, we decided to make the switch to Python. In particular, we decided that the IPython/Jupyter notebook would be an ideal format for the assignments. The notebook is a cross-platform, browser-based application that seamlessly interleaves code, text, and images. With the notebook, it is possible for us to write instructions in the notebook, include a coding exercise after the instructions, and then ask for their interpretation of the results immediately after that. For an example of what the notebook looks like, you can check out try.jupyter.org for a demo.

Read More

How did we serve more than 20,000 IPython notebooks for Nature readers?

The IPython/Jupyter notebook is a wonderful environment for computations, prose, plots, and interactive widgets that you can share with collaborators. People use the notebook all over the place across many varied languages. It gets used by data scientists, researchers, analysts, developers, and people in between.

Read More

Configure Keystone to Utilize Apache

Keystone and many current OpenStack API components run in an Eventlet based http server. Eventlet is designed to perform well in networked environments and handles everything in a single thread.

The developers responsible for the Keystone project have recently recommended using Apache (with the mod_wsgi module) as a front-end rather than the traditional “Keystone” Eventlet-based process.

By using Apache as the front-end for Keystone, one gains better performance due to Apache’s ability to do multithreading. One can also take advantage of the variety of http server modules currently available for Apache. One popular module, Shibboleth, provides the ability to use one set of credentials to authenticate against multiple OpenStack clouds (more info here).

Here is a straight forward guide on how to setup Keystone to utilize Apache in your existing OpenStack deployment.

Read More

The not-so-sorry state of SSL in Python

Introduction

TLS and SSL are two critical technologies which underly much of the secure communications that occur on the internet. Over the past few years, spurred by increasingly effective attacks and a desire for new functionality, SSL and TLS have seen many new features, as well as practical improvements.

Python is currently in a transitional period between Python 2 and Python 3. For the past few years, all new feature development has been happening on Python 3, including new features in Python’s ssl module. This means that Python 3 users have had acccess to these improvements to TLS, but Python 2 users (still the majority of Python users) have been falling behind.

Read More

Python magic and remote APIs

This is a guest post by Jesse Keating, a Racker working on DevOps for Rackspace’s Cloud Server products. You can read his blog at http://raxcloud.blogspot.com/ or follow @iamjkeating on Twitter.

I’m a pretty big fan of Python as a programming language. It allows me to program by discovery, that is poke and prod at things until the work. Not having to compile an entire program every time I change something is pretty fantastic, as is the ability to insert a debug statement and be able to break a program at that point, then run arbitrary python code within that context. Pretty indispensable to how I write software.

Another thing I like about Python, which some may not, is the ability to do magic things. Not quite so magic as xkcd would like us to believe, but fun stuff indeed.

Read More