Salty Infrastructure, Part One
I recently attended PyCon 2013 out in beautiful California, and as expected I absorbed quite a bit of knowledge from all of the great speakers there. Yeah, I know that just about the only thing that people remember from this recent PyCon involved crude jokes, public shaming, and a crazy Internet storm of hate. It’s unfortunate that all of that occurred, but that’s all I’m going to say on the matter. Not “choosing a side” or anything of the sort. Instead, I’d like to talk about something positive that I got out of the trip.
While I’m currently a backend Python developer for my current employer, in the past I’ve been a Linux system administrator. I’ve always leveraged that experience along my various jobs throughout my career and continue to find myself interested in infrastructure development, maintenance, and administration. Just something about that line of work that I always find incredibly appealing. This all lead to me attending a great talk by Kate Heddleston titled “Chef: Automating web application infrastructure”.
Obviously, as the name implies, this talk covered Chef. It’s hard to be in the development or system administration communities and not at least be aware of Chef and/or Puppet. Now, both of these are great tools that I completely recommend for anyone to check out. I’ve played a bit with them both, but Kate mentioned another tool during her talk that wasn’t quite “mature” in her eyes. That tool was SaltStack, and it’s written in Python. I was immediately interested to see what this group was up to given that it’s relatively new and written in Python. I’ve spent a couple of weeks off and on using it to bootstrap the servers for a new side project of mine and would like to share my initial thoughts. I hope to make a couple of posts about it as I learn more and get more comfortable. For now I’ll warn that what I share here should not be seen as “Best Practices” or anything such as that.
As I started to sit down and sink my teeth into Salt, I had a pretty distinct goal in mind. I’m working on a new side project with a friend of mine, and I wanted to quickly be able to spin up new servers when needed. Frankly, scaling isn’t an issue at all for us yet, but half the fun of side projects is learning something new. I also wanted to run this on EC2. I made a quick list of requirements that my server(s) were going to need:
- misc packages (git, zip/unzip, imaging libs)
Naturally, I started with the Official Salt Walkthrough. This is a pretty great method of getting yourself rolling. After going through it, you should have yourself with a “salt master” and a “salt minion”. Their names are pretty self explanatory. The “master” is the center of control, managing keys, issuing commands, and so on. Meanwhile, minion(s) are the nodes that you’ll be configuring as web servers, database servers and so on. Of course, the “master” server can also have a “minion” running on it. In my current situation, I just have 1 server running, and thus it’s a master and a minion.
Getting through the installation and initial walkthrough of Salt is pretty straightforward. I’m not going to cover that aspect right now, and instead focus on the seemingly less talked about Salt Cloud. My only real complaint so far with this tool is the documentation, but it seems to be getting some love as we speak. What they have up now in terms of documentation is better than what I recall when I first started tinkering with the tool. Still, you’ll notice that the docs I linked you to don’t have an “installation” guide. This can be handled in a couple of different ways.
The simplest way is through your operating system’s package manager. If you
followed the walkthrough earlier by wgetting a script and piping it through
then you’re already halfway there. For instance, on Ubuntu you should then be
able to do:
sudo apt-get install salt-cloud
If you decided to go through a less conventional mean, then there’s always
sudo pip install salt-cloud
I will say that you’re really best off if you install
salt-cloud on whichever
server you plan on being your “Salt Master”. Doing so will ensure that newly spun
up nodes will automagically have their keys accepted, and they’ll be instantly
a proper part of your Salt ecosystem. You certainly can spin up servers from
servers other than your master, but you’ll eventually have to pop into your master
node to accept the Salt keys.
After you have
salt-cloud installed, you’ll want to edit
your Salt Master. Now the changes you make to this file will depend on your
cloud provider of choice. They currently cover a variety of providers, but their
documentation really only discusses AWS, Rackspace and Parallels. I’ll be discussing
AWS strictly, as that’s who I use, but there aren’t many differences.
Assuming you have
/etc/salt/cloud open, the first thing you’ll want to do is
minion option in the config. You’ll want to tell it who the master
is, and then you’ll also want to set the
provider, like so:
provider: aws minion: master: your.saltmaster.com
Now we’ll want to find the AWS section in the config, which will look like so:
AWS.id: YOUR_AWS_ID AWS.key: 'YOUR_AWS_KEY' AWS.keyname: YOUR_AWS_KEYNAME AWS.securitygroup: YOUR_SECURITY_GROUP AWS.private_key: /PATH/TO/PRIVATE_KEY.pem # Specify whether to use public or private IP for deploy script # private_ips or public_ips AWS.ssh_interface: public_ips AWS.location: us-west-2 AWS.ssh_username: ubuntu
That’s pretty much all there is to the configuration. The rest of that configuration
file is mostly for configuring other cloud providers. Now we need to define a
cloud profile. Open up
/etc/salt/cloud.profiles in your editor of choice, and
add something like the following:
base_aws: provider: aws image: ami-YOURIMAGEID size: Micro Instance ssh-user: ubuntu
Now you should be able to run the following command:
salt-cloud -p base_aws NAME_TO_GIVE_INSTANCE
Afterwards, you should have a brand new EC2 instance running. If you ran that command from your “Salt Master” then the new node should already be configured as a minion that’s ready to receive commands.
You can find a Getting Started With AWS tutorial via Salt’s documentation, which can provide you with more fine tuning options than my bare bones tutorial provides here. I hope, in the coming weeks, to get more in depth with some of the things I’m using Salt for. For now, feel free to take a look at my Salt configurations in my GitHub Repo.