The Rails Machine Blog.
Views and actions.

Rails Machine & Savannah

Posted by Steven Scarborough on May 17, 2012 | 0 comments

It’s always flattering to be mentioned alongside great companies and it happened on Fast Company’s technology blog where they wrote a thoughtful article about the advantages of founding your next company in Savannah, GA, a city that has long been a magnet for the tourism industry and it’s nice to see some national press discussing why creating a startup here is a good idea.

We are proud to be involved with the startup community in Savannah and we also believe that it’s a great place to start the next big thing. And when you do decide to start the next great startup, we’ll be ready to help you get there smoothly.

http://www.fastcompany.com/1837424/why-to-start-a-company-in-savannah


Hi, I'm the New Guy

Posted by Kevin Lawver on May 11, 2012 | 0 comments

Hi, my name is Kevin Lawver and I’ve been at Rails Machine for almost a month now. I would have written a blog post sooner, but I’ve been busy learning things. I came to Rails Machine from a startup here in Savannah where I was CTO (and sysadmin and lead developer and project manager, etc) and have known the Rails Machine guys for a couple of years, so I thought I knew what I was getting myself into when I took this job. I didn’t.

I knew there would be the usual “new guy” transition: learning how things work, internal processes, etc. What I didn’t bank on was getting to know dozens of Rails apps intimately in a couple of weeks, or the massive amounts of git-fu involved in managing deployments of hundreds of apps and servers, or all the systems used to deploy, monitor, troubleshoot and fix hundreds of servers.

It’s exciting, to say the least. I’ve learned more about git in three weeks than I did in the three years before that. I’ve re-learned everything I forgot about profiling slow MySQL queries and then some (remember, kids, indexes are your friends). And all of that on top of learning all about Moonshine, the magic glue that keeps all those servers up to date with the latest versions of everything and make sure they can talk to everything they need to make things work.

The best part is, I love all of it. There’s a huge adrenaline rush in figuring out where problems are and then fixing them quickly and well.

Oh, and did I mention how much fun learning is? Because it’s awesome. Working at Rails Machine is a crash course in scaling apps and operational badassery, and I’ve just scratched the surface. You might want to think about working here too

Speaking of learning, I’m going to try to blog here regularly on things I learn, trends I see in the things our customers face and some of the lessons I’ve learned over the years on growing Rails apps from my laptop to a bunch of servers.


Growing the Team

Posted by Bradley Taylor on December 14, 2011 | 2 comments

When I started working on Rails Machine over six years ago, all of my thoughts were about the technology, solution, and who our customers are. During that time with my head down to get to launch, I never thought about what a team would look like and how it would change over time. Most aspects of running a business can be fairly well understood going into it. If you have a good solution or product, most tech companies are fortunate that making money is not the greatest challenge, but rather building and growing a team.

With this in mind, I’ve been a little tardy with making some joyful welcomes and sad farewells to our team. In the past few months, Jesse Newland and Lee Jones have turned in their tools and moved on. We’ve enjoyed our time with them and wish them well in their new adventures. We’ll miss their smiling faces around the office.

With great joy, I’d like to announce two amazing hires that have plugged right into the Machine over the past few months. First to join us was Travis Graham as cloud operations engineer. Travis comes to us with several years experience working in data centers and managed services. With his skills in hardware, operations, and Ruby, Travis is the Keeper of the Cloud.

Our next exciting hire is Ernie Miller as Lead Solutions Architect. Ernie comes to us with 15 years of development experience and most recently served as Senior Architect at Mission Data. He is deeply involved with the Ruby on Rails community as author of several ActiveRecord related gems and a contributor to Rails core. This background makes Ernie distinctly qualified to listen, evaluate, and propose solutions for our customers as their web applications go big.

Although these two fine gentlemen do the work of giants, we’re not done hiring! We’re looking for Rails developers interested in pursuing an exciting (no, seriously) career in our special brand of DevOps that we call RailsOps. Please see the this job description for more details.

2011 was a great year for us and I look forward to 2012 bringing more great success with new team members and continuing to help our customers succeed.


Moonshine: Now with Ruby 1.9

Posted by Will Farrington on September 12, 2011 | 2 comments

We’re proud to announce the official release of Ruby 1.9 support in Moonshine. This update has been several months coming as we thoroughly tested a number of different configurations to make sure the release was rock solid. We’d like to thank everyone on the Moonshine Mailing List for their added input as well.

This support is mainly geared towards new applications using Ruby 1.9.2, but we do support upgrading from Ruby 1.8.6 or Ruby 1.8.7. If you’re running stable on Ruby Enterprise Edition 1.8.7, we do recommend you continue to use REE unless you have any strict requirement to use Ruby 1.9.2 — in our experience, Rails still performs better under 1.8.7 (though this may change when 1.9.3 comes out).

That said, upgrading is a pretty simple experience.

New Deploy

Make sure your Ruby version in config/moonshine.yml is set to ‘src192’. That’s it!

Upgrading

Upgrading your existing Moonshine application to use Ruby 1.9.2 is simple enough, assuming it’s already working locally on your development machine on 1.9.2.

First, update your version of Moonshine:

# Rails 2
script/plugin install git://github.com/railsmachine/moonshine.git --force
# Rails 3
rails plugin install git://github.com/railsmachine/moonshine.git --force

Next, update your Ruby version in the Moonshine config:

# config/moonshine.yml
:ruby: src192

Now upgrade Ruby:

cap ruby:upgrade

Finally, re-deploy to make sure everything’s up-to-date:

cap deploy

We recommend anyone performing this upgrade do so on their staging server first just to make sure no surprise show-stoppers in production.

Going Forward

Now that we have base Ruby 1.9 support in Moonshine itself we’ll be making sure all the Moonshine plugins we provide are 1.9 compatible as well in the coming months.

If you encounter any issues with Moonshine and Ruby 1.9, or any of the Moonshine plugins we provide and Ruby 1.9, please file issues on GitHub.


Paging the Pager

Posted by Will Farrington on August 31, 2011 | 1 comments

Here at Rails Machine we actively monitor hundreds of production Rails applications so that we can tackle any problems promptly and keep our customers up and running. Sometimes that means a phone call at 4am from our automated systems, and for that we’re using PagerDuty. While PagerDuty has a lot of features that we love and use, there are a few things we want that haven’t been implemented quite yet. In the last couple months, we made a couple small tools to ease the pain.

Lee Jones started with a little Rails 3 application to generate iCal calendars from the PagerDuty schedules. This application, called Pager Today, creates iCal calendars for each on call rotation, for each user, and also “master” schedules for each rotation that contain all users. Here’s a screenshot of what it looks like in action:

Aside from the main page, there’s also a page to let you know if you have the pager today or not:

We’ve open sourced this app today and it’s dead simple to get deployed — you need only configure a few items and then use capistrano to deploy via Moonshine.

But that’s not all — Josh Nichols and I got a bit tired of having to do things like remembering to check a calendar to see when we’re on call.

That’s where the PagerDuty Shift Announcer comes in. The announcer is a script we throw in cron along with our Firetower configuration for our Campfire bot. This script will check to see if the person who is on call changed in the last 5 minutes. If a new person is on call, our Campfire bot — Mister Robot — will alert everyone in Campfire and send an SMS to that person.

Here he is in Campfire:

Here he is in the SMS message:

There are a couple other tools we’ve written for services we use, but those aren’t quite ready for us to let them out into the wild just yet. In the mean time, we’d love to see what you’ve come up with if you’re using PagerDuty or Campfire as well!


Hoedown Bound: the Redux!

Posted by Will Farrington on August 23, 2011 | 1 comments

It’s that time of year, y’all!

Josh Nichols and I (Will Farrington) will be representing Rails Machine at this year’s Ruby Hoedown in Nashville, TN — the Southeast’s premier regional Ruby conference. We’ll be flying in Thursday and after we’re setup at the hotel, we’re planning on grabbing some grub and drinks at Jackson’s around 11pm that night — we’d love to have you join us.

Friday afternoon, I will talk about Vagabond — a library I’ve been working on to do rspec-style integration tests for your infrastructure and some of the no-nonsense attitude we’ve taken towards testing and understanding complicated infrastructures and how it can help you be sure your application and the configuration management that powers it will always “just work”. Saturday morning, Josh gives a talk on Operations with Rails and he’ll be touching on a number of topics, including operations anti-patterns, scaling, and offer some insight into how Operations has to work at a large scale to be successful.

If you’re a customer, are interested in hosting at Rails Machine, or just want to ask us some questions, come find us (or hit us @railsmachine) and we’ll be happy to grab a beer and chat. And remember, our “ask us anything” support is always available in person. :)

We hope to see you at the Hoedown and that you enjoy our presentations!


Fresh from the stills: moonshine_couchdb

Posted by Josh Nichols on August 09, 2011 | 0 comments

Thanks to our friends at PeepCode, I’m happy to announce moonshine_couchdb, the moonshine way of managing couchdb.

Combine it with PeepCode’s excellent CouchDB with Rails screencast, you will be relaxing with couch in no time.

Usage

moonshine_couchdb is pretty straightforward as far as things go. Install it as a Rails plugin:

# Rails 2.x.x
script/plugin install git://github.com/railsmachine/moonshine_couchdb.git
# Rails 3.x.x
script/rails plugin install git://github.com/railsmachine/moonshine_couchdb.git

Update your application_manifest.rb to use the couchdb recipe:

# app/manifests/application_manifest.rb
class ApplicationManifest < Moonshine::Manifest:Rails
  # other configuration ommitted for brevity
  recipe :couchdb
end

Commit and push it:

git add vendor/plugins/moonshine_couchdb
git add app/manifests/application_manifest.rb
git commit -m "Add moonshine_couchdb"
git push origin master

Now deploy. Make sure to use staging if you have it!

cap staging deploy

Other things of note

Usage aside, there’s some interesting things that went on the development side. First, this plugin was generated using the recent additions to the moonshine_plugin generator. Specifically:

  • plugin code is namespaced under Moonshine
  • more thorough README
    • uses markdown
    • including default links to homepage, source, issues, wiki, mailing list, etc
    • better usage notes and explanation of options

In addition, it’s been documented using rocco for some so called literate programming documentation. wfarr and I started experimenting with this for moonshine_pdfkit, and this is more of the same. I like to think that moonshine is excutable documentation for how your infrastructure is managed, with rocco being a perfect addition to make it easy to generate pretty, human readable documentation from it. See it in action.


Developing and testing system automation with capistrano-cowboy

Posted by Josh Nichols on December 16, 2010 | 0 comments

I’m not sure if you’ve heard, but we love us some system automation.

I think most of us can agree system automation (chef, puppet, moonshine, etc) is a good thing. As much as we love it, things can get tedious when it comes to actually developing and testing the automation. The process usually degrades into something like:

git commit -m "Update configuration to install widgets"
cap staging deploy # => argh, fail
git commit -m "Update configuration install widgets for real."
cap staging deploy # => argh, fail!
git commit -m "Blarg, really install widgets for real."
cap staging deploy # => argh, fail!!!
git commit -m "widget install thing go please go"
cap staging deploy # => argh, FAIL!!!

Not only are you about to pull your hair out, but you also have just littered your commit history with noise, making it harder to see what changes went in when you’re done. Basically, we want to reduce the feedback cycle between making changes, seeing them work, and being able to commit and move on to the next task. Here, the version control just gets in the way.

Capistrano, as it happens, does supports deploying from the current working directory, avoiding having to commit locally and checkout remotely. You can manage this with a few lines to your Capfile or config/deploy.rb:

set :repository, "."
set :deploy_via, :copy
set :scm, :none

Now the workflow looks more like:

cap staging deploy # => argh, fail
cap staging deploy # => argh, fail!
cap staging deploy # => argh, fail!!!
cap staging deploy # => argh, FAIL!!!
cap staging deploy # => oh... SUCCESS!
git commit -m "Update configuration to install widgets"

This is much better, particularly the absence of committing in anger. But now, you’re always deploying from the current directory. It should be a special case, not the default behavior. The next thing we can do is this into a task, so you can opt in:

task :cowboy do
  set :repository, "."
  set :deploy_via, :copy
  set :scm, :none
end

Here’s the new workflow:

cap staging cowboy deploy # => argh, fail
cap staging cowboy deploy # => argh, fail!
cap staging cowboy deploy # => argh, fail!!!
cap staging cowboy deploy # => argh, FAIL!!!
cap staging cowboy deploy # => wait... SUCCESS!
git commit -m "Update configuration to install widgets"
git push
cap staging deploy # =>; still success, ship it!

We found ourselves doing this on multiple projects, and really, we just want it everywhere. Capistrano will load a ~/.caprc if it exists, and so our little cowboy ended there. He’s grown up a bit since then, and he eventually made his way to a gist

Personally, I draw a line at sharing gists. It was time to become a gem. Behold! Install the usual way:

gem install capistrano-cowboy

Next, require it in your ~/.caprc:

require 'capistrano-cowboy'

And now you are ready to deploy from the hip in any capistrano-enabled project on your system:

cap staging cowboy deploy

Before you get any crazy ideas, please be extremely careful, or you might just shoot yourself in the foot. I’ll leave you with imaginary scenario that definitely has never happened:

git checkout crazy-awesome-but-experimental-feature
cap production cowboyw deploy # => what... nyoooo!
cap seppuku # => http://www.realultimatepower.net/ninja/seppuku.htm

Oh mans, RubyConf 2010

Posted by Josh Nichols on November 10, 2010 | 0 comments

It’s that time of year again for RubyConf. Jesse, Lee, and I are on the long road from Savannah to New Orleans as I type this very blog post.

Oh so far...

We love meeting and reconnecting with customers and other members of the Ruby community, so drop me an email or tweet if you’d like to catch up (especially over beers)!


Totally at RailsConf 2010

Posted by Josh Nichols on June 08, 2010 | 0 comments

Hi everybody!

RailsConf is under way this week. We love Rails as much as you do, so we’re totally there.

Our own Jesse Newland will be giving a talk, the coolness of which rock your world. Continous (Production) Integration: Ruby on Rails Application Monitoring with Cucumber on Wednesday at 4:25PM . His words sooth wild mares and pacify wars. Do not miss it.

I’ll also be around and hacking Moonshine throughout RailsConf. If you’d like to pitch in, or have any moonshine-related questions in general, definitely find me by (look for the jar of moonshine) and say hi! In particular, I’m planning to work on:

  • Rails 3 support for latest beta
  • Full Ubuntu 10.04 (Lucid) support
  • Really full stack integration testing, including deploying and verifying the system is up and running

We love meeting and remeeting our customers and other friends of Rails Machine. If you’re going to be around, we’re organizing a dinner, so definitely drop us a line (bradley at railsmachine dot com).