more custom salt grains

2015-03-09

At my day job, we’ve taken some large steps into the modern web application era. We have a great in-house development team, with fantastic leadership and direction.

One of the best parts about my role, is that I get to help move obstacles out of our developers way. Simply put: when they are busy, I am busy.

After a full year and a half, Don and his team have been very productive. They have consolidated the primary language to Ruby, and thus, they are using a lot or Ruby on Rails based applications.

The default ruby version in FreeBSD has changed rapidly over the past 2 years, from 1.9 to 2.1 as of today. In most cases, this is fine. Even better if we stay on top of our package build cluster and set the default options.

However, how do you manage a nice tool like Passenger with SaltStack, when on a 6+ month old server it could have Ruby 1.9 installed, or, a fresh FreeBSD 10.1 system that has Ruby 2.1?

One simple and understandable method is that you don’t. You lock all of your running servers and VM’s to one version, set it in your build server’s make.conf.

I have not taken that step, which can create a bit more work, so here is my newest workaround.

Since Passenger is a ruby gem, it is installed in /usr/local/lib/ruby/gems/, and depending on what version of Ruby you have installed, it will be in 1.9/gems/passenger or 2.1/gems/passenger.

This was causing a bit of a problem for my nginx servers running passenger, a major ruby upgrade would break the setup and I would have to manually correct it.

Salt’s grains seemed like a reasonable place to store this system information. Grains are mostly static, and should not change often

I did not want to rely on using tools like find, and I know just enough about FreeBSD’s pkgng system that I could use the pkg-query tool to get most of what I wanted.

After a few minutes of tinkering around, I decided that this little command would get me most of the way there: 

pkg query '%Fp' rubygem-passenger | egrep gems/passenger$
/usr/local/lib/ruby/gems/2.0/gems/passenger

I have never let terrible command-line get in my way, so I immediately threw this right into a new python script:

import re
import subprocess


def ruby_passenger():
    """
    Returns the location of the passenger gem file
    """

    grains = {}

    try:
        lines = subprocess.check_output(['pkg', 'query', '%Fp', 'rubygem-passenger']).splitlines()
    except:
        lines = ['none']

    for line in lines:
        line = line.rstrip()
        if re.search('gems/passenger$', line):
            grains['ruby_passenger'] = line

    return grains

With this new grain, I went ahead and sync’d all of my FreeBSD servers using the saltutil.sync_grains:

salt -G 'os:freebsd' saltutil.sync_grains
...
wos-app-3.discdrive.bayphoto.com:
    - grains.ruby_passenger
...

And accessing that grain looks like this:

salt '*-app-*' grains.item ruby_passenger
wos-app-4.discdrive.bayphoto.com:
    ----------
    ruby_passenger:
        /usr/local/lib/ruby/gems/1.9/gems/passenger
pricing-app-1.discdrive.bayphoto.com:
    ----------
    ruby_passenger:
        /usr/local/lib/ruby/gems/2.0/gems/passenger
pricing-app-2.discdrive.bayphoto.com:
    ----------
    ruby_passenger:
        /usr/local/lib/ruby/gems/2.0/gems/passenger
ordering-app-1.discdrive.bayphoto.com:
    ----------
    ruby_passenger:
        /usr/local/lib/ruby/gems/2.0/gems/passenger
wos-app-3.discdrive.bayphoto.com:
    ----------
ordering-app-3.discdrive.bayphoto.com:
    ----------
    ruby_passenger:
        /usr/local/lib/ruby/gems/2.0/gems/passenger
ordering-app-2.discdrive.bayphoto.com:
    ----------
    ruby_passenger:
        /usr/local/lib/ruby/gems/2.0/gems/passenger

With that, I can update my passenger salt state:

include:
  - nginx

passenger.conf:
   file:
      - managed
      - name: {{ pillar.etc_prefix }}/nginx/vhosts/passenger.conf
      - template: jinja
      - source: salt://nginx/passenger/passenger.conf.jinja
      - context:
         port: 8080
         passenger_root: {{"{{ grains.ruby_passenger" }}}}
         passenger_ruby: {{"{{ grains.ruby_version" }}}}
         passenger_max_pool_size: {{"{{ (((grains.mem_total * 0.75 ) - (grains.num_cpus * 100))|int / 100) |round|int" }}}}
         passenger_min_instances: {{"{{ (((grains.mem_total * 0.75 ) - (grains.num_cpus * 100))|int / 100) |round|int" }}}}
         passenger_pool_idle_time: 0
         rails_app_spawner_idle_time: 0
         rails_env: {{"{{ pillar.passenger.env" }}}}
      - require:
         - pkg: nginx
         - file: nginx-vhosts-dir