Information Technology Dark Side

Struggles of a Self-Taught Coder

Information Technology Dark Side header image 1

Finally! An excuse to override method missing!

December 2nd, 2013 · 1 Comment

I load a lot of data into TroopTrack from CSV files by mapping fields in the CSV file to active record. Most of the time I am importing data from my competitors using CSV files they provide (and control the format of). In the past I have tended to write pretty sucky code for doing this where I build a hash of values that I then use to update active record, like this:

CSV.parse(@file_to_import, { :col_sep => "t" }) do |row|
  household_hash_1 = {
          :name => row[25],
          :line1 => row[27],
          :line2 => row[28],
          :city => row[29],
          :state => row[30],
          :zip_code => row[31]
        }
end

This code seemed okay when I wrote it, but then I learned that at least one of my competitors would add a new field to the CSV file every couple of months, which meant I had to redo the mapping of the attributes. PFFT.

Last week I started a project loading data into ActiveRecord from Microsoft SQL Server. There are about a hundred different tables that I need to transform and load into TroopTrack. Needless to say I wasn’t feeling very thrilled about using the approach mentioned above. The mere thought of THOUSANDS of lines of code mapping a hash to a CSV file makes me want to vomit.

Method Missing to the Rescue
If you don’t know about method missing you’re probably not reading my blog, so I’m not going to explain. If you wandered here somehow with no ruby or rails experience and still care, just google “override ruby method missing”.

All the code below does is let me treat a row from a CSV file more like an active record entry, so that I can write mapping code that is more readable and avoid thousands of lines of that crap above.

In other words, given a row from a csv file with a header that includes “last_name”, I can write code like this:

row.get_last_name

That’s pretty handy.

class DataGenius
  def initialize(filename)
    @data_items = []
    rows = CSV.read([Rails.root, 'db', 'data', 'csv', filename].join('/'))
    rows.each_with_index do |row, i|
      if i == 1
        @header = Hash[*row.each_with_index.map{|val, idx| [val, idx]}.flatten]
      elsif i > 1
        @data_items << DataItem.new(row, self)
      end
    end
  end

  def header 
    @header
  end

  def data_items
    @data_items
  end

  class DataItem
    def initialize(row, data_genius)
      @row = row
      @data_genius = data_genius
    end

    def row
      @row
    end

    def method_missing(meth, *args, &block)
      if meth.to_s =~ /^get_(.+)$/
        p $1
        @row[@data_genius.header[$1]]
      else
        super # You *must* call super if you don't handle the
              # method, otherwise you'll mess up Ruby's method
              # lookup.
      end
    end
  end
end

→ 1 CommentTags: Uncategorized

Conventional Rails

June 27th, 2013 · No Comments

Otherwise known as…
This post could also be called “Stuff I’ve figured out about Rails but didn’t want to blog about because I suspect everyone else already knows it and will figure out that I’m a moron”.

A code picture is worth a thousand words
Here’s some code. I’ve got some routes in there to set the stage, followed by pairs of roughly equivalent code demonstrating some conventions I’ve learned about and have grown to appreciate. I’ve not seen these conventions documented anywhere, but that doesn’t mean they aren’t. It just means I learned them the hard way.

# Routes
resources :users do
  resources :addresses do
    resources :phone_numbers #NEVER
  end
end

# users
# users/6
# users/6/edit
# users/new
# users/6/addresses
# users/6/addresses/5/edit
# users/6/addresses/5
# users/6/addresses/5/phone_numbers/11
# etc...

resource :tricks
namespace :magic do
  resources :tricks do
    get :secret, on: :member
  end
end

# magic/tricks
# magic/tricks/new
# magic/tricks/6
# magic/tricks/6/edit
# etc...

######### Example 1 - in a view #########
= render 'users/user', user: @user
= render partial: 'users/user', locals: {user: @user}
# Partial in ./_user.html.haml

# is roughly equivalent to...
= render @user
# Partial in users/_user.html.haml

######### Example 2 #########
= render partial 'address', collection: @user.addresses, as: address
# ./_address.html.haml

= render @user.addresses
# addresses/_address.html.haml

######### Example 3 - controller redirects #########
redirect_to show_user_path(@user)
# OR how about this?
redirect_to @user

redirect_to users_path
# OR how about this?
redirect_to :users

######### Example 4 - It works in links too #########
# In a view
link_to :users

# Some equivalents...
link_to edit_user_path(@user)
# Is the same as
link_to [:edit, @user]

link_to edit_user_address_path(@user, @address)
# Is the same as
link_to [:edit, @user, @address]

# Namespaced index view
link_to magic_tricks_path
# Is the same as
link_to [:magic, :tricks]

# Regular edit action in a namespace 
link_to edit_magic_trick_path(@trick)
# Is the same as
link_to [:edit, :magic, @trick]

# Custom action in a namespace
link_to secret_magic_trick_path(@trick)
# Is the same as
link_to [:secret, :magic, @trick]

######### Example 5 - rendering partials with a namespace #########
# In a view
= render [:magic, @tricks]
# Partial in views/magic/tricks/_trick.html.haml

= render @tricks
# Partial in views/tricks/_trick.html.haml

= render [:user, current_user.tricks]
# Partial in views/user/tricks/_trick.html.haml

######### Example 6 - the challenge #########
# What does this redirect to?
class AddressesController < ApplicationController
  def destroy
    @user = User.find(params[:user_id])
    @address = @user.find(params[:id])
    @address.destroy
    redirect_to user_path(@user)
  end
end

# Whatever you said is probably wrong. It actually redirects to a the user destroy action. WAT!?! Or, at least I think that's what I observed... 

# I THINK THIS IS A RAILS BUG!!!!

# Which brings up a point... how do you create a path to delete stuff using this convention?
link_to [:magic, @trick], method: :delete, confirm: "Are you sure?"

README
These conventions are all about two things for me: readability and the principle of least surprise. I don’t think the readability comment needs to be explained, but maybe the principle of least surprise does. When you render partials in this way, there is no question about where the partial lives in the file structure, even when you are rendering an object in a view outside of its normal view folder.

I also like the way this forces you to think about all those partials you create as you build an app. Every time you find yourself in a situation where you need to render an object and you can’t just do ‘render @thingy’ in your view, you should take a minute and think. Are you really doing the right thing? Do you really need to render a thing six different ways? This isn’t just about clean, dry code – it’s also about good UI design. The choice to create a new partial should be a deliberate one.

But, hey, you can have your cake and eat it too. Sometimes. For example, on TroopTrack I have 5 major functional areas: plan, manage, communicate, achieve, and share. Three of these functional areas have different renderings of users, but I can still follow this convention using namespaces, like this:

# In a view

= render [:manage, @user]
# partial: app/views/manage/users/_user.html.haml

= render [:achieve, @user]
# partial: app/views/achieve/users/_user.html.haml

= render [:communicate, @user]
# partial: app/views/communicate/users/_user.html.haml

Happy, happy, happy
I really lourve this convention for rendering. It changes my approach to the user interface in ways that encourages goodness and drives consistency, but that’s not the only reason. When the time comes to start worrying about performance, it makes russian doll cacheing a lot easier.

Ugh. I can’t seem to wrap this post up, so I’m just gonna stop typing.

→ No CommentsTags: Uncategorized

Adding Events to Various Calendars (Google, Live, Yahoo)

March 6th, 2013 · 1 Comment

Everything’s a nail
In the past I’ve used a jquery plugin for generating links to add events to a calendar, but it recently broke when I upgraded jquery on TroopTrack. So I started banging around trying to get it to work for about an hour before I realized I was being stupid. The jquery library was 1) building a drop down menu and 2) generating links.

You don’t need to use jquery to do that. It’s pretty easy to make drop down menus (twitter bootstrap) and generate links (helper). In fact, since my event details were all being stored in active record, using javascript made it harder than it needs to be.

Anyway, in the event you ever find yourself in my shoes, here’s what I did:

Build the urls

  def google_calendar_url(event)
    # https://www.google.com/calendar/render?action=TEMPLATE&text=Joe's+40th+Birthday&details=Joe+turns+40+just+this+once&dates=20111212T190000/20111212T200000&location=Gillette+Stadium&sf=true&output=xml
    "http://www.google.com/calendar/event?action=TEMPLATE&trp=false" +
    "&text=" + event.title + 
    "&dates=" + event.activity_at.to_s(:ics) + 
    "/" + event.end_at.to_s(:ics) +
    "&location=" + event.location +
    "&details=" + event.description 
  end

  def live_calendar_url(event)
    "http://calendar.live.com/calendar/calendar.aspx?rru=addevent" +
    "&dtstart=" + event.activity_at.to_s(:ics) +
    "&dtend=" + event.end_at.to_s(:ics) +
    "&summary=" + event.title + 
    "&location=" + event.location
  end

  def yahoo_calendar_url(event)
    "http://calendar.yahoo.com/?v=60" + 
    "&TITLE=" + event.title + 
    "&ST=" + event.activity_at.to_s(:ics) +  
    "&in_loc=" + event.location +
    "&DESC=" + event.description +
    "&URL=" + plan_event_url(event) + 
    "&DUR=" + event.duration 
  end

With a little help from some friends
The time formats matter, so I created a time formatter in an intializer:

[Time].map do |klass|
  klass::DATE_FORMATS[:ics] = lambda { |date| date.strftime("%Y%m%dT%H%M%S")}
end

and Yahoo expects a duration in the HHMM format, which I did by adding a method on my event class:

def duration
  "%02d" % ((end_at - activity_at)/3600.floor) + "%02d" % (((end_at - activity_at)/60).modulo(60))
end

Boom shaka laka
At this point adding an event to a calendar is as easy as pie. Here’s my Twitter Bootstrap worthy HAML

%ul.nav.pull-left
  %li.dropdown
    = link_to icon('icon-calendar'), '#', {'data-toggle' => "dropdown", :class => 'dropdown-toggle'}
    %ul.dropdown-menu{'role' => "menu"}
      %li= link_to 'Google', google_calendar_url(@event), :target => '_blank'
      %li= link_to 'Live', live_calendar_url(@event), :target => '_blank'
      %li= link_to 'Yahoo', yahoo_calendar_url(@event), :target => '_blank'
      %li= link_to 'iCal', plan_event_path(@event, :format => :ics)

The last link is to a downloadable .ics file, which is a topic for another day.

→ 1 CommentTags: Uncategorized

Useful Way to Automagically Add Controller/Action Selectors to an App

February 4th, 2013 · No Comments

Not Invented Here
I swiped this idea from the work Craig Ulliott did on bodyshopbids.com. Thanks Craig!

Don’t hard-code selectors for action and controller specific CSS and Javascript
Sometimes you need a bit of CSS or javascript that only applies to a specific action, or maybe to every action on a controller. So, you go in and add a class or an id to every view that controller supports.

Then you sigh.

You can do something like this instead:

  %body{:id => body_id, :class => body_class}

where body_id and body_class are helper methods. For TroopTrack, these methods look like this:

def body_id
    body_id = controller.class.name.sub(/Controller$/, '')
      .sub(/::/, '').underscore+'_page'
    body_id.sub!(/^devise_/, "devise_#{controller.scope_name}_") 
     if body_id.match(/^devise_/)
    
    return body_id
  end

  def body_class
    if @body_class.nil?
      @body_class = ""
    end
    
    @body_class << " #{controller.action_name} "
    
    @body_class << " #{controller.class.name.split("::").first.downcase}" 
      unless controller.class.name.index("::").nil?

    @body_class << " unit_type_#{@troop.unit_type_id}" if @troop

    return @body_class
  end

I think TroopTrack’s helpers are a bit overly complicated, but that’s because many of them exist in a namespace and I want a class for the namespace as well as the controller. I also need to adjust my CSS based on the type of unit they are, so that’s in there too. Here’s an example body tag from TroopTrack:

<body class=' index  share unit_type_3' id='share_share_page'>

Index is the action. Share is the name space. Unit_type_3 means it’s a cub scout pack. The id, share_share_page, means it’s on the share controller in the share namespace.

Sometimes when I write a blog post and share my code I see opportunities to dry things up or refactor them. This is one of those moments.

A more reasonable place to start would be something like this:

  def body_id
    controller.class.name.sub(/Controller$/, '').sub(/::/, '').underscore
  end

  def body_class
    controller.action_name
  end

As usual, I’m suspicious that there is a ruby or rails idiom out there that can make that code better. I’m all ears.

With this in place, it’s very easy to override CSS or write javascript that is specific to a given page. If you are following the Rails 3 convention for assets, you could just add this scope to the beginning of every controller-specific CSS and javascript file so that you are forcing the behaviors in those files to only apply to the controllers they relate to. I’m not a big fan of this per se, but I don’t condemn it as a practice either.

Don’t Overdo It!
It’s easy to let this get out of control and start putting every new bit of CSS you write inside a selector so that you don’t accidentally mess up something else. Please don’t fall into this trap. It only makes your CSS worse and will eventually lead to inconsistency in your site styling. Instead, if you can, take the time to make sure you are building a clean and consistent set of styles for your site and use the selectors only when you truly need to override something.

→ No CommentsTags: Uncategorized

Getting Started with the AWS Elastic Transcoder API in Rails

January 31st, 2013 · 5 Comments

Elastic What?
Elastic Transcoder is a brand new product from Amazon that lets you transcode videos into your favorite formats. It comes with lots of awesome preset formats, so you can use it through the AWS console to transcode a hi-def video to a web-friendly or iPhone-friendly resolution. You can even use it to create thumbnails and rotate the video. I’m going to show you how I use it to transcode a video from its URL in a Ruby on Rails app.

Elastic Transcoder only works on AWS S3, so you have to get the video into S3 before you can transcode it.

The Set Up
I have a video at a location on the web. In this case, it’s in Dropbox and I have a signed URL for it, but it really could be anywhere. I’ve stored the URL on a model and set up a resque worker to save the video to S3 and then kick off the transcoder. I am using CarrierWave for uploading files.

Code
You will need to add aws-sdk 1.8.1.1 to your Gemfile first. Prior versions don’t include the transcoder.

require 'open-uri'

class VideoProcessorWorker
  WEB_MP4_PRESET_ID = '1351620000000-100070'
  @queue = :eo_video_queue
  def self.perform(lecture_segment_id)
    lecture_segment = LectureSegment.find(
      lecture_segment_id)
    lecture_segment.remote_video_url = 
      lecture_segment.video_dropbox_path
    lecture_segment.save!

    lecture_segment.update_column 'web_video', 
      File.basename(lecture_segment.video.path)

    transcoder = AWS::ElasticTranscoder::Client.new
    transcoder.create_job(
      pipeline_id: APP_CONFIG[Rails.env][:pipeline_id],
      input: {
        key: lecture_segment.video.path,
        frame_rate: 'auto',
        resolution: 'auto',
        aspect_ratio: 'auto',
        interlaced: 'auto',
        container: 'auto'
      },
      output: {
        key: lecture_segment.web_video.path,
        preset_id: WEB_MP4_PRESET_ID,
        thumbnail_pattern: "",
        rotate: '0'
      }
    )
    
  end
end

Enough code, let’s talk
CarrierWave makes this problem a lot easier because it lets you save files from the interwebs to S3. All you have to do is put the url in the remote_whatever_url field and save it. Lourve the carrierwave!

You might be wondering why I do this:

lecture_segment.update_column 'web_video', 
  File.basename(lecture_segment.video.path)

CarrierWave won’t let you set the file name directly, and “web_video” represents the file that AWS Elastic Transcoder is going to create for me, so it’s not there yet, but AWS still wants to know it’s key. So, I use the update_columm method to update it in the DB without any callbacks, etc to trick CarrierWave into thinking the file exists. Once that is done, I can get its key (S3 talk for path) when I define the output portion of my transcoding job.

There may be better ways to do this, I’m all ears.

Incidentally, the preset id is from Amazon. They have a lot of awesome presets and you can create your own. The one I’m using is for web videos, which is really kind of funny cuz it means…

Boom. Web video from an URL for a web video.

→ 5 CommentsTags: Uncategorized

A Convention for Displaying Privileged Links

January 28th, 2013 · 4 Comments

I really hate view code like this:

- if can_manage_users
  = link_to 'Add a user', new_users_path

I want to get rid of it for a number of reasons, but the biggest reason is for cacheing. It really complicates my cacheing strategy because then I have to make keys that take into account the logged-in user or the privileges they possess, both of which will reduce the benefit of cacheing because I need a different cache for every user or privilege combo. Pfft.

Here’s what I’m replacing it with:

.privileged.can_manage_users = link_to 'Add a user', 
  new_users_path

Or maybe this…

= link_to 'Add a user', new_users_path, 
  'data-privileged' => 'can_manage_users'

To make it all work, I’ve also added the privileges to the body tag using a helper method. So, the body tag for a given user might look like this:

<body class="can_manage_privileges can_manage_users 
  can_manage_leadership can_manage_events">

Then, all I need is a little CSS to make the right links show up, like so:

.privileged {
  display: none;
}
.privileged.can_manage_users, 
.privileged.can_manage_leadership, 
.privileged.can_manage_events {
  display: block;
}

Obviously, you could do the same thing with javascript. You could argue that it makes more sense to do it with javascript because it’s a bit weird to use CSS like this and my response is… Shrug. You might be right. I’m just figuring stuff out here and am open to that approach to.

One more caveat. Obviously, this doesn’t secure your controller actions, so it’s not a replacement for authentication and privilege checking in your controllers. Sheesh, I can’t believe I felt the need to write that.

Please note this is only a half-baked idea, so feel free to suggest better ways but try not to judge me if you think this is stupid.

→ 4 CommentsTags: Uncategorized

Uploading Huge Files with Rails

January 27th, 2013 · 7 Comments

I’m working on an application that involves a lot of video content – great big files of 1 GB or more – and we found that uploading those huge files through a multipart form sucked. Duh. I started looking around at various solutions, such as chunked uploaders, and then a thought occurred to me: uploading huge files through the browser SUCKS. It locks up my browser, and to a certain degree, my laptop for the entire upload (if I start an upload, close my laptop and go somewhere, and then reopen it, the upload ain’t gonna work, so I’m stuck carrying my laptop around the house with the screen up like some crazed internet cats junky).

But you know what rocks for moving huge files around? Dropbox! They’ve already figured all this crap out – what to do if the computer gets turned off mid upload, loses it’s network, etc. And they have a super intuitive user interface – drag the file and wait for it to sync.

Also, when you’re uploading 1.5 GB of video, time isn’t exactly your main concern. You’re totally cool with dragging the file into the dropbox and going to sleep. If it shows up where you want it the next day, Yay!

So, here’s what I ended up building. I might redesign the workflow a bit once a real user sees it.

1) The user puts the huge file in a special dropbox folder that my app has access to
2) When they are ready to “upload” the huge file, they go to my app, pick the file, pick the model they want it to be associated with (in this case, it’s a LectureSegment), and save.
3) Resque kicks off a worker that copies the file to S3 using Carrierwave.

The code for copying the file is pretty simple, once you get the Dropbox session stuff figured out, which is weirdly convoluted IMO.

require 'dropbox_sdk'
class VideoProcessorWorker
  @queue = :eo_video_queue
  def self.perform(lecture_segment_id)
    log = Logger.new 'log/resque.log'
    @lecture_segment = LectureSegment.find(lecture_segment_id)
    dropbox_session = DropboxSession.deserialize(Organization.first.dropbox_session)
    client = DropboxClient.new(dropbox_session, :app_folder)
    log.debug("Got the client for #{@lecture_segment.video_dropbox_path}")
    tmp_file_name = "#{Rails.root}/tmp#{@lecture_segment.video_dropbox_path}"
    tmp_file = File.open(tmp_file_name, 'wb')
    tmp_file.write(client.get_file(@lecture_segment.video_dropbox_path))
    @lecture_segment.video = tmp_file
    @lecture_segment.save!
    log.debug("File saved to: #{@lecture_segment.video.url}")
    File.delete(tmp_file_name)
    
    @lecture_segment.update_attribute :video_dropbox_path, nil
    @lecture_segment.update_attribute :upload_status, "Upload Complete"

    log.debug("Clean up complete")
  end
end

I’m not sure that’s the best way to move the file to S3, but it’s easy to just let CarrierWave worry about it. It’d be cool if I didn’t have to create a local copy of the file. I’d be very interested in opinions on how to make this better. Also, I only just finished early testing of it – I haven’t busted it out on the 1.5 GB file, so it may still not even work!

UPDATE
I switched to the Dropbox Chooser, as suggested, and the code for the worker gets event simpler:

require 'open-uri'
class VideoProcessorWorker
  @queue = :eo_video_queue
  def self.perform(lecture_segment_id)
    lecture_segment = LectureSegment.find(lecture_segment_id)
    lecture_segment.remote_video_url = lecture_segment.video_dropbox_path
    lecture_segment.save!
    
    lecture_segment.update_attribute :video_dropbox_path, nil
    lecture_segment.update_attribute :upload_status, "Upload Complete"

    lecture_segment.update_column 'web_video', File.basename(lecture_segment.video.path)
  end
end

→ 7 CommentsTags: Uncategorized

When is ugly code okay?

January 24th, 2013 · 5 Comments

Yesterday I retweeted @donnfelker, who said

Ugly code that ships and solves a problem is a 100 times better than beautiful code that doesn’t ship

One of my friends later told me that if he didn’t already know me well he would have added my name to his list of people to never hire. Although I’m not really worried about being on a do-not-hire list I think it’s helpful to clarify why I feel this way and what I think the limitations are.

First of all, it’s just a fact
Code that works, regardless of it’s quality, is better than no code. Beautiful code that doesn’t ship is NOT code that works. So, in reality, ugly code that works is actually INFINITELY better, in a divide-by-zero kind of way, than nothing (i.e. beautiful code that doesn’t ship).

Beautiful code that ships and solves a problem is way better than ugly code that does the same thing
Who wouldn’t rather have great code? Who wouldn’t choose a car that is safe, fast, efficient, and beautiful over one that is just safe and fast if they both cost the same?

The truth of the matter is that an experienced, disciplined programmer with a comprehensive understanding of the problem space can write beautiful code just as efficiently as a programmer who lacks experience, discipline, or understanding can write ugly code. Even experienced, disciplined programmers will make mistakes as they learn the ins and outs of a problem space. Just look at the evolution of attachments in Ruby on Rails – from attachment_fu to paperclip to carrierwave, you can see the evolution of understanding how to deal with files in a web application.

Making ugly code beautiful is how we become experienced, disciplined, understanding programmers
I re-entered programming after a long hiatus as an “enterprise architect” and project manager. With the help of google, indy.rb, and my friends, I taught myself ruby and rails. I’ve written a lot of bad code. Some of it is still out there, open to the world, on my github repository. You could look through some of the silly projects I’ve done and conclude that I suck. That’s totally fine with me – if you are not smart enough to put publicly shared code in an appropriate context I don’t want to work with you anyway.

All those projects represent learning experiences for me, and many of them are essentially abandoned. Does that mean I should go back and delete them all? Maybe, but so what if I don’t? What if I don’t want to delete daibatsu even though it’s really, really crappy code I wrote to experiment with a javascript plugin two years ago. It made me a better programmer, and it was also the basis for what I think was one of the funnest presentations I’ve ever given. So, even though Daibatsu is a bit of an embarrassment outside of the context in which I wrote it (a funny sample app for a presentation), I’m still proud of it. So there.

Does all code have to be beautiful the day it’s released?
I don’t think so, but I also don’t see code as a one-and-done thing. Code is not like building a desk to me, where at some point it is exactly the way you want it and you are done. It’s more like an extrapolated portion of my brain that has been encoded – I don’t see it as “done” any more than I see my own learning as “done”. Code that is beautiful to me today will be ugly to me four months from now when I have learned something else.

For example, a few years ago I got on a “skinny controller” kick and cleaned up a whole lot of ugly code in TroopTrack. I was pretty happy with the result – nice simple controllers and big fat models. Over time, I started to experience the problems this causes for you, and I started looking for ways to make it better. I experimented with concerns and service objects, and now when I touch code I consider whether it should be re-factored into one of those two things, then I include that refactoring in the work I do.

There are a number of situations where spending as little effort as possible is the right thing to do. Here are some that I have seen in the past:

  • You are performing an experiment to understand how something works
  • You are performing a short-term market experiment on a feature and the results of that experiment are likely to result in the feature being killed
  • You have a meaningful deadline and you are going to miss it if you spend more time on the code
  • Your production environment is down and you are losing money every minute until it comes back
  • You don’t know any better and releasing it will expose you to an educational experience. I have written code that I thought was good, shown it to someone else and learned a ruby idiom that eliminated huge chunks of code.
  • You are building an OSS product and you want to gauge the potential for a community of contributors to develop
  • When it’s just a simple utility for automating something you find cumbersome on your own personal computer that you never intend to use in a production “Application”

In these cases, do the best you can, but understand that releasing the code into the wild is the true value of your efforts and that you should minimize the energy required to do that.

If you work at a place where continuously improving code quality is not allowed… You made your choices – please don’t blame the programmers.
Some of my friends have spent years working with code bases where, because of the ineptitude of the organization where they have chosen to work, continuously improving code quality is discouraged or prohibited and crappy code is actually encouraged in the name of deadlines that are contrived or meaningless. These are unhappy places. I’m sorry that you work there, and that you aren’t allowed to put forth your best effort. I sincerely hope you will find a way to make it better or leave. The crappy code you have to deal with is not the fault of people like me who live in a world where they can make the code better every time they touch it.

A little git fu can help you hide your “learning experiences”, and it’s good discipline
Want to work on something you aren’t sure about without exposing all the mistakes you made along the way to the world? Create a feature branch, work there until you are happy, and then don’t fast forward when you merge. Then, people will only see the final beautiful result and not the horrible creatures you spawned along the way.

I am not advocating you write crappy code
Write the best code you can under whatever circumstances you find yourself, and ship it. If people call you names, ask them to give you advice. If they don’t, write them off. If they do, you will become a better programmer and you might also make a friend.

→ 5 CommentsTags: Uncategorized

Two Helper Methods I Use Too Much

January 23rd, 2013 · 3 Comments

Carets and Icons
There’s nothing particularly brilliant about these, and they are probably only moderately useful. Mostly I’m putting them out there so that you can (preferably nicely) tell me how stupid I am if there is a better way.

Sometimes I will add options to control whether the caret goes after the string or before it.

  def caret(string)
    "<b class='caret'></b>#{string}".html_safe
  end

  def icon(icon_name)
    "<i class='#{icon_name}'></i>".html_safe
  end

  def to_paragraphs(string)
    return '' if string.nil?
    html = []
    string.split("n").each do |paragraph|
      html << "<p>#{paragraph}</p>"
    end
    html.join().html_safe
  end

After I originally published this I remembered another one… that’s why there are now 3

→ 3 CommentsTags: Uncategorized

Taking the Suck Out of Active Admin Performance

December 20th, 2012 · 3 Comments

Problem #1: A nagging memory problem
TroopTrack memory gets out of control every couple of days. It bumps up to 95% of available memory and stays there, triggering notifications from New Relic and slowing down traffic as TroopTrack uses swap. When this happens I will kick the web server over and life goes back to normal for a while.

Problem #2: A really slow dashboard
Yesterday I gave a TroopTrack demo to a small national scouting organization. The presentation included a demo of the backend tools I use to manage TroopTrack. I found myself apologizing for the 40 seconds it took to load the dashboard. In my embarrassment I silently chided myself for not figuring out what the problem was and fixing it.

Problem #3: A really slow help desk
I’ve recently started working on a set of features for TroopTrack I call Mister Smith. Mister Smith is my replacement for Zendesk. I have some ideas for how to make a helpdesk for SaaS that will better facilitate the sales and support process I want to have. These ideas don’t fit the models followed by Zendesk, Uservoice, etc. so I am rolling my own. I’ve already rolled out a “test” feature that lets users ask questions, but the speed of the active admin interface has been horrible, so much so that I’ve been considering dropping active admin for Mister Smith related features.

Trying to figure out problem #1 solves #2 & #3
On Tuesday I added a gem called Oink to my stack. It adds memory logging to my middleware and provides a tool for identifying which requests add the most memory to the heap. I let it sit in prod for a day, then ran it. Here are the results:

—- MEMORY THRESHOLD —-
THRESHOLD: 50 MB

– SUMMARY –
Worst Requests:
1. Dec 19 18:00:27, 655360 KB, admin/dashboard#index
2. Dec 19 05:12:47, 463092 KB, admin/questions#index
3. Dec 19 18:23:53, 394820 KB, admin/questions#index
4. Dec 19 20:33:34, 393216 KB, admin/questions#edit
5. Dec 20 04:35:28, 393216 KB, admin/dashboard#index
6. Dec 19 20:34:52, 327680 KB, admin/questions#edit
7. Dec 19 05:13:15, 323340 KB, admin/questions#edit
8. Dec 19 20:34:28, 262144 KB, admin/questions#index
9. Dec 19 04:04:51, 80156 KB, manage/users#show
10. Dec 19 12:00:58, 80156 KB, manage/manage#index

Worst Actions:
10, plan/plan#index
3, admin/questions#edit
3, admin/questions#index
2, manage/users#show
2, achieve/users#send_email_achievement_report
2, admin/dashboard#index
2, manage/manage#index
1, achieve/user_achievements#show
1, plan/events#new
1, share/comatose_pages#calendar

Worst requests #1-8 are problems 2 & 3. This got me thinking, and looking at what active admin was doing.

Fix #1: Not being excessive
My dashboard had a lot of stats that were counting instance that met certain requirements, like the number of troops that have an active paid subscription, etc. Loading the dashboard was running a ton of queries, a lot of which were for metrics that I thought might be useful when I added them but in practice were ignored. I cleaned those up and the page became acceptably quick.

Fix #2: Default filters for belongs_to relationships
The memory requirements for admin/questions/index confused me at first. This view uses pagination, and the table involved doesn’t have a lot of columns. You would expect it to be fast and not need much memory, but neither was true. So I watched the log and I saw a query that was something like this:

SELECT * from USERS

TroopTrack has 30,000 users and for some reason ActiveAdmin was loading every single one of them into memory. That’s when I noticed something I hadn’t even considered previously – the automatic filters on the right side of the view. Because a question has a belongs_to relationship with users, active admin was automatically adding a select filter for users by default. With 30,000 users, a drop-down list like that is not very useful anyway, so I just overrode the default filters with the ones I will actually use and boom, just like that the page loads fast and doesn’t hog the memory.

If I ever need a filter based on users then I will use some sort of autocomplete or name search instead of a dropdown, but for now just dropping the filter is enough.

Fix #3: Default forms for belongs_to relationships
If I hadn’t already figured out #2, then understanding the admin/questions/edit problem might have taken a while. I hadn’t overridden the form, so active admin was just creating default fields for every attribute on the model, including the user. I really don’t ever have any need to change the user associated with a question, so I just overrode the default form with one that doesn’t include the user field.

ActiveAdmin is awesome again!
I was starting to think that ActiveAdmin wasn’t going to scale up for me and that I would have to roll my own. I’m very glad I turned out to be stupid and wrong in this case. By simply digging in to figure out what was the real problem instead of merely chalking it up to “active admin is just slow”, I have been able to avoid what would have been a pointless and expensive development effort. Yay!

What about that memory problem?
After I realized that certain active admin pages were causing the memory problems I did a little test. After restarting the web server, I went to the trouble pages and watched the memory. Sure enough, it took off like a bat out of a hell. After I deployed the changes described above I repeated the test and memory held steady.

→ 3 CommentsTags: Uncategorized