Automation of the week: fetchr

When a program tells me that there is a new version available, I usually get that new version and put it on a shared directory on our file server, so anyone else can take it from there, enjoy a quicker install and save some precious bandwidth. Recently I fired up VirtualBox on a box that I didn't use for a while, so it complained about being outdated.

What bugs me about these situations is that downloading a package like VirtualBox takes a few minutes and that's, at least in my opinion, a pretty annoying time span. It's not enough to reasonably do anything else, but enough to think about how to get rid of these breaks. This time I used these few minutes to come up with an idea, which resulted in the following piece of Ruby.

require 'open-uri'

TARGET = '/some/place/to/store/the/files'
packages = {
  :virtual_box => lambda do
    latest = open('http://download.virtualbox.org/virtualbox/LATEST.TXT').read.chomp
    index = open("http://download.virtualbox.org/virtualbox/#{latest}").read
    index.
      scan(/VirtualBox.+?OSX.dmg|virtualbox.+?Ubuntu_jaunty_i386.deb/).
      uniq.select {|filename| !File.exists? "#{TARGET}/#{filename}"}.
      map {|filename| "http://download.virtualbox.org/virtualbox/#{latest}/#{filename}"}
  end,
  :firefox => lambda do
    %w(mac win32).map do |platform|
      %w(2.0 3.0 3.5).map do |series|
        index = `lynx -dump ftp://ftp.mozilla.org/pub/firefox/releases/latest-#{series}/#{platform}/en-US/`
        index.
          scan(/Firefox .+?(?:exe|dmg)/).
          uniq.select {|filename| !File.exists? "#{TARGET}/#{filename}"}.
          map {|filename| "ftp://ftp.mozilla.org/pub/firefox/releases/latest-#{series}/#{platform}/en-US/#{filename}"}
      end
    end.flatten
  end
}

packages.each do |name, url_fetcher|
  puts "fetching urls for '#{name}'"
  url_fetcher.call.each do |url|
    `wget -P #{TARGET} '#{url}'`
  end
end

Writing the code to fetch VirtualBox was too easy to stop there, so I added a block for Firefox. mozilla.org didn't respond to open, so I lynxed instead. I put this script on our file server, pointed TARGET to the directory we usually put setup files in and created a cronjob to run it once every night. So whenever I need the latest version of VirtualBox or Firefox, there's a good chance it's already in local network range.

Sorry, comments are closed for this article.