I’ve been playing around some with the Google Maps API, via ym4r. I discovered, however, that its geocoding API doesn’t do anything with the “accuracy” information that Google gives you, which tells you if the address you fed it actually means much. The XML looks something like this:

<Placemark>
  <address>Italy</address>
  <AddressDetails Accuracy="1" xmlns="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0">

I opened up geocoding.rb, and had a look, figuring it had to be pretty simple, and indeed, with REXml’s XPath stuff, you can easily grab the elements you want. For example:

data_country = data['//CountryNameCode']

So… data['//AddressDetails'].attributes['Accuracy'] ought to give me what I want, right? Nope. The namespace in the sub element confuses everything. After some googling, I finally found the recipe:

data_accuracy = data['//*[local-name()="AddressDetails"]'].attributes['Accuracy']

Kind of a mess, but it works, I have my accuracy attribute, and I submitted a patch to the ym4r guy.

I found this to be very interesting:

http://www.techcrunch.com/2007/03/09/demo-day-y-combinators-spring-chicks/

Specifically:

Tsumobi

Mobile applications have so far been nightmares to implement. It’s often hard to gain adoption due to complicated installs and near impossible to get users to upgrade their version once the product has shipped. Tsumobi hopes to solve this problem by creating their own language. The new language will sit on top of J2ME and process applications downloaded (via URL) for Tsumobi enabled sites. This means that developers will be able to change Tsumobi applications on the fly and have Tsumobi enabled phones automatically get the updates just by visiting a link.

That sounds very familiar… Hecl, maybe?

On one hand, I’m sort of jealous that someone is getting paid to do something I did for free (and hang out with Paul Graham, to boot). On the other, it certainly validates the idea doesn’t it?

It’s official! If all goes well, Ilenia and I should be settling in to an appartment in Innsbruck, Austria by April 2nd. She has been hired to work with Andreas Villunger’s group, which is a great opportunity for her, and also for me to work on some things that are not yet public… We’ve got a lot of good friends here in Padova, and not seeing them very regularly will be the hardest part of leaving. However, Innsbruck is only about 4 hours away, and Ilenia managed to find us a place on short notice that has a guest bedroom, so we hope to have lots of visitors!

I just like open source too much to not be constantly bumping into things that I would love to improve or fix… I have way too much fun getting my hands dirty with something new.

Today’s adventure regards my need to stream large quantities of data from a ruby on rails installation, covered previously here:

https://journal.dedasys.com/articles/2006/06/08/streaming-programmatically-generated-content-from-rails

The current block is that ActiveRecord doesn’t deal with a “stream” of results, but rather, fetches everything into one big, fat array and then deals with that. Luckily, I’m not the only person to have found this suboptimal:

http://dev.rubyonrails.org/ticket/6921

However, as I comment there, to truly make things work, some low level coding is required. For mysql, since it doesn’t seem to support cursors to the degree that something more standards-compliant like Postgresql does, the answer lies in the C library and language interfaces built on top of it:

http://dev.mysql.com/doc/refman/5.0/en/mysql-use-result.html

Unfortunately, Ruby’s C API, or at least the versions that I downloaded, don’t properly support fetching things one record at a time. It didn’t take more than a few minutes to whip this up:

http://dedasys.com/freesoftware/patches/mysql-ruby.patch

(although there’s a good possibility it could be improved…)

However, according to this posting, the upstream maintainer may not be terribly responsive, and the project isn’t really run as much of an open source project, with a public mailing list, bug tracker, version control, and so forth.

Perhaps it’s time to create that infrastructure?

We take a very simple class:

class Foobar {
var $myvar = 1;

function Foobar() {
}

function printit() {
    echo "var is " . $this->myvar . "n";
}

function incr() {
    $this->myvar += 1;
}

}

$f = new Foobar();
$a = $f;
$b = $f;

$a->printit();
$a->incr();

$b->printit();

And run it with PHP4 and PHP5:

davidw@byron:/tmp$ php4 foobar.php 
var is 1
var is 1
davidw@byron:/tmp$ php5 foobar.php 
var is 1
var is 2

Apparently it’s possible to use the clone function to do what we want with PHP5, but… that means we have to track down every single place in the code where there is something like $foo = $bar. And hope that we got them all, because otherwise subtle bad things might happen. The codebase I’ve inherited isn’t great to begin with, and hunting down every object copy is bound to be torture. Ouch.

Update

It’s been pointed out that my description of exactly what’s happening internally isn’t correct. However, from my point of view, the end result is the same: a massive search and replace operation in order to obtain the “old” behavior by using clone. Most likely, this means that we won’t be able to upgrade soon, which means that we won’t be able to take advantage of some of the nice new PHP5 features, which will slow down other works in progress. For want of a nail…

I’ve been investigating ways of testing out applications like Stuff to Do on Windows, and boy has it been frustrating. It’s not a fun task in the first place, as I’d rather be coding than trying to deal with crappy incompatibilities between browsers and operating systems.

I first looked into VMWare products, which look pretty good. The Workstation version has a free 30 day trial download, and the basic player is free of charge. Unfortunately the Ubuntu player packages appear to be seriously broken. Qemu also looks like a good possibility. I don’t care if it’s not blazing fast, just as long as reloading a page in IE doesn’t take 5 minutes. I think I could make either one of those options work.

However, the hairy, gorilla size problem is the microsoft operating system itself.

I don’t want to deal with Vista, because it’s a hog, and emulating a hog is bound to be slow. So I’d like to get XP, and since I’d prefer people to respect the licenses of software I write, I think it’s only fair to get a legitimate copy, even if my friends here in Italy will probably think I’m batty.

Overlooking the fact that the basic retail price is $200, which is somewhere north of double what I think is reasonable, it seems as if you can’t even download the damn thing and pay with a credit card. How antiquated is that? I want to be up and running now, not fiddle around with a CD that might arrive a week from now.

The end result: massive amounts of frustrating and nothing to show for it. That I would have avoided if I just found a cracked version. I think for now I’ll just put up some extra ‘get firefox’ banners on my sites:-/

I need to iterate over a big set of data, and I want to do it without sucking the whole thing into memory. Unfortunately, as far as I can tell ActiveRecord does just that, which is a pity because it could probably be made to do something like this

Foo.find_and_loop(:all, :conditions => '....') do |res|
  do_something(res.foo, res.bar)
end

without too much trouble. You’d have to have some fancy interaction between AR and the connection adapters so as to run your loop at a low level, but it should be possible. All the databases that I looked at let you operate that way in their C API’s.

Doubtless, this is something “opinionated”, and of use to a small percentage of rails users, so it’s probably not worth submitting a bug report, but it’s a minor annoyance all the same.

In a previous article, I illustrated one way of generating streaming content programatically.

The application utilizing it is still slow in sending it, though, because it’s got to interact with the database, process the data, and so on. The ideal solution would be to cache it, however, the normal rails cacheing solutions don’t work, because the return value is a Proc, and if you try and write that to disk, it just writes the equivalent of proc_object.inspect, which doesn’t do me much good.

However, I figured out a clever way of running the exact same controller method “standalone”, and stashing the results to disk. I placed this file in scripts/cacher:

#!/usr/bin/env ruby

ENV['RAILS_ENV'] = ARGV.first || ENV['RAILS_ENV'] || 'development'

require File.dirname(__FILE__) + '/../config/boot'
require "#{RAILS_ROOT}/config/environment"
require 'console_app'

app.get '/mycontroller/big_results_method'

resp = app.controller.response
File.open("#{RAILS_ROOT}/public/cached_big_results.csv", 'w') do |out|
  resp.body.call resp, out
end

I think it could be improved, because it seems to be running the DB query twice – once during the actual app.get, and once again when the Proc is called. But, in any case, it works, and now I can call it from a cron job.

Yep, time to stock up on some new books. I’m interested in:

• Business (startups and the like)

• Economics

• Marketing

• Technology. Classics, though, books about the latest this or that are less appealing these days, because the net simply has so much information available.

That’s certainly a broad range, but the reason I’m writing here, is that I’m looking for books other people like me considered a good read, and not just an idea I can glean from the wikipedia summary of the book.

Yoav seems to share some of my tastes, for instance.

Some of the books I’m considering:

• Founders at Work, by one of Paul Graham’s collaborators. Looks interesting, but I’m a little bit dubious about the ‘lots of anecdotes’ format. Do you learn anything repeatable from it, or just motivational tales of yore?

• Something from Guy Kawasaki – Rules for Revolutionaries?

• Over 20 Years of High-Tech Marketing Disasters

• Don’t Make Me Think: A Common Sense Approach to Web Usability

What I’ve already read, and would recommend to others:

http://www.dedasys.com/freesoftware/ (under ‘Recommended reading’)

In a previous article about Stuff To Do, my ajaxified priority queue program, I talked about how best to keep track of the user’s state of activity, should they so desire.

I opted for the popup, as that was the best compromise between simplicity, and ease of use for the end user. From an end-user point of view, it’s similar to how gtalk can detach from gmail if you want to run it in its own window.

The trick was to implement the popup/popout in Rails, which actually ended up being pretty easy, and, all things considered, is a pretty cool technique. It’s neat to pop stuff in and out of a web page.

The easy part is popping out the window:

<%= link_to_remote("Pop out " + image_tag('popout', :border => 0),
                   {:url => { :controller => 'tasks', :action => 'popout' } },
                   :title => 'Pop the status window into a popup window') -%>

Which subsequently calls popup.rjs:

page.replace_html('status', "<i><small>status in popup window</small></i></br>" +
                  link_to_remote("<small>Restore</small>",
                                 {:url => { :controller => 'tasks', :action => 'popin' }},
                                 :title => 'Restore the status window'))
page << 'statuspopup();'

Which is pretty simple – the first call replaces the html in the status div, and the second calls the javascript function statuspopup. This lets us replace what was in the status window with some placeholder text and a ‘restore’ link in case things get screwy and the user wants to simply restore the old content of the div in question. It’s worth noting that we also store the popped out status in session[:popped_out] in order to let the application know that the window is “popped out”.

This doesn’t actually pop the window itself up. The statuspopup function in javascript (located in application.js) takes care of that:

function statuspopup() {
  window.newwindow = window.open('/tasks/openstatus','status',
                                 'height=10,width=210,status=no');
  return false;
}

And then the openstatus action opens the same rails partial that the regular page does:

<%= render(:partial => '/tasks/status', ...

Meaning that you don’t have to rewrite the same code in two places.

So, that’s all it takes to pop up a new window, and replace the contents of the old div in the html page giving the user the sense of having popped a piece of the page out into a popup.

So far so good. Now what happens when the user closes the popup? Luckily, we get a reference to the window that opened the popup called window.opener that we can use to comunicate with the main window. For instance, in our case we set up an event listener to fire when the window closes:

window.addEventListener("unload", statusClosed, false);

And this, in turn, takes care of popping the window back into the main page:

function statusClosed() {
new Ajax.Request('/tasks/popin', { method: 'get', asynchronous: false })

$('popoutlink').style.display = 'inline';
w = window.opener;
target = w.window.document.getElementById('status');
code = $('onidle').childNodes[1].innerHTML;
target.innerHTML = $('status').innerHTML;
target.style['backgroundColor'] = '#bbddbb';
target.style['backgroundColor'] = '#bbddbb';
w.eval(code);
}

Once again, pretty simple. What’s happening:

• Ajax.Request just takes care of letting the rails app know that we’ve popped in, which resents session[:popped_out].

• Then grab the window opener, and find the status element within that window.

• Instead of re-rendering everything with a rails partial, we simply transfer the HTML that was residing in our popup to the main window:

  target.innerHTML = $(‘status’).innerHTML;
  target.style[‘backgroundColor’] = ‘#bbddbb’;
  target.style[‘backgroundColor’] = ‘#bbddbb’;

We also fix up the color just a bit, to mark the user as active.

All in all, it’s pretty useful for time tracking. In Linux, it’s ver easy to set the new popup to follow you around the screen, making it so that it’s not a bother at all to wave the mouse at it to let the application know you’re online. All you do is set the ‘on top’ and ‘always on visible workspace’ options – at least in Gnome.