Google Maps’ terrain view now has contour lines

Amongst other things, I have long had a passion for maps. I remember what a nice discovery USGS topo maps were as a kid, and as I developed an interest in cycling, all the cool places I’ve found over the years by poring over maps. There’s also an element of discovery in getting detailed information about a place you’ve never been. So I’m excited to notice that Google Maps’ terrain view now has contour lines to show elevations, rather than just shading.

Here’s the mountain north of Innsbruck, for example.

Or Oregon’s North and Middle Sisters

Steens Mountain, Little Blitzen and Big Indian gorges

Cool!

“Password can only contain letters and numbers”

Sourceforge is giving me this junk:

Please note: passwords should contain only letters and numbers. Inclusion of other characters in your password may result in inability to access some SourceForge.net systems.

How annoying. Throwing a few symbols in there is a good way to make a password a little bit more secure.

Lots of banks have systems that are similarly crippled.

Mass customization of Android applications

As I mentioned in an earlier article, one of the goals for the Android port of Hecl is to make it as easy as it is to create Java ME applications:

  1. Write the script.
  2. Wrap it up and send it to the phone.

No “compiling” (well, none that’s obvious or explicitly asked of the user, at least, as we’ll see) or fiddling with build files or writing new Java classes or any of that.

We’re still not there yet 100%, but I did managed to hack together a system that, from an original Hecl.apk, creates a new application that can be installed on the emulator. This is tricky, because Android looks at the uniqueness of the package and class utilized for the application, meaning you can’t just rename Hecl.apk to Foo.apk and install it as a separate app. The emulator will overwrite your old copy of Hecl.

Here’s what we go through to make this work. Be forewarned – big, ugly hacks that are not for the faint of heart follow. The command line isn’t exactly beautiful either, but it beats all the individual steps that it replaces, as we’ll see.

java -jar /home/davidw/workshop/hecl/jars/AndroidBuilder.jar -android 
    /opt/android-sdk_m5-rc15_linux-x86/ -class Example -label "Hecl Example" 
    -package my.example -permissions ACCESS_LOCATION,ACCESS_GPS
  1. The first thing that needs to be done is to replace the AndroidManifest.xml that is contained within our “template” application, that resides in Hecl.apk, a copy of which is contained within the AndroidBuilder jar file. Since the compiled version of the xml file is a binary format that hasn’t been reverse engineered or documented yet, the best way to do this was to simply:
  2. Write out a new AndroidManifest.xml in our temporary work directory that uses our provided package name, label and class.
  3. Using the aapt command in the SDK, create a new, empty apk file, specifying the newly created xml file as the manifest file to use.
  4. Now we can extract the compiled version of the xml file, for later use.
  5. At this point, we extract a version of Hecl.jar contained within AndroidBuilder.jar to the temp directory.
  6. Now we need to generate some Java classes that correspond to the new package and class names that we’ve passed to AndroidBuilder. Ideally, I would have done this with some tools to write the byte code out directly, since the classes are so simple: all they do is subclass the Hecl and SubHecl classes so that methods fall through to those.
  7. But instead, I just write out the .java code and compile it.
  8. The two new classes get stashed inside the copy of Hecl.jar that we’re working with.
  9. We now use the dx command to compile the entire jar into the classes.dex file that Android uses.
  10. Since Hecl.apk is just a zip file, we can replace the AndroidManifest.xml file and classes.dex file with the new files we have created.
  11. At this point, we rename the whole works, and move it out of the temporary working directory to the current directory.

Phew! That’s a lot of ugly work. It’s obvious that a lot of this should and could be handled in a less “brute force” way, but for now it gets the job done. The only thing left to do is replace the Hecl script, which is very easy:

zip -r Example.apk res/raw/script.hcl

This makes it possible to whip out new Android Hecl scripts without carrying around a lot of project infrastructure – all you really need is AndroidBuilder.jar and your script.

Naturally, all of this is very much a work in progress, so it may not work as advertised, and, given the mess that we go through, bugs are likely as well. Do report them if you find any, and have fun! In the future, it will be necessary to make it so that it’s possible to add resources to the apk file with zip, and have them be available to Hecl applications, which was actually the focus of the previous article.

A new kind of spam?

Lately, I’ve seen multiple comments like the first one here:

https://journal.dedasys.com/articles/2008/03/09/bypassing-androids-resources#comments

It seems like spam, in that the comment is kind of off topic and irrelevant. However… it doesn’t link to obvious spam topics like viagra, porn, etc… and nor does the actual site that it links to. Or it doesn’t seem to at least.

Just someone trying to promote their site, or is there a trick to it?

Bypassing Android’s “Resources”

Android relies heavily on something called “resources” to access portions of the system that aren’t directly in the Java code. For instance, if you had an ogg file that you wanted to use to play a sound with, you would add it as res/raw/foo.ogg. Then, during compilation, the aapt tool transforms all your resources into a Java file called R.java, which is basically a series of integers that can be used to reference the actual resources. For instance, to set up our hypothetical ogg file, we would do:

MediaPlayer mp = MediaPlayer.create(context, R.raw.foo);

That’s all well and good as far as it goes, and if you don’t have any reason not to use it, you should.

However, with Hecl, one of my goals is to make it very easy to create applications. This means:

  • No complex directory layout structure.

  • No big long build files that require tools like ant.

  • No complex compilation steps.

I’m not there yet, on the Android platform, which is a bit more complex than Java ME (where it is indeed so easy to create new Hecl .jars that it’s possible to do it on the fly via this web site: http://builder.hecl.org), but I am making progress.

One thing that a user might want to do is add their own files to the .apk file that actually gets installed on the phone. Since it’s just a zip file, that’s pretty easy:

zip -r Hecl.apk res/raw/foo.ogg

However, without a resource, there is no way to access that file, so the question is: how can we get ahold of it somehow?

The first piece of the puzzle is Java’s ZipFile class, which lets us open up a zip compressed file, such as the .apk file that contains the ogg file we’re interested in in this case.

The critical passage is getting ahold of the actual filename on the Android filesystem of the code that’s currently running. It turns out to not be too difficult, if a bit obscure:

String filename = this.getPackageManager().getApplicationInfo(packagename, 0).sourceDir;

With that information in hand, we can feed it to ZipFile, and then access the contents at run time.

It’s not a perfect nor complete solution, yet. So many of the Android API’s require a resource ID, so it takes some extra coding to do things from the InputStream that we can get out of the zip file, but there may be ways to make this easier. For instance, we could simply dump the contents of raw/ into an application specific data directory the first time an application runs, in order to at least be able to access the files as files, rather than InputStreams connected to ZipEntries. In any case, at least it’s possible to access files added to the .apk without requiring the compilation step.

I’m not sure how many other use cases there might be for direct access to the .apk file, but if other people need to do this, hopefully this information will be helpful.

Cryptonomicon

I just finished re-reading one of my favorite books, Neil Stephenson’s Cryptonomicon. I don’t often reread books, but this one is really a classic, and I got even more out if it the second time around. If for some reason you haven’t already read it, and you are the type of person who likes sci-fi, startups, computers, crypto and the like, I’d highly recommend it. I’m a pretty quick reader, so having a book that lasts a week or two is also a pleasant change from briefer, quicker material. There are lots of subtle references and nuances to keep your mind entertained outside of the main story line, which is pretty good in and of itself, spanning two timelines, one during the second world war, and one in the late nineties (it’s been a while since I’ve had to type startx to launch a GUI on Linux!).

iPhone – nice, but not for me

Being fairly interested in this whole mobile phone software thing, it’s pretty hard not to sit up and take notice of the iPhone SDK that was just announced. A few thoughts:

  • Of course it’s nice, polished, and lets you do some interesting things with the phone. That was not too surprising though.

  • The “investment fund” seems like something of a response to the Android Developer Challenge, and is certainly an interesting move on Apple’s part. Strategically, it certainly alters the focus of the money flowing into the equation. Comparing the two, the dollar amounts in the ADC are much smaller, but they are also more “concrete”: N people will get X dollars in time frame T. As an independent developer, that works fairly well for me as something to have a go at and then move on from, win or lose. An investment fund backed by Kleiner Perkins is on a whole different scale and level. The amount of money is huge (they won’t even bother with less than 100K), but who knows how that works out in practice, or how much of that is actually 100% committed to iPhone companies. KP aren’t the sort of people that are going to just hand that out to any old company that does an iPhone app. They’re also going to want to be involved in the companies, which means there won’t be too many of them. In short, it brings in all the good and bad aspects of venture capital. In short: in one corner, lots of money for relatively big, organized players, and in the other, small, quick incentives to do cool things that anyone has a shot at. We won’t really be able to judge the results until we see how it plays out. (As an aside, I still have some bad memories of the management that KP foisted off on Linuxcare).

  • The system for selling applications is smart. You will have one place to go to to get your apps, and as a user you’ll have some assurance that they’re up to certain quality standards. People will like that kind of thing, and is far nicer than anything JavaME ever had. Instead of multiple, fractured marketplaces, you get one big one. Of course, it also keeps Apple in control of everything, which also has some negative implications as well.

  • 37 Signals seem to be wildly ecstatic about Apple’s potential, but I’m a bit less sanguine. I don’t doubt that the iPhone has a bright future, that much is pretty obvious. However… dominate? First of all, if it’s so widespread, it loses some of its cachet, doesn’t it? I don’t think Apple wants that. I don’t see them licensing the system so that other phone manufacturers can use it, so it will only be adapted as widely as Apple sees fit. Cheaper Nokia models are very popular over here in Europe. Will iPhones at those price points (less than 100 euro) be available and competitive with Nokia’s offerings? Open and adaptable aren’t necessarily things that Apple is known for, which might be the difference between “strong presence” and “dominate”, as we saw with Mac vs Windows. Getting 10-15% towards the high end of the market might be what they aspire to, rather than ubiquity.

  • I could, however, very easily envision Apple getting into some kind of business niche and dominating that completely, as they did with education for a while. That might not extend to the broader market, though.

  • Moving on to other considerations, this one is important to me: apparently, according to the license agreement, you can’t run an interpreter that fetches code and executes it, amongst the various other restrictions! That’s kind of a deal breaker for Hecl, which counts that ability as one of its strengths. I don’t think a port was in the works in any case since Hecl is written in Java. Still, though, that’s the kind of annoying restriction that reaffirms my commitment to open source.

  • As an aside, I wonder what Sun is up to at this point – between this and Android, they have to be under enormous pressure to release a successor to Java ME, which was a good effort for its time, but is beginning to look dated and limited, even though it is very widely deployed.

I’m an open source guy at heart. I love openness and source code and the freedom to tinker… and while I’ll certainly give Apple their due for making some fine products, closed and controlled is simply not for me. Others may feel differently – it’s a matter of taste and preferences.

So for the moment, my efforts will be focused on Android, which I see as the best, most open thing out there at the moment.

Apache Commons CLI and the Paradox of Choice

I’m looking around for a way to handle a bunch of command line options, and so naturally I was drawn to the Apache Commons CLI, where I found this line:

There are currently no plans to continue the 1.x line beyond bugfixes. The 2.x design is generally preferred and is in use, however there is no current activity to make a 2.0 release. To this end, the 1.1 release is recommended to most users while the 2.x line is recommended for anyone interested in helping to get this better API released.

So: one is not being worked on because it’s old, and the other isn’t being worked on… for some other reason? This is, in part, what The Paradox of Choice is about. I’m sure either one would meet my needs, but the doubts cast on the whole thing by the lack of a clear and easy choice make you a bit frustrated because you have to choose between two things, neither one of which is clearly the “right” choice. This is clearly a bit irrational given that you’re getting, in any case, a bunch of good code that very likely does its job, but there you go… that’s why it’s called “the paradox of choice”.

Android – commercial meets open source

One of the interesting things about Google’s Android project is the mix of cultures.

There are a number of people like me, firmly grounded in the open source world, who are happy to get something that 1) works, 2) looks to have legs in terms of popularity, and 3) will be open source. Think of a lot of your favorite open source projects when they first launched. “Polished” isn’t a word you would probably use to describe them, although it usually happens in later releases (Ubuntu looks better and better, for example).

There are also a lot of people who come from a ‘commercial’ background, who expect to see everything ready and waiting on a plate for them, which isn’t a mistaken expectation if you’re used to using products from big companies who are staking their reputations on them. They want nice, thorough docs for everything, almost all bugs worked out, and lots of easy to use tools.

Android lies in a middle ground somewhere between these two camps, which I think at times makes people uneasy.

It’s not open source yet. You can’t hack it as much as you may want. You can’t rebuild it from the ground up. You don’t get access to the communication between the developers working on it, and cannot participate in that development. However, there is a promise to open source it that would reflect very, very poorly on Google were it not met and is therefore pretty credible. The developers and many people involved with it certainly seem to “get” open source, and are taking good steps to create a community around Android (a few even hang out in the #android IRC channel on Freenode) that will most likely not take too long to start assimilating interested outsiders once the code is open. In terms of quality, it wasn’t bad when it came out, but it was very definitely an “early look”, put out there to solicit feedback and change according to it, something that’s part and parcel of a release early, release often open source model.

On the other side, Android looks to be a big effort from a big company to get into a big market in a big way – it’s not a “under the radar” type of strategy that the open source world is often constrained to follow for lack of monetary and marketing resources to do it any other way. This has attracted a number of developers who either don’t see or don’t really get some of the “open source” aspects of what we have so far: it’s still changing! The docs aren’t complete! The emulator is still ugly in places! And from their point of view, it must seem odd to getting something that’s obviously not a finished product, without crystal clear direction and goals and a roadmap, and without actual hardware only available in the distant (well, in internet terms! – 8 months or so) future. That kind of thing probably isn’t done very often in their world, or where it’s done, is certainly kept quieter and perhaps restricted to registered developers. They came over to have a look at something that was ready to go, and it isn’t, yet. This has caused some disappointment for people who expected to see something ready to go out of the box.

Perhaps Google needed to manage their initial communications slightly better in order to manage the second group’s expectations more carefully. In terms of us open source folks, I think that what we want is more access, as soon as Google is willing to hand it out. More source code access, but also access to the development process and people. “Real” open source isn’t just about the code, it’s about being more or less equal participants in a community. Luckily, I think the people at Google get this, which is one of the reasons I’m enthusiastic about Android, even if I wish we didn’t have to wait so long for the source code.

Looking under the Android hood

Here’s one way of having a look under the android hood:

mkdir /tmp/android/
cp .../android.jar /tmp/android
cd /tmp/android
unzip android.jar
... lots of files unzipped ...
for c in `find  android/ -name "*.class" | sed -e 's/.class//'`; do nice javap "$c" ; done > androidclasses.txt

That gives you a nice dump of all the classes available. It’s a useful way to get an idea of some of the under the hood capabilities of Android. Perhaps someone wiser in the ways of Java hacking can suggest even better ways of accomplishing the same thing.

Of course, anything that’s not officially documented is liable to change!