David's Computer Stuff Journal

Since I recently added sensor callback support to Android Hecl, I thought I'd write about the process of doing so. It's really pretty easy.

Ideally, it would not be necessary to write and compile any Java code to create callbacks in Hecl - you could do it all inline. However, I don't think this is possible at the current time with Android's bytecode engine, although I could be mistaken. It would be necessary to generate a class, or hang some new methods on an existing class. So, for the time being, we do it the old fashioned way whenever anyone wants a new callback to utilize:

The heart of the matter is this class:

public class HeclCallback implements
          android.app.DatePickerDialog.OnDateSetListener,
          android.app.TimePickerDialog.OnTimeSetListener,
          android.hardware.SensorListener,
          android.widget.AdapterView.OnItemClickListener,
          android.widget.AdapterView.OnItemSelectedListener,
          android.widget.CompoundButton.OnCheckedChangeListener,
          android.widget.DatePicker.OnDateChangedListener,
          android.widget.TimePicker.OnTimeChangedListener,
          android.view.View.OnClickListener {

Which as you can see, implements the various Java listener classes. Each of these requires one or more methods to be implemented in order to receive the actual callbacks. Generally, they are done like so:

public void onItemSelected(AdapterView parent, View v, int position, long id) {
try {
    Vector vec = ListThing.get(script.deepcopy());
    vec.add(ObjectThing.create(parent));
    vec.add(ObjectThing.create(v));
    vec.add(IntThing.create(position));
    vec.add(LongThing.create(id));
    interp.eval(ListThing.create(vec));
} catch (HeclException he) {
    Hecl.logStacktrace(he);
    Log.v("hecl onitemselected callback", he.toString());
}
}

Which is pretty simple as well: the objects that are passed in are wrapped up as Hecl Things, and then the command is run.

The command to use is set from Hecl like so, when we create a new instance of the HeclCallback class:

set callback [callback -new [list [list SelectDemo]]]
$lview setonitemclicklistener $callback

So, the steps are:

1. Add an "implements" to HeclCallback.
2. Define the methods that need defining, copying one of the existing methods.
3. Recompile
4. Use the 'callback' command to instantiate a new HeclCallback class with the command of your choice as the handler for callbacks.

Thanks to Neal McBurnett, Hecl for Android finally got to run wild and free on a real phone. There were some glitches, but by and large, things seem to work, which is always exciting, and something of a relief. Neil says it runs fairly quickly, and looks good. Once again, a big thanks to Neil, and now on to fixing the bugs!

Android itself continues to evolve:

http://fredstechblog.blogspot.com/2008/12/big-updates-ahead-for-android.html

Now, if only I could get ahold of one of those dev phones! Although, truth be told, I think the "G2" or whatever it is, will probably be a significant improvement in terms of fixing up some of what needs improvement over the first generation, so I'll probably wait until sometime next year and get one of those.

As most people probably know by now, the Android source code is available, as promised:

http://source.android.com/

The next part is the interesting part, though, in my opinion. Source code is wonderful, and a huge gift from Google, but the real "secret sauce" for the most successful open source projects is the community. It will be interesting to see how Google manages this aspect of their project. It could be a situation where they create, and the public consumes, and it's fairly one way, with the odd patch here and there going back. Or it could be that they'll really let people get involved. Running a community for something this big, with this much code, and this much interest to the world at large is a daunting task, but Google certainly has the open source expertise to figure it out, so what we end up seeing will depend on exactly how Google wants to manage things.

All in all though, a big step for open source - congratulations to all the people at Google who made it happen!

Early details:

• Coming out in October.
• UK in early November, Europe by first quarter 2009.
• Simlocked to T-Mobile.
• Source code! When the phone comes out. Hopefully someone will use it to unlock the thing.
• 179$ - not bad.
• MP3's from Amazon.
• Market app built in.
• Neat use of Google Maps street view.

I'm sold, I want one. It might not be quite as flashy as the iPhone, but I want something open.

This was unexpected, but very welcome: a new Android SDK, with a bunch of updates, and a "roadmap".

http://android-developers.blogspot.com/2008/08/announcing-beta-release-of-android-sdk.html

It also looks as if we may see some phones right when they were supposed to be available:

http://www.alleyinsider.com/2008/8/gphone-on-its-way-fcc-approves-htc-dream

Of course, there is no source code, yet, but it's still good to see things kicked into gear and running again.

Well, to be charitable, it was a bit late and so perhaps what I said came out wrong, but in this article:

http://www.theregister.co.uk/2008/07/14/android_developer_unrest/

I'm quoted as saying the problem is with "upper management", which I don't really believe to be the case, going by snippets of conversations and things I've read. Frankly, at this point in time, I have no idea why Google can't talk about Android.

Also, I had several other positive things to say to the reporter, but I guess the 'focus' of the piece was about the problems that Android is currently experiencing. Notwithstanding the recent weirdness, I'm quite happy that someone with a lot of money and muscle is going to be backing an open source mobile phone system, and hope that the current troubles are merely a small speed bump.

Like many people, I participated in the Google Android contest this spring, and like most of them, I didn't win, but I did enjoy working with the platform, and am very excited about the idea of an open source mobile platform that's relatively powerful.

From the outset, due to the lack of actual source code for a lot of the system, there have been "doubters" about what Google is really up to. I'm not one of them, and am largely ok with their reasons for holding back: I buy their reasoning that the long term benefits of a strong, properly QA'ed launch probably outweigh the possibility of failure that might come from the press and the public at large that doesn't care much about software freedom seeing phones come out with half-baked, early implementations. I think most of us working with Android were ok with that strategy, and the ones who weren't self-selected out of the platform, most likely.

However, lately Google has made something of a mess of their communications regarding Android. The problem stems from the fact that they gave an updated version of the SDK only to the winners of the competition, which for a lot of people felt like a bit of a slap in the face: they helped popularize the platform by participating in the contest and writing applications for it, and in return got excluded from the upgrade path. Personally, it's not really any skin off my nose, as I'm sanguine about the Hecl port proceeding apace when the new SDK is public, but I can see that if you were, say, a company porting your app to the platform and now are left behind the competition winners, you might be a bit irritated.

The biggest mistake they've made though, is a big lack of communication regarding this business. As I said, I don't really have that much skin in the game, and if they explained in a convincing way why they needed to do things that way, they would go a long ways to allaying the frustrations felt by many.

I'm not much of a believer in conspiracy theories and plots, especially where companies are concerned. Inept bumbling is a far more likely explanation in most cases, including this one. What is odd, though, is that Google has a lot of people who "get it". I met Dan Morrill at the Munich Android event, and he definitely gets it. He writes about his frustrations as a developer advocate and all the crazy things people say. Well, one way to reduce the number of batty internet theories (eliminating them is clearly impossible due to the alien mind control rays) is to consistently communicate quality information about what's really going on, something that has been lacking with regards to the SDK question. Here's another Google employee who understands this need to communicate, and apparently does so despite being worried about being "slapped for talking publicy about all this":

http://groups.google.com/group/android-developers/msg/244edfad99870c63

But the root of the problem is certainly not licensing but that there hasn't been a new public SDK release since M5, while at the same time a small group of people received updated versions privately.

I really don't know precisely why this happened; but I'm sure it has more to do with logistics and reducing the burden of support while we shift priorities (to shipping real devices) rather than politics or any will of our part to "hurt the community" (come one guys, we are not that stupid... !)

While others in the team may disagree, I think it was very very unfortunate; some of us are trying to prepare a new SDK release, but it's a lot harder than I can comment on here, so don't hold your breath because it might not happen that soon.

This explains things in part, but still leaves us in the dark, and leaves me with the feeling that someone in power, somewhere at Google, despite his or her clued in colleagues, really doesn't get it with regards to open source.

I'm still unconcerned about the Android source code: I think we'll see it, but to be a true open source project, Android will need an open community as well, not one where decisions are taken exclusively at Google, and not even communicated to the development community at large. All in all, this mistake is more molehill than mountain, but however you look at it, in terms of open source, it is a step backwards.

I don't know if androids dream of electric sheep, but I've certainly been losing a lot of sleep over Google's Android, and am glad that the Android Developer Challenge is winding up, as I've been doing all Android, all the time for the past month or so.

I have greatly enjoyed working with Android, as it's much more capable than Java ME. It's still very much a work in progress, and only time will tell if Google and their partners are able to best current mobile programming systems... or if the web will simply obviate the whole thing, as it's doing in many other areas. Win or lose, I will continue to support Android as a platform that Hecl runs on, but in the meantime, I hope to go back and look at some less future-oriented code and get back to some Java ME work. More on that soon...

In any case, here's hoping that the contest judges see the value in bringing a scripting language to mobile phones! I also hope to see actual hardware soon, as I'm really looking forward to getting my own Android phone.

As I mentioned in an earlier article, one of the goals for the Android port of Hecl is to make it as easy as it is to create Java ME applications:

1. Write the script.
2. Wrap it up and send it to the phone.

No "compiling" (well, none that's obvious or explicitly asked of the user, at least, as we'll see) or fiddling with build files or writing new Java classes or any of that.

We're still not there yet 100%, but I did managed to hack together a system that, from an original Hecl.apk, creates a new application that can be installed on the emulator. This is tricky, because Android looks at the uniqueness of the package and class utilized for the application, meaning you can't just rename Hecl.apk to Foo.apk and install it as a separate app. The emulator will overwrite your old copy of Hecl.

Here's what we go through to make this work. Be forewarned - big, ugly hacks that are not for the faint of heart follow. The command line isn't exactly beautiful either, but it beats all the individual steps that it replaces, as we'll see.

java -jar /home/davidw/workshop/hecl/jars/AndroidBuilder.jar -android \
    /opt/android-sdk_m5-rc15_linux-x86/ -class Example -label "Hecl Example" \
    -package my.example -permissions ACCESS_LOCATION,ACCESS_GPS

1. The first thing that needs to be done is to replace the AndroidManifest.xml that is contained within our "template" application, that resides in Hecl.apk, a copy of which is contained within the AndroidBuilder jar file. Since the compiled version of the xml file is a binary format that hasn't been reverse engineered or documented yet, the best way to do this was to simply:
2. Write out a new AndroidManifest.xml in our temporary work directory that uses our provided package name, label and class.
3. Using the aapt command in the SDK, create a new, empty apk file, specifying the newly created xml file as the manifest file to use.
4. Now we can extract the compiled version of the xml file, for later use.
5. At this point, we extract a version of Hecl.jar contained within AndroidBuilder.jar to the temp directory.
6. Now we need to generate some Java classes that correspond to the new package and class names that we've passed to AndroidBuilder. Ideally, I would have done this with some tools to write the byte code out directly, since the classes are so simple: all they do is subclass the Hecl and SubHecl classes so that methods fall through to those.
7. But instead, I just write out the .java code and compile it.
8. The two new classes get stashed inside the copy of Hecl.jar that we're working with.
9. We now use the dx command to compile the entire jar into the classes.dex file that Android uses.
10. Since Hecl.apk is just a zip file, we can replace the AndroidManifest.xml file and classes.dex file with the new files we have created.
11. At this point, we rename the whole works, and move it out of the temporary working directory to the current directory.

Phew! That's a lot of ugly work. It's obvious that a lot of this should and could be handled in a less "brute force" way, but for now it gets the job done. The only thing left to do is replace the Hecl script, which is very easy:

zip -r Example.apk res/raw/script.hcl

This makes it possible to whip out new Android Hecl scripts without carrying around a lot of project infrastructure - all you really need is AndroidBuilder.jar and your script.

Naturally, all of this is very much a work in progress, so it may not work as advertised, and, given the mess that we go through, bugs are likely as well. Do report them if you find any, and have fun! In the future, it will be necessary to make it so that it's possible to add resources to the apk file with zip, and have them be available to Hecl applications, which was actually the focus of the previous article.

Android relies heavily on something called "resources" to access portions of the system that aren't directly in the Java code. For instance, if you had an ogg file that you wanted to use to play a sound with, you would add it as res/raw/foo.ogg. Then, during compilation, the aapt tool transforms all your resources into a Java file called R.java, which is basically a series of integers that can be used to reference the actual resources. For instance, to set up our hypothetical ogg file, we would do:

MediaPlayer mp = MediaPlayer.create(context, R.raw.foo);

That's all well and good as far as it goes, and if you don't have any reason not to use it, you should.

However, with Hecl, one of my goals is to make it very easy to create applications. This means:

• No complex directory layout structure.

• No big long build files that require tools like ant.

• No complex compilation steps.

I'm not there yet, on the Android platform, which is a bit more complex than Java ME (where it is indeed so easy to create new Hecl .jars that it's possible to do it on the fly via this web site: http://builder.hecl.org), but I am making progress.

One thing that a user might want to do is add their own files to the .apk file that actually gets installed on the phone. Since it's just a zip file, that's pretty easy:

zip -r Hecl.apk res/raw/foo.ogg

However, without a resource, there is no way to access that file, so the question is: how can we get ahold of it somehow?

The first piece of the puzzle is Java's ZipFile class, which lets us open up a zip compressed file, such as the .apk file that contains the ogg file we're interested in in this case.

The critical passage is getting ahold of the actual filename on the Android filesystem of the code that's currently running. It turns out to not be too difficult, if a bit obscure:

String filename = this.getPackageManager().getApplicationInfo(packagename, 0).sourceDir;

With that information in hand, we can feed it to ZipFile, and then access the contents at run time.

It's not a perfect nor complete solution, yet. So many of the Android API's require a resource ID, so it takes some extra coding to do things from the InputStream that we can get out of the zip file, but there may be ways to make this easier. For instance, we could simply dump the contents of raw/ into an application specific data directory the first time an application runs, in order to at least be able to access the files as files, rather than InputStreams connected to ZipEntries. In any case, at least it's possible to access files added to the .apk without requiring the compilation step.

I'm not sure how many other use cases there might be for direct access to the .apk file, but if other people need to do this, hopefully this information will be helpful.

Being fairly interested in this whole mobile phone software thing, it's pretty hard not to sit up and take notice of the iPhone SDK that was just announced. A few thoughts:

• Of course it's nice, polished, and lets you do some interesting things with the phone. That was not too surprising though.

• The "investment fund" seems like something of a response to the Android Developer Challenge, and is certainly an interesting move on Apple's part. Strategically, it certainly alters the focus of the money flowing into the equation. Comparing the two, the dollar amounts in the ADC are much smaller, but they are also more "concrete": N people will get X dollars in time frame T. As an independent developer, that works fairly well for me as something to have a go at and then move on from, win or lose. An investment fund backed by Kleiner Perkins is on a whole different scale and level. The amount of money is huge (they won't even bother with less than 100K), but who knows how that works out in practice, or how much of that is actually 100% committed to iPhone companies. KP aren't the sort of people that are going to just hand that out to any old company that does an iPhone app. They're also going to want to be involved in the companies, which means there won't be too many of them. In short, it brings in all the good and bad aspects of venture capital. In short: in one corner, lots of money for relatively big, organized players, and in the other, small, quick incentives to do cool things that anyone has a shot at. We won't really be able to judge the results until we see how it plays out. (As an aside, I still have some bad memories of the management that KP foisted off on Linuxcare).

• The system for selling applications is smart. You will have one place to go to to get your apps, and as a user you'll have some assurance that they're up to certain quality standards. People will like that kind of thing, and is far nicer than anything JavaME ever had. Instead of multiple, fractured marketplaces, you get one big one. Of course, it also keeps Apple in control of everything, which also has some negative implications as well.

• 37 Signals seem to be wildly ecstatic about Apple's potential, but I'm a bit less sanguine. I don't doubt that the iPhone has a bright future, that much is pretty obvious. However... dominate? First of all, if it's so widespread, it loses some of its cachet, doesn't it? I don't think Apple wants that. I don't see them licensing the system so that other phone manufacturers can use it, so it will only be adapted as widely as Apple sees fit. Cheaper Nokia models are very popular over here in Europe. Will iPhones at those price points (less than 100 euro) be available and competitive with Nokia's offerings? Open and adaptable aren't necessarily things that Apple is known for, which might be the difference between "strong presence" and "dominate", as we saw with Mac vs Windows. Getting 10-15% towards the high end of the market might be what they aspire to, rather than ubiquity.

• I could, however, very easily envision Apple getting into some kind of business niche and dominating that completely, as they did with education for a while. That might not extend to the broader market, though.

• Moving on to other considerations, this one is important to me: apparently, according to the license agreement, you can't run an interpreter that fetches code and executes it, amongst the various other restrictions! That's kind of a deal breaker for Hecl, which counts that ability as one of its strengths. I don't think a port was in the works in any case since Hecl is written in Java. Still, though, that's the kind of annoying restriction that reaffirms my commitment to open source.

• As an aside, I wonder what Sun is up to at this point - between this and Android, they have to be under enormous pressure to release a successor to Java ME, which was a good effort for its time, but is beginning to look dated and limited, even though it is very widely deployed.

I'm an open source guy at heart. I love openness and source code and the freedom to tinker... and while I'll certainly give Apple their due for making some fine products, closed and controlled is simply not for me. Others may feel differently - it's a matter of taste and preferences.

So for the moment, my efforts will be focused on Android, which I see as the best, most open thing out there at the moment.

One of the interesting things about Google's Android project is the mix of cultures.

There are a number of people like me, firmly grounded in the open source world, who are happy to get something that 1) works, 2) looks to have legs in terms of popularity, and 3) will be open source. Think of a lot of your favorite open source projects when they first launched. "Polished" isn't a word you would probably use to describe them, although it usually happens in later releases (Ubuntu looks better and better, for example).

There are also a lot of people who come from a 'commercial' background, who expect to see everything ready and waiting on a plate for them, which isn't a mistaken expectation if you're used to using products from big companies who are staking their reputations on them. They want nice, thorough docs for everything, almost all bugs worked out, and lots of easy to use tools.

Android lies in a middle ground somewhere between these two camps, which I think at times makes people uneasy.

It's not open source yet. You can't hack it as much as you may want. You can't rebuild it from the ground up. You don't get access to the communication between the developers working on it, and cannot participate in that development. However, there is a promise to open source it that would reflect very, very poorly on Google were it not met and is therefore pretty credible. The developers and many people involved with it certainly seem to "get" open source, and are taking good steps to create a community around Android (a few even hang out in the #android IRC channel on Freenode) that will most likely not take too long to start assimilating interested outsiders once the code is open. In terms of quality, it wasn't bad when it came out, but it was very definitely an "early look", put out there to solicit feedback and change according to it, something that's part and parcel of a release early, release often open source model.

On the other side, Android looks to be a big effort from a big company to get into a big market in a big way - it's not a "under the radar" type of strategy that the open source world is often constrained to follow for lack of monetary and marketing resources to do it any other way. This has attracted a number of developers who either don't see or don't really get some of the "open source" aspects of what we have so far: it's still changing! The docs aren't complete! The emulator is still ugly in places! And from their point of view, it must seem odd to getting something that's obviously not a finished product, without crystal clear direction and goals and a roadmap, and without actual hardware only available in the distant (well, in internet terms! - 8 months or so) future. That kind of thing probably isn't done very often in their world, or where it's done, is certainly kept quieter and perhaps restricted to registered developers. They came over to have a look at something that was ready to go, and it isn't, yet. This has caused some disappointment for people who expected to see something ready to go out of the box.

Perhaps Google needed to manage their initial communications slightly better in order to manage the second group's expectations more carefully. In terms of us open source folks, I think that what we want is more access, as soon as Google is willing to hand it out. More source code access, but also access to the development process and people. "Real" open source isn't just about the code, it's about being more or less equal participants in a community. Luckily, I think the people at Google get this, which is one of the reasons I'm enthusiastic about Android, even if I wish we didn't have to wait so long for the source code.

Here's one way of having a look under the android hood:

mkdir /tmp/android/
cp .../android.jar /tmp/android
cd /tmp/android
unzip android.jar
... lots of files unzipped ...
for c in `find  android/ -name "*.class" | sed -e 's/.class//'`; do nice javap "$c" ; done > androidclasses.txt

That gives you a nice dump of all the classes available. It's a useful way to get an idea of some of the under the hood capabilities of Android. Perhaps someone wiser in the ways of Java hacking can suggest even better ways of accomplishing the same thing.

Of course, anything that's not officially documented is liable to change!

I whipped this up as a quick Sunday project. The code is based on dims' code here.

Its translations aren't great, but would probably do in a pinch. They'll likely improve as Google improves its translation bots.

I also used it to to test out Google's code hosting, which is quite nice. The code is located here:

http://code.google.com/p/seathreepo/

The latest component of Android Hecl that I have been working on is a simple shell server that lets you interact with Hecl remotely, in order to be able to telnet in, run commands and see what they do. In order to give it a whirl, fetch Hecl from svn:

svn co http://hecl.svn.sourceforge.net/svnroot/hecl/trunk/hecl hecl

Then install the Hecl.apk:

adb install android/bin/Hecl.apk

Select the Hecl Server demo, at the bottom of the list:

You also need to make sure that port forwarding is set up:

adb forward tcp:7405 tcp:7405

At this point, you can telnet to port 7405 on localhost, and run Hecl commands:

> puts hi
hi
> androidlog "hello world"

> java android.telephony.TelephonyManager telman
telman
> set phoneinfo [[activity] getSystemService phone]
android.telephony.TelephonyManager@435c8f80
> $phoneinfo getLine1Number
15555218135

Since this simple server runs in its own thread/interpreter, you can't control the GUI from it. To do that, there is a 'maineval' function that sends code back to the main interpreter:

> maineval {set layout  [linearlayout -new [activity]]}
1
> maineval {$layout setorientation VERTICAL}
1
> maineval {set layoutparams [linearlayoutparams -new {FILL_PARENT WRAP_CONTENT}]}
1
> maineval {$layout addview [textview -new [activity] -text "Hi!" -layoutparams $layoutparams]}
1
> maineval {[activity] setcontentview $layout}
1

Gets us this very, very simple screen:

Hecl's Java reflection support is still fairly alphaish, but you can use it to query classes about their methods and constructors, and call them most of the time; errors should be sent to me or the mailing list.

I found this little server particularly useful when fiddling around with some of the undocumented android classes, as a way to experiment in a running system.

I'm a bit tired after driving up to Munich for the Android developer meeting, hosted by Google, and being "on" all day, but it was worth it. I was a bit apprehensive that it might be too 'introductory' in the sense of the Google guys telling you in person things that you could have easily looked up on the web, without spending a bunch of time driving around Austria and Bavaria. Fortunately, that wasn't the case. There was some of that, but also plenty of unstructured time to chat with Dan Morrill and Jason Chen, the two Google "Developer Advocates", who are both very friendly, informative and informed, approachable and put a lot of energy into what they do, despite being a bit tired from flying over from the US. It was nice to get a chance to talk with them, and considering that the platform is still changing, useful as well, because they were definitely listening to what people had to say to them. My thanks to them for a job well done.

Here are some "impressions" I took away - note that they're not what anyone said, just ideas I got about what they said:

• Source code is being released when handsets ship because they want to work very closely with the hardware partners to make sure the phone experience is good (they said this). They want to "set the bar high", and they probably run the risk of someone snatching the code and slapping together a phone if they let out the source code to early. (They definitely didn't say this, it's just me connecting the dots).

• Despite interest for 'other languages', it really looks like they want everything to go through Dalvik in some way or another. Clearly, at some point in the future, there will be specs for writing Dalvik byte code.

• They won't talk about the upgrade strategy, something I asked about. I want to know that as a developer, I won't be stuck with N combinations of things that I have to support over the years. I want to know how they'll minimize that pain, but they're not currently able to discuss it.

• Talking with other people attending, there seemed to be some doubts about security and permissions for user data, since Android is moving away from the j2me "everything is private" model, to a model where data can be shared between apps, which opens up lots of good things, but also some potentially bad ones as well.

There's still really a sense of Android continuing to evolve, in a true spirit of openness. I liked that, as it means there is space for people to tell Google what they want, rather than simply being given a finished product and told that that's that... but then again, I would, being the open source guy that I am.

I didn't get to show off Hecl quite as much as I would have liked to, but I did get to mention it a few times, so I guess I did ok.

Tomorrow (January 28th), I'll be in Munich for this:

http://www.mobile-monday.de/web/guest/android

Unfortunately, the event filled up super fast and I barely managed to squeeze in, so there are probably no spaces left. Still, if anyone would like to meet up afterwards, send me an email soonish. If you managed to get in, and would like to see a demo of Hecl, let me know, because I'd like to have the chance to show that off some too!

Hopefully it will be a valuable experience. Android is a pretty convincing system in many ways, but of course has yet to fully come into contact with reality, so I'm curious to meet some of the people involved and hear about it from them. I really hope it doesn't end up being a rehash of information that's available on the web, as it's a two hour drive from Innsbruck.

After four or so months, it's time for another Hecl "checkpoint". Those who are seriously interested really ought to be getting Hecl from subversion, however, it's a good idea to tie up the loose ends every now and then, and put something that's easy to fetch for the casual user.

https://sourceforge.net/project/showfiles.php?group_id=122383

This release contains a number of interesting things:

• Alpha support for Java integration. For example:

  java java.lang.System sys
  set ms [sys currentTimeMillis]
  

• Alpha support for Google Android. This utilizes the Java integration system above heavily. What sets it apart from just writing code in Java is that on top of the building blocks, I've started to write little bits of Hecl code to make common things very straightforward, layering on top of the Java API.

• Numerous fixes in the code, documentation, and build system.

Credit for finding out how to do this goes to dims, who has been doing lots of cool Android hacking lately:

http://davanum.wordpress.com/2007/12/18/android-task-manager-primitive-prototype/

If you're interested in Android and what it can do, you should be following his online journal!

As a way to keep pushing and testing Hecl, I decided to see if I could copy what he'd done in Hecl. It has indeed produced a few improvements to the Java code. That done, making it work was short and sweet:

set layoutparams [linearlayoutparams -new {FILL_PARENT WRAP_CONTENT}
set layout [linearlayout -new $context]
$layout setorientation VERTICAL

# Create ourselves some commands to access Android internals.
java android.app.ActivityManagerNative activitymanagernative
java android.app.IActivityManager iactivitymanager
java {android.app.IActivityManager$TaskInfo} taskinfo
java android.content.ComponentName componentname

java java.util.List javalist
set am [activitymanagernative getdefault]
set tasks [$am gettasks 10 0 [null]]
set len [$tasks size]
set tasklist [list]
for {set i 0} {< $i $len} {incr $i} {
    set baseactivity [[$tasks get $i] -field baseactivity]
    lappend $tasklist "Task: [$baseactivity getpackagename]"
}

$layout addview [textview -new $context -layoutparams $layoutparams \
         -text "Currently running tasks"]

set lview [basiclist $context $tasklist -layoutparams $layoutparams]
$layout addview $lview
$lview requestfocus

set callback [callback -new [list [list SelectTask]]]
$lview setonitemclicklistener $callback

heclcmd setcontentview $layout

proc SelectTask {parent view position id} {
    [activitymanagernative getdefault] movetasktofront $position
}

It could have been squeezed further by avoiding some of the temporary variables, and perhaps also by implementing a mapping between java.util.List and Hecl lists.

I think one of the next things I want to do is work on some infrastructure so that people will be able to actually work on scripts on their phone. That's sort of a departure from the past (most phone keyboards would be sheer torture for inputting long scripts), but it's a cool demo in any case.

Alpha Android Hecl

I'm a little bit nervous about the idea, but I decided that I might as well release early, because for the foreseeable future, there's always going to be "one more feature" I could add.

So, without further ado, I'm going to announce that I have a version of Hecl running on Android that has a access to a number of Android wigets. It is available here:

http://www.hecl.org/Hecl.apk

However: this is an alpha release, for those interested in hacking on it. If you are bothered by broken code, don't like to report bugs, don't want to think about how to improve it, complain loudly about missing features, then this code is not for you yet. Soon, I hope, it will be stable and useful for many more people, but for the moment, a lot of things are up in the air:

• Hecl's java reflection interaction code, which lets us interact with Android's API without writing wrappers for all of it, is very new. It was written specifically to facilitate the Android port. So it might change. For hackers, the opportunity is to be able to take part in shaping it.

• More work needs to go into the threading model. Wolfgang Kechel built a nice system of timers and events into Hecl, but work will be needed to make those play nicely with Android.

• More API coverage is needed. We have a number of the basic widgets, but more are needed.

• Storage is still something being worked on. I want to have a Hecl script Provider so that other applications can access Hecl scripts, as well as Hecl commands to access the various types of permanent storage that Android makes available.

• Phone features - we want to have callbacks and commands to deal with phone features like sms, phone calls, the camera, and so on.

• Needless to say, none of what currently exists is documented besides some comments in the code!

Of course... Hecl itself is a pretty cool language, I like to think, and the fact that it's interpreted means that you don't have to recompile anything, you just

1. Edit your script.
2. Replace the script in the zip file: zip -r bin/Hecl.apk res/raw/script.hcl
3. Load it up on the emulator: /opt/android_sdk_linux_m3-rc37a/tools/adb install bin/Hecl.apk

You can do other cool things too, like downloading bits of code over http and running them.

Hecl @ Android Developer Challenge

I hope that people interested in hacking on Hecl will do so because it's fun and interesting. Being an early adopter means you have a lot of influence in shaping the future of this programming language, and the API's to interact with Android (or Java ME if that's your thing).

However, seeing as how Google has put together this contest:

http://code.google.com/android/adc.html

I have decided to submit the Android port of Hecl, and split the prize money, should we win any, with those who have contributed to Hecl. That includes any and all contributions to Hecl, whether or not it's directly related to Android or not. So, with documentation, Java code, Hecl code, tests, and even improvements to the web page as valid things, there's room for everyone to get involved. Of course, in order not to paint myself into a corner, I reserve the right to make final decisions about percentages, but it should be roughly equivalent to lines of code added, deleted or modified (yes, deleted counts too - if you make the project that much slimmer without sacrificing functionality, that's wonderful!).

Also more than welcome are people who want to submit their own projects to the developer challenge based on Android. We make no claim to your winnings, and we'll be happy to give you a hand.

Interested?

Join the Hecl mailing list, that's where all our discussions take place:

http://sourceforge.net/mailarchive/forum.php?forum_name=hecl-devel

Hope to see you there!

This was informally confirmed by someone at Google:

Given this layout code:

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
  android:orientation="vertical"
  android:layout_width="fill_parent"
  android:layout_height="fill_parent"
  id="@+id/ll">
</LinearLayout>

And this Java code:

public void onCreate(Bundle icicle)
{
    super.onCreate(icicle);
    setContentView(R.layout.main);

    LinearLayout ll = (LinearLayout)findViewById(R.id.ll);

    RadioGroup rg = new RadioGroup(this);
    rg.setLayoutParams(new LinearLayout.LayoutParams(
            LinearLayout.LayoutParams.WRAP_CONTENT,
            LinearLayout.LayoutParams.WRAP_CONTENT));

    ll.addView(rg);

    RadioGroup.LayoutParams rlp = new RadioGroup.LayoutParams(
        RadioGroup.LayoutParams.WRAP_CONTENT,
        RadioGroup.LayoutParams.WRAP_CONTENT);

    RadioButton foo = new RadioButton(this);
    foo.setText("foo");
    foo.setLayoutParams(rlp);
    rg.addView(foo);

    RadioButton bar = new RadioButton(this);
    bar.setText("bar");
    bar.setLayoutParams(rlp);
    rg.addView(bar);

    RadioButton baz = new RadioButton(this);
    baz.setText("baz");
    baz.setLayoutParams(rlp);
    rg.addView(baz);
}

The resulting radio buttons fail to be mutually exclusive. This is a bit frustrating, as Hecl doesn't utilize the XML layout system. Or rather, you can use that if you want, or you can create everything in Hecl.

The individual at Google says the problem will be fixed in a future SDK release, so hopefully things will work in the near future. I also hope it's open sourced soonish, so that problems like this can be fixed without requiring the intervention of "the hand of Google". Of course, in this market, there's also the issue of things being shipped on hardware, where open source or not doesn't matter much if the bug is out there, because you simply can't fix it on millions of devices. Hopefully the product that ships will be very solid.

Yesterday, I took the Java integration I'd been building into the Hecl Android port, and moved it into its own directory, so now it's built in to the command line/j2se (non JavaME) version of Hecl for people to experiment with. Here's a slightly modified version of my posting on the subject:

I created a java/ subdirectory with an org.hecl.java package that does a few things. The core of the system is in JavaCmd and Reflection. The first creates commands out of Java classes. It utilizes, obviously, the reflection stuff in Reflection. Both are, at the moment, sort of hacky and tentative, but work. For instance, I checked in this bit of test suite today:

test java-2 {
   java java.util.Hashtable ht
   set hasht [ht -new {}]
   $hasht put foo "bar"
   $hasht get foo
} {bar}

The 'java' command takes a class, and string, and creates a new Hecl command 'ht' that is tied to the Hashtable class.

ht -new {} (could have been -new [list]) is how we call the constructor. After this, $hasht points to a Hashtable, and we can utilize its methods. Up to a point, at least - there have to be mappings in Reflection to deal correctly with types. For instance, there isn't a mapping for Enumeration right now, so calling the .keys() method wouldn't do the right thing.

There are some more bits and pieces of magic:

• Static methods are called like this:

  java java.lang.System sys
  sys gc
  

  Which calls System.gc().

• Static fields, for now, are called like this, although once again, I'm still thinking about the syntax:

  sys -field err
  

  Which is of course System.err

• In addition to -new, when you instantiate an object, you can pass in things like -text "foo", and that will attempt to call setText("foo"). This is a convenient way, in Android, at least, of setting up an object from within Hecl, without requiring the use of the XML layout files.

• Furthermore, in order to deal with constants in a convenient way, something that's reasonably common in certain types of API's (like Android), the following pieces of code are equivalent:

  layout.setOrientation(LinearLayout.VERTICAL);
  $layout setorientation VERTICAL
  

  What's happening in the Hecl code is that VERTICAL is looked up in the context of the class that's calling it, so as to avoid having to do [layout -field VERTICAL] to fetch the value. This won't work everywhere, and I suppose it could be criticized as too much 'magic'. Naturally, these sorts of things will evolve as the code does, and survive if they're useful and don't create problems, or die out if they cause bugs and aren't utilized.

There are some open issues with the ongoing work:

• I like the API for already instantiated objects, but I'm not 100% wedded to the idea of -new for instantiation. The fact that it takes a list is also a bit tricky, because if you want to do something like construct an argument to be passed in, you might end up with something like this:

  callback -new [list [list SelectDemo $spinner]]
  

  which I don't find all that pleasant. You can't just do -new [list SelectDemo $spinner] because then it thinks it's got two arguments, when it really only has one, which happens to be a list.

• The APIs created like this tend to follow Java pretty closely, for obvious reasons, and can be a bit verbose at times. I'm still thinking about this. One idea might be to provide Hecl wrappers that make common operations easier.

Aside from my hard drive crash, I've been working to figure out the best way to integrate Android and Hecl. Since Hecl has been a small, simple, humble language suitable for J2ME (or JavaME or whatever), that has meant in the past that it hasn't really been able to utilize more complex or advanced Java features like reflection, because they aren't present in the J2ME API. Android's more complete implementation of "Java" (yes, I know about the VM, but the programming language I'm writing in is still Java) opens up a number of possibilities.

First on my list of things to do was aim for a generic access layer for Java objects using reflection, so that you can do things like this:

$button settext "blah blah"

Now that we have that working, I also wanted to be able to create new Android widgets from Hecl, which is where things get tricky. The "problem" is that I want something that feels like a scripting language, not simply a reflection of the underlying Java. For instance, the XML description of a button looks like this:

<Button id="@+id/execute"
android:layout_width="fill_parent"
android:layout_height="wrap_content"
android:text="Execute"
/>

Which is actually not so bad in terms of verbosity. The Java code is uglier, in my opinion:

Button button = new Button(activity);
button.setLayoutParams(new LinearLayout.LayoutParams(LayoutParams.FILL_PARENT, LayoutParams.WRAP_CONTENT));
button.setText("Execute");

Especially the LayoutParams business, that is nowhere near as clear as the XML, where it's very obvious what values you're giving to what parameters. You wouldn't know that from the Java code without looking up the constructor for the LinearLayout.LayoutParams class.

Now, with Hecl I could of course write the code by hand to make things 'friendlier', but I'm worried that if I have to do that too much, it will take a long time to cover the rather extensive API, and it will also bloat the code. The ideal solution would be subvert the XML stuff for my own purposes, however, that's proving tricky. My first idea was to utilize the AttributeSet parameter that you pass to many View derived objects' constructors, in order to transform code like this:

button -text hello -layout_width 100 -layout_height 34

Into something that Android can digest. Unfortunately, that's proving difficult, as the internals of Android barf on the class I'm passing them, even though it, to my knowledge, implements the AttributeSet interface without any problems.

This really made me wish that they would hurry up and release the source code, because I would have a far better understanding of the relationship between the XML attributes and the Java methods if I had access to what's going on under the hood. Harumph.

My next attempt was a hack that takes things like -text and transforms them into calls to setText, but that isn't as good, because you need a little bit more custom code for things like the layout parameters, which require special handling. However, lacking the source code, or some help from the Google folks with access to it (which isn't likely over the next few days as they're doubtless off consuming turkeys), it looks like that's what I'll have to do.