The Java Conference in Milan was a pretty good experience on the whole. Took a train there in the morning, and managed to arrive just in time for James Gosling’s talk. It wasn’t a bad talk, but he’s not a great speaker – you can tell he’s more of a technical guy who’s interested in the details, and perhaps would do a better job of transmitting some of what he’s interested in/working on in a small group or individually. He did talk some about utilizing Java in real time embedded environments, which I found interesting, because those guys tend to be very careful, ‘no bullshit’ sorts of engineers, who have people’s lives riding on their work. It’s not an environment where enterprisey fad things go over very well.

In general, the technical level was reasonably high – I was pleasantly surprised, expecting a bit more in the way of marketing driven presentations.

Indeed, the association of Java with Big Business sorts of applications has long been something that has turned me off on the language. As an individual, or part of a small team, I’m much more interested in systems like Tcl and Tk or Ruby on Rails that let me do more with less code. I’ve felt that it made more sense to avoid competing on terrain where the differentiating factor is the ability to crank out massive amounts of code (getThis, setThat, etc…), because alone, I can’t take on a larger group.

As I was explaining some of this as part of ‘Why I wrote Hecl’ to one of the other guys slated to talk about J2ME in the speakers’ lunch room, I was talking about how “non e` che Java mi piaccia in modo particolare” (I’m not particularly wild about Java), the person I’m talking with starts gesticulating at me, and I turn around and look, and who should be there, talking with someone else, but James Gosling. Hope no one provided a translation for him!

To tell the truth, I think Java has its place. It seems to be an appropriate technology for something like OFBiz, or other apps that genuinely do need to be reasonably large and complex. Just that I have more fun with other languages like Ruby and Tcl.

We’ve been having some interesting discussions on the Hecl mailing list about how to do concurrency:

http://sourceforge.net/mailarchive/message.php?msg_id=19566676

(although it seems SF has managed to lose a portion of the conversation!)

When all things are said and done, we have a couple of options:

• Try and immitate Tcl’s select-based event loop. Not really concurrency, but it does a good job of “faking it” for things like GUI’s, if done correctly. For the right kind of code, it makes life very easy for the programmer, as you don’t have to think too much about concurrency issues, and you don’t have to deal with threads. The downside for J2ME is that we don’t have the equivalent of select, so we’d have to do the system with threads, which would probably be costly in terms of the resources required to set it up.

• Tcl style threads, where each thread gets its own interpreter, and data is mostly shared by message passing between the threads. Conceptually, I think this is a cleaner way of doing threads, compared with ‘sharing everything and using mutexes’, and it’s a little closer to how Erlang handles concurrency, which is very nice indeed. For J2ME, I’m a little bit worried that this might be too resource intensive, and potentially slow to launch new threads.

• A low-level interface to threads that just exposes the Java API. This would be nice, and lightweight, but potentially quite complex to program if we attempt to make the interpreter reentrant. There are a lot of issues with things like stack frames – for instance, say one subroutine (proc) adds a stack frame, then a second thread kicks in and exits a proc, tearing down the stack frame again. If you put a big lock around procs, that just leads to blockage if a proc is doing some sort of long running calculation, defeating the whole point of the exersize.

• Deal with concurrency on a case-by-case basis, building it into each command that needs it. Not very elegant, and probably requires just as much work, if not more, than other options.

At this point, I’m tending towards each thread having an interp, because option three seems very prone to extremely difficult bugs. Not sharing the interpreter has worked well for Tcl both from the implementation and end user perspectives. It is, however, not an easy subject and I’m still thinking about it.

Note comments are broken, so email me with questions or comments.

Ciao everyone…

My talk on Hecl has been accepted for the Italian Java Conference in Milan!

http://it.sun.com/eventi/jc06/programma/milano_more.html

If you’re going to be there, let me know… it’s always fun to see people I haven’t had the occasion to see in person for a while.

Unfortunately, I won’t be going to ApacheCon Europe in Dublin, which seems like it will be a fun event.

Note comments broken due to Typo being…abandoned?

Si Chen has a nice write up of a talk he gave discussing OFBiz, and more general, open source in the business world:

http://opensourcestrategies.blogspot.com/2006/06/why-enterprises-are-adopting-open.html

When I started getting involved in the free software world, open source was very squarely aimed at tools, and applications for our fellow geeks. It’s interesting to watch it march off in new directions, such as Ubuntu on the desktop (my wife requested that I put Ubuntu on her new laptop!) and systems like OFBiz, which are very much aimed at higher level business processes.

This is noteworthy – running Tcl on the eCos embedded operating system:

http://ecos.sourceware.org/ml/ecos-discuss/2006-06/msg00083.html

http://ecos.sourceware.org/ml/ecos-discuss/2006-06/txt00000.txt

Looks like my friend Salvatore’s Jim interpreter is gaining some mindshare in the embedded space.

Note

Comments are broken… ask the Typo guys when they’ll be fixed.

Today, I needed to do something that I didn’t find a handy method for in Rails. The API has both a send_data and send_file method, but they are both ‘one shot’ in that you have to send all your data at once.

def generate_file
  @headers["Content-Type"] = "text/comma-separated-values;"
  @headers["Content-Disposition"] = "filename="some.file.txt";"

  i = 0

  render :text => Proc.new { |response, output|
    ...
    output.write("some generated text...")
    ...
  }, :layout => false
end

I more or less copied that from the send_file method, and indeed, I’ll even admit that I’m not entirely sure why it works. What I don’t get is what “connects” the output variable in the Proc to an actual file descriptor somewhere. render_text doesn’t really clear things up (well, for me, at least!).

  def render_text(text = nil, status = nil) #:nodoc:
    @performed_render = true
    @response.headers['Status'] = (status || DEFAULT_RENDER_STATUS_CODE).to_s
    @response.body = text
  end

In any case, the code does do what I want – perhaps someone else will find it useful.

NOTE

Comments are broken, as the typo journal system seems to have been somewhat abbandoned… argh!

While working on Hecl, it has sometimes really surprised me how much it’s possible to delve into seemingly trivial details. Just for starters, we’ve been talking about what to call string commands.

Originally, I’d used names like slen and sindex to have something quick and easy to write, but no one seemed to like them that much. Wolfgang implemented a bunch of new string commands (very much appreciated!), and took the Tcl approach of command subcommand, like string index, and also suggested some other things like str.index. I am strongly opposed to the string length style commands though, because I think that’s just too much typing for a language that’s supposed to get you up and running quickly. The compromise I proposed and implemented was to use more ‘C style’ commands like strlen, strindex, strfind, and so on. I’m not a big PHP fan, but they seem to have taken a similar approach. It’s not too verbose, but you can tell more or less what the command is for.

Even more seemingly trivial – indexes for strings and arrays. Hecl takes the Lisp approach rather than the C approach, and utilizes commands instead of syntax, even for list and string access, so to get the second character of the string “foo”, you would do: strindex "foo" 1 . Like C, Hecl indexes start at 0. Where things get tricky is in calculating indexes “from the end” – how do you tell the language to “fetch the last character from the string ‘foo'”?

Tcl lets you write a string like “end” or “end-1”, which is handy compared to the process you would go through in a language that doesn’t help you out: take the length of the string, and subtract from that as needs be. I prefer the approach that I first saw in Python, though, use -1 for the last character:

>>> "abcde"[-1]
'e'

Ruby takes the same approach. It’s nice to be able to just insert a -1 rather than have “end-1”, which sort of looks like an expression, but isn’t really (Hecl doesn’t even have normal infix expressions), especially because you can easily calculate the -1 and insert it as is, instead of having to do some string interpolation like "end-$foo".

Where things get tricky is when you want to do something like specify a range, say from the 3rd letter to the end of the string.

In Python, they use syntax, specifically by simply not including the end of the range:

>>> "abcde"[3:]
'de'

That doesn’t work really well with Hecl – I suppose we could pass in a blank string instead of a number, like so:

strrange "abcde" 3 ""

But that’s not very clear, and IMO it’s slighly ugly to pass in a string where you want to use a number. I had a look at Ruby’s behavior, and found it more to my liking:

irb(main):001:0> "abcde"[3..-1]
=> "de"

The difference with Python is that the second index is inclusive, whereas Python includes the character at the first index, and excludes the second:

>>> "abcde"[3:-1]
'd'

I decided the Ruby approach worked best for Hecl, so now we have:

hecl> strrange abcde 3 -1
de

If you’re still reading this, I guess you see what I mean by how much one must delve into what must seem like very unimportant minutiae to… well… normal people!

An interesting post by Ian Holsman, one of my colleagues in the Apache Software Foundation who is also interested in open source business and economics:

http://feh.holsman.net/articles/2006/05/12/is-your-project-a-hydra

I take a more lassez-faire approach, myself. Ian’s advice is valuable to those sorting through open source projects to use or get involved with – a full-fledged free software project with many users and committers who are independant of one another is usually going to be a better proposition than something run by one company, or one person. A lot of ASF thinking focuses on the community being the real value in an open source project, and it’s obvious that it adds a lot of value. However, the licensing is important too, because it’s your escape hatch. It means that if the company goes away, or decides to create a proprietary product, that you have the option to create a community around the open source code, by taking over its development and maintainance.

ASF Incubator

Ian also mentions the ASF Incubator, which I’ve been involved with first hand through my involvement in incubating OFBiz. One of the big hurdles that the project faces is getting a scrap of paper from everyone who has ever contributed anything important to the project. And since OFBiz is a real open source project with committers all over the world, and, over the years, many contributions, that is a lot of paper to collect! Thanks to the efforts of the OFBiz team, they’re doing an admirable job of completing the task.

However, I can’t help but observe that “incubation” is a far easier process to go through if the code arrives in the form of a corporate donation, because it all comes from one place. Unfortunately, I think that leads to some selection for “hydra style” projects, although to their credit they may be trying to break out of that by joining the ASF (they need to if they want to successfully complete incubation). Still, I think it is likely to lead to a more “corporate” organization.

We have a decision to make in the Hecl project that highlights how much computer programming is really an art, rather than a science. Namely, how to name a series of string handling commands. Tcl has a series of string commands like

string first string1 string2 ?startIndex?
string equal ?-nocase? ?-length int? string1 string2
string index string charIndex

which are mostly consistent. At times, they’re also just a bit more unwieldy than I might like. For instance;

set somechar [string index "the nth char is" 4]

is just not as quick to write as

somechar = "the nth char is"[4]

For something that you need to use frequently like ‘string equals’, a shorter, quicker version is really a must – infact, Tcl’s expressions recognize the ‘eq’ operator. Hecl has ‘eq’ and ‘ne’ as commands, which makes things a bit less consistent, sure, but a lot more convenient.

The decision we need to make, though, is where to draw the line? String equality is so common that it doesn’t take much thought to opt for convenience. How about string index? string length? The artistry involved is to make something that’s appealing to people, but also practical. Elegant, but not so much that it remains unsullied by use in real work.

Is it just me, or do these kinds of side trips happen to other people in the open source world?

For instance, I want to add SMS capabilities to Hecl, so I did some research, realized that it’s not too hard, and wrote the code. The build system we use is based on Antenna, which turns out to have a problem with the 2.2 version of suns Wireless ToolKit in that it doesn’t let you select wma11.jar or wma20.jar, as the 2.2 version requires you to do.

Poking around some more turned up a patch for Antenna that would do just what I need. However, it’s from late 2005. It seems to be quite popular in the J2ME world, so what happened to it? Is the project no longer maintained? I wonder if it would be possible to fork it and get some people involved in keeping it up to date?

And so it goes… and I get sidetracked from what I’m working on. Sometimes it leads me to new and interesting projects, but I suppose that, long-term, it’s not the best use of my time. And yet I find that the pattern repeats itself, and all things considered, I really enjoy hacking on open source software, so I do end up having fun.