Jeff “Tiara” Atwood release of LangPop.com

Jeff Atwood, of “Coding Horror”, snarkily writes, via twitter:

programming language popularity. they should award tiaras! http://langpop.com/

Since LangPop.com site was due for an update, in the same spirit of snarky fun, I am calling this release the Jeff “Tiara” Atwood release as a way of saying thanks for driving a lot of traffic my way.

Jeff "Tiara" Atwood

Nothing much new this month, really. A lot of people have been asking for Scala, so perhaps that will appear in the coming months. I think it probably shows up on most of the metrics now.

Yes, I know, the Amazon results are still borked. If anyone out there knows the team at Amazon that handles that stuff, it would be really great if you asked them to fix the fact that “C” == “C++” == “C#” in their search. Thanks!

PS, let he who is without copious amounts of “merit badges” cast the first tiara…

Objective C vs Cocoa

I’ve been adding a few things to LangPop.com, specifically ColdFusion (it’s not something I’m interested in, but I’ve had a lot of requests for it…) and Objective C, which is quite popular indeed, thanks to Apple.

Indeed, it is Objective C that is a bit problematic: “Cocoa” is more widely used than the name of the language itself! On http://search.yahoo.com, Cocoa programming turns up a lot more hits than Objective C programming:

  • “objective c programming: 71,300
  • “cocoa programming” : 608,000

That’s an order of magnitude!

Now, on one hand, I aim to measure popularity of the language itself, not of libraries. I simply can’t add every popular library, toolkit or framework for every language – it would be a huge task, and would likely only really make the results worse. However, it might not be entirely fair to leave that many actual users uncounted just because they all use Objective C in one particular environment. And just to further complicate things, while most people using Cocoa are writing Objective C code, there are other bindings for it…

My inclination, for simplicity’s sake, is still to simply go with Objective C. What do you think?

New LangPop.com stats, open source bits and pieces

I released some new statistics on LangPop.com. There are some new things:

  • The normalized results are now user configurable – you can put in your own weights for the various data sources (such as Amazon, Craigslist, and so on), and see how the results change.
  • Freshmeat.net fiddled with their web site, and statistics are not going to be available this month. Hopefully they will be able to correct the problem soon.
  • On the other hand, I wrote some code to grab the data from Ohloh, so there’s something new to look at while we wait for Freshmeat to come back on line.

So there is a fair amount going on. Hopefully Freshmeat will come back soon, and I can also add a few fixes to the IRC statistics which I’ve been thinking about.

In the process of doing the above, I also managed to do some open source work, albeit of a minor nature:

  • I updated LangPop.com to use the Flotr Javascript chart library instead of Plotr. So I also updated my Ruby code to integrate it. It’s a fairly thin layer at this point, but it does what I need. It’s available here: http://github.com/davidw/chartr/tree/master
  • I did some hacking on the Simile Timeline code, (used here on LangPop.com), correcting a bug, and also creating a system to install the whole thing with a simple command. Normally, it’s kind of difficult to just use the Simile code on your own site, so I created an (admittedly hacky) way of making it easier to install. That code is here: http://github.com/davidw/standalone-timeplot/tree/master

Oh, yeah, and we moved from Innsbruck, Austria back to Padova, Italy!

Javascript Charts

I’ve been doing some work on LangPop.com, and one of the things I’d like to do is update the chart software. What I have now, Plotr seems to work ok, but being the tinkering type, I want to fix things, even if they’re not broke. Truth be told, my worry is that Plotr isn’t maintained any more, so it would be a good idea to find something that’s receiving a bit of attention.

The candidates:

These seem to be fairly up to date in the sense that someone worked on them recently. There are some older ones like PlotKit, that do not appear to be maintained any more. It is entirely possible that I missed a good one. Another possibility would be to use Google’s chart API, but I’d rather be a little bit more in control of things than to farm that out, and I also am planning on doing some interactive features in the near(ish) future.

So let’s have a look:

Flot

Based on JQuery, this one looks fairly complete, and has a lot of different, nice looking charts. Since I don’t really care what library it’s based on JQuery seems as good as any other, being quite popular these days. Installation is pretty simple, and, having good defaults, it’s easy to get nice looking data up on the screen in short order.

Flotr

Modeled after Flot, Flotr uses Prototype (the Rails default) instead of JQuery, and is the follow up to Plotr. I’m not quite sure what the motivation was behind the name/project change, but this seems to be where the guy is spending his time. Since he did a good job with Plotr, this one seemed worth a look too.

ProtoChart

This one is based on Prototype too. It immediately annoyed me by uncompressing from the .zip file in the current directory, scattering files around. I’ve always found this behavior a bit antisocial. This code claims to be “motivated by Flot, Flotr and PlotKit libraries”, which indicate that it’s fairly recent. However, my feeling is that the problem is not necessarily how old code is in these projects, but how quickly they spring up, bloom, and then stagnate. I’d like to use something that’s got some staying power… But anyway, ProtoChart looks like pretty good code, even if the distribution is a bit minimal, and doesn’t come with examples.

If you look at LangPop.com, you’ll notice something that’s very important for my choice: I need a barchart that has horizontal bars, rather than the more traditional vertical bars. This is because it would be quite difficult to squeeze so many languages across the screen horizontally. This ended up being the deciding factor: like Plotr, Flotr supports horizontally oriented bar charts, making it the obvious choice. The other two libraries looked pretty good too – Flot, in particular, looks like quite solid code.

So, there you have it – very brief reviews of an incomplete selection of libraries! In my defense, the goal is to attempt to dominate my maximizing nature, pick one, and get on with doing some cool stuff with LangPop. I do, however, welcome comments and suggestions.

Langpop.com – now with IRC

Someone suggested the clever idea of counting IRC channel users as a metric to use for langpop.com, so I went ahead and gathered that data. Since this is the first month, there are a few glitches (I missed #fortran – sorry guys), and a thing or two to iron out, but I thought I’d go ahead and publish the new results. IRC counts towards the “discussion” portion of the stats, rather than the main part. Like everything else, it has its biases – I used the Freenode network, which is definitely more about free software than other places might be. One number that really jumps out is just how popular the Haskell channel is – it’s right up there with all the most popular languages. It will be interesting to see if, over several years, all the “talk” about Haskell starts to translate into people using it for fun and for their jobs.

I welcome comments on this post, but there’s also a Google group I’ve set up for discussion of the results, data sources, languages, and so on:

http://groups.google.com/group/langpop/

Another idea I’m kicking around is to create a separate page for “up and coming” languages, stuff like Scala, Clojure, F#… that kind of thing. It might have to use different metrics, because the newer languages often don’t show up on many of the ones I use for the languages we currently have.

Angry Perl Users

I work on LangPop.com for fun. I make a tiny bit of money from the adsense there, but nothing much to speak of. Unlike the TIOBE folks, I don’t sell my data to anyone. So it can be a bit frustrating when people get all bent out of shape about the results. A case in point was my recent article here, entitled Python “Surpasses” Perl?.

In case the title wasn’t a dead giveaway, I’ll spell things out:

  • I was poking a bit of fun at the idea that any statistics of this nature could pinpoint, to a specific month, the moment when one language “passes” another. They’re way too fuzzy for that!

  • I point out that Perl is still quite popular. According to the Freshmeat metric I was reporting, Perl actually still has more code out there than Python.

And just to be clear: I don’t use Perl or Python or really care which one is more popular. I mostly use Ruby these days, with Tcl, Java, and bits of C thrown in for fun. In other words: langpop.com is not an evil conspiracy to discredit Perl.

However, I also had the temerity to suggest that Python might be a little bit more popular in terms of new things happening. Shock! Horror! Well, that certainly irritated some Perl users. They called my statistics lies, they misspelled my name (as well as the word “you”, which is a lot simpler than my name by any standard), and some of them went off on rants that were pretty badly off target (SourceForge is never mentioned in Langpop or this online journal!). However, not once did they really get around to answering my points, or proposing any sort of valid alternative metrics.

One of my favorite Linux/Free Software news sources, lwn.net published their 2009 predictions, and had this to say about Perl:

It will be a make-or-break year for Perl. If the Perl developers cannot either bring new life to Perl 5 or turn Perl 6 into something real, this language will, by the end of the year, have moved well down the road to “legacy” status.

As can be imagined, that set them off all over again. The mental image I get is a big angry bull, frothing at the mouth, and ready to stomp that damn red cape.

Here’s the problem, though, guys: the cape isn’t what’s holding you back. Trust me, I know.

Once upon a time, when people spoke of scripting languages, they talked about “Perl, Python and Tcl”. No Ruby, no PHP, although both existed. I happen to like Tcl a lot, and it measures much worse than Perl on all the popularity charts… worse than plenty of other languages, these days. Lately, I don’t use it so much, but once upon a time I dedicated a lot of mental energy to wondering why things weren’t going better. And in that community too, there were people who consistently held their fingers in their ears and yelled “LALALALA there is no problem!”. In the long term, it’s not an effective strategy: it makes everyone but the other True Believers think you’re going off the rails, because the evidence is plain to see that things weren’t quite like they used to be, that there are problems. And it doesn’t really address the problem in a constructive way, either. Long term, what works is to produce good code, and get it in people’s hands.

The most effective way of dealing with problems of this nature includes these points:

  1. Not froth at the mouth any time you perceive a slight to your technology of choice.
  2. Talk up the cool things you’re working on. Be positive, explain what’s new and exciting.
  3. Acknowledge reality. For instance with Perl (or Tcl), it’s plain to anyone that they aren’t as popular as they once were. Don’t deny it, but focus on the fact that they’re still widely used, and lots of new code is still written in them, and that they’re still the focus of a lot of work to improve them.
  4. Actually follow through. Jon Corbet’s (Linux Weekly News) point is not a bad one – you can have all the internal releases and milestones and general fun hackery that you want, but at a certain point, you need to get working code out in front of people. This is something Ruby seems to be suffering from these days too (as observed from afar) – lots of next generation fiddling, and tons of good ideas and energy, but to me there is a lack of technology actually flowing my way in a timely manner. With the possible exception of JRuby, which is making lots of progress and putting good code out there for people to use.

To conclude things, since the Perl guys were so unhappy with Freshmeat, here are some statistics from Amazon and Craigslist, comparing about a year ago with the most recent numbers:

As can be seen, both Perl and Python are dropping in Craigslist postings, but Perl is falling farther.

As a final footnote, I am sorry about this journal not accepting comments past a certain point (10 days is the cutoff) – otherwise I get 100’s of spam postings a day. Unfortunately, the software I’m using didn’t deal well with alerting people to the fact that comments were closed. I filed a bug, and they’re working on it.

langpop update

One of the things I have noticed running http://www.langpop.com is that, pulling data from different sources, the odds are surprisingly hight that at any given point in time, one of the sources will be out of wack, often in a subtle way that’s hard to see in the results. For instance, in the December 2nd run, the Amazon results were a bit wonky, and Delicious has been gradually falling out of the Yahoo search results (weird, that). I decided to go with screen scraping for Delicious, even though, frustratingly, Delicious can’t tell the difference between C, C++ and C#, so while we wait and hope that that gets fixed, those languages are all going to show nearly identical results. I have updated the site.