Angry Perl Users
I work on LangPop.com for fun. I make a tiny bit of money from the adsense there, but nothing much to speak of. Unlike the TIOBE folks, I don't sell my data to anyone. So it can be a bit frustrating when people get all bent out of shape about the results. A case in point was my recent article here, entitled Python "Surpasses" Perl?.
In case the title wasn't a dead giveaway, I'll spell things out:
I was poking a bit of fun at the idea that any statistics of this nature could pinpoint, to a specific month, the moment when one language "passes" another. They're way too fuzzy for that!
I point out that Perl is still quite popular. According to the Freshmeat metric I was reporting, Perl actually still has more code out there than Python.
And just to be clear: I don't use Perl or Python or really care which one is more popular. I mostly use Ruby these days, with Tcl, Java, and bits of C thrown in for fun. In other words: langpop.com is not an evil conspiracy to discredit Perl.
However, I also had the temerity to suggest that Python might be a little bit more popular in terms of new things happening. Shock! Horror! Well, that certainly irritated some Perl users. They called my statistics lies, they misspelled my name (as well as the word "you", which is a lot simpler than my name by any standard), and some of them went off on rants that were pretty badly off target (SourceForge is never mentioned in Langpop or this online journal!). However, not once did they really get around to answering my points, or proposing any sort of valid alternative metrics.
One of my favorite Linux/Free Software news sources, lwn.net published their 2009 predictions, and had this to say about Perl:
It will be a make-or-break year for Perl. If the Perl developers cannot either bring new life to Perl 5 or turn Perl 6 into something real, this language will, by the end of the year, have moved well down the road to "legacy" status.
As can be imagined, that set them off all over again. The mental image I get is a big angry bull, frothing at the mouth, and ready to stomp that damn red cape.
Here's the problem, though, guys: the cape isn't what's holding you back. Trust me, I know.
Once upon a time, when people spoke of scripting languages, they talked about "Perl, Python and Tcl". No Ruby, no PHP, although both existed. I happen to like Tcl a lot, and it measures much worse than Perl on all the popularity charts... worse than plenty of other languages, these days. Lately, I don't use it so much, but once upon a time I dedicated a lot of mental energy to wondering why things weren't going better. And in that community too, there were people who consistently held their fingers in their ears and yelled "LALALALA there is no problem!". In the long term, it's not an effective strategy: it makes everyone but the other True Believers think you're going off the rails, because the evidence is plain to see that things weren't quite like they used to be, that there are problems. And it doesn't really address the problem in a constructive way, either. Long term, what works is to produce good code, and get it in people's hands.
The most effective way of dealing with problems of this nature includes these points:
- Not froth at the mouth any time you perceive a slight to your technology of choice.
- Talk up the cool things you're working on. Be positive, explain what's new and exciting.
- Acknowledge reality. For instance with Perl (or Tcl), it's plain to anyone that they aren't as popular as they once were. Don't deny it, but focus on the fact that they're still widely used, and lots of new code is still written in them, and that they're still the focus of a lot of work to improve them.
- Actually follow through. Jon Corbet's (Linux Weekly News) point is not a bad one - you can have all the internal releases and milestones and general fun hackery that you want, but at a certain point, you need to get working code out in front of people. This is something Ruby seems to be suffering from these days too (as observed from afar) - lots of next generation fiddling, and tons of good ideas and energy, but to me there is a lack of technology actually flowing my way in a timely manner. With the possible exception of JRuby, which is making lots of progress and putting good code out there for people to use.
To conclude things, since the Perl guys were so unhappy with Freshmeat, here are some statistics from Amazon and Craigslist, comparing about a year ago with the most recent numbers:
As can be seen, both Perl and Python are dropping in Craigslist postings, but Perl is falling farther.
As a final footnote, I am sorry about this journal not accepting comments past a certain point (10 days is the cutoff) - otherwise I get 100's of spam postings a day. Unfortunately, the software I'm using didn't deal well with alerting people to the fact that comments were closed. I filed a bug, and they're working on it.
Trackbacks
Use the following link to trackback from your own site:
http://journal.dedasys.com/trackbacks?article_id=2128
about {{count}} hours later:
What you and TIOBE are doing is you guys keep looking at the PDF and not the CDF.
You are looking at the differential and not the cumulative.
Even worse TIOBE relies hype machines to generate stats. I'm sorry but external marketing is not a good indicator, take Toronto, ruby in Toronto is dead. Even the people making money of off training for ruby and rails are dead. But will that show up?
about {{count}} hours later:
David,
Please accept my sincere apologies for your name typo. I don't know where I got that from and have now fixed it on my blog (with updated note).
"U" is just a shortcut for "you" which is the curse of the SMS/text age! This post is verbatim to what I tried to leave in your blog comments hence the use of "U/you". So please don't take it to be offensive in any way.
/I3az/
BTW.... I've nothing against langpop.com and its good that someone has put this together.
about {{count}} hours later:
You may not get the bots, but you definitely get people who spam. I think every new posts should be checking for content identity with the post before. ;-)
Anyway, to the point, I think that Tcl dropped off quickly is largely the design of Tcl's fault. It never was meant as a real "glue" language like perl (anyone look at how perl grew).
Unix popularity helped perl grow a LOT. This is the biggest difference between perl and Tcl.
However, years later designers learned from perl's mistake. The biggest mistake was to not offer a good OOP model. Even PHPs OOP model is more solid (but php itself is a horrible language, it is even more amazing that one can do great things in php).
I claim both python and ruby have better OOP than perl. (I am a ruby user, so I am biased in favour of ruby, but I think python is a fine language as well.)
A solid measure of how popular a language is is:
programming.reddit.com
And I am sorry to say but python and ruby easily surpass perl there (haskell is also very popular)
In top 20 right now I counted 2 posts about python with specific python code, one post about rails (i count this as "ruby) with specific code, and one with haskell and specific code. (I did not count the reference to language names, because someone could only rant about a specific programming language, instead of "using" it somehow)
People need to look at where the cool ideas happen. This will be the heart of programming.
about {{count}} hours later:
With respect, when you admit that you know that basing conclusions on Freshmeat numbers is flawed, but you persist on doing so anyway, and when you poison the well with a headline you know is misleading and based on false premises but hide that behind the old marketing saw of slapping a question mark on it, do you expect that people will consider your comments unbiased?
Oh, and I suggested you look at the growth and change rate of the CPAN as an alternate statistic. I'm sorry if that makes your job of correlating language popularity more difficult, but shouldn't a reasonable statistician try to find the best data possible? Perl programmers don't use Freshmeat, by and large. They use the CPAN. If you ignore the largest and most lively community of Perl programmers, you'll have bad statistics.
Python's popularity may be increasing over Perl's -- but if you're going to show that with data, show it with good data, and don't wave away complaints that your data is bad when you yourself admit that it's bad.
about {{count}} hours later:
One more thing -- I can't read the Y axis labels on your graphs in this post, so I can't interpret them. Would you mind terribly reuploading them?
about {{count}} hours later:
chromatic, in terms of "flawed", that is the nature of the beast. It's simply not possible to have perfect data for something like LangPop, so we do the best with what we have. All the metrics used have various flaws. I try and point them out in the text of the site, as I think that it's only fair to alert people to potential biases and shortcomings. So, I don't think the data is "bad", as in garbage, I think it is imperfect.
Now, on to Freshmeat. You say that Perl programmers don't use it. Why then, is it, that there are more things tagged Perl than Python on Freshmeat? Obviously, at some point, quite a few Perl programmers did use it. And they did so despite having CPAN, which has been around for longer than Freshmeat. So saying "it's because of CPAN" just doesn't make sense to me. Perl is to this day one of the most popular languages on Freshmeat!
To tell the truth, I don't think anyone uses Freshmeat a lot any more, but that effect should more or less hold true across languages - or even be in Perl's favor, if it were relatively more popular in Freshmeat's heyday, which I suspect might be the case. Even with Tcl, which doesn't have a nice package repository like CPAN, I never really used Freshmeat to find stuff, nor did most Tcl users. My guess is that Freshmeat does not take the place of a package repository for other languages, either.
I'd be happy to include other data sources, but it has to be a source that at least attempts to include various languages, because apples (CPAN) to oranges (Rubygems) to NULL (Java, C, etc... most languages don't have one) comparisons aren't very interesting. One thing I could think of is "X language library" with Yahoo search, which might be a decent stab at looking at how much people are talking about extensions for a given language.
about {{count}} hours later:
Chromatic: The reason people think Perl 6 isn't out yet is a marketing failure. You need to make a big splash saying "Perl 6 is ready for production use" then people will shut up (or start criticising the actual language rather than its perceived status as vaporware). Commenting here rather than at use.perl since I have far too many accounts on random websites.
about {{count}} hours later:
mark: I do use reddit stats for LangPop.
As to why Tcl declined in popularity, I could write a lot about it, and intend to do so, some day.
about {{count}} hours later:
I certainly don't mean to sound angry (I'm not) and I am very grateful for langpop, as it's much better than almost every other popularity contest I've ever seen -- but I do believe that the data in your previous post was flawed to the point where it didn't support your conclusions, no matter how accurate those conclusions might be.
The library search approach sounds like an improvement, thought you might have to tailor it to language-specific jargon. ("Perl module" or "CPAN module" will likely give more results than "Perl library".)
If you want accurate data, you have to go where the data is. I haven't submitted anything to Freshmeat in nearly a decade, though I've produced a lot of software in that period. Many other people who've produced a lot of Perl can say the same thing. Ignoring CPAN wholesale because other languages don't have one is a little bit like discounting web programming (sorry Python, PHP, JavaScript, and Ruby) because it's painful in C and C++.
1 day later:
You found an angry person on the internet? That's a surprise, who'd have expected that?
{{count}} days later:
programming.reddit.com programmers are not programmers. They are undergrads. If you trust anything they say you're being quite foolish.
{{count}} days later:
What a pointless bickering... Ruby beats them both.
{{count}} days later:
Hi David,
I found this searching summize.com for Perl. I try to answer people's Perl questions and clarify things for them, and do so without descending into advocacy. I used to be a fierce Amiga advocate -- we were a legendary breed. Commodore died anyway, and after some time, I realized that that had everything to do with Commodore screwing up and nothing to do with people giving them free advertising.
http://use.perl.org/~Ovid/journal/38010 -- Ovid, a very well known name in Perl circles with many years of service to the community under his belt, agrees and presents other numbers plotting Perl's declines.
http://chainsawblues.vox.com/library/post/why-i-use-perl-and-still-hate-dynamic-language-weenies-too.html -- Matt Trout, another extremely well known name in Perl circles with countless important modules and talks under his belt talks about the benefits of not having so many fanboys in the community as other more popular languages. Whether he's right that Perl has fewer, he's a community leader who doesn't want these guys around.
Larry Wall has been quoted as saying "there's nothing in this world worth being a snot over".
The message is pretty clear... if you're trying to defend your favorite language from perceived attacks, you're not making yourself useful. If you're creating things people want or helping to do so, then much more likely, you are useful.
Not to shift blame, but talking about language popularity, you're going to attract these sorts. Regardless, I wanted to go on record as renouncing these idiot tactics.
As you said, Ruby is having some growing pains. I sincerely wish Ruby the best. Java came out as a model of cleanliness but then pressure mounted for such things as autoboxing and generics, and the internal architecture started to creak a bit. Tcl had serious growing pains moving from "everything is a string" to "some things are numbers" in its internal representation which peeked through to its syntax. Perl had serious growing pains growing from a better sed+awk+bash into an OO-ish (if you squint just right) language with closures and other fun stuff. Each language has its own story and they're all a hell of a lot more interesting than trying to quantify it down into a continuum of good/bad sucks/rules.
Data is fun to play with, and I hope you continue to enjoy playing with it despite the frothing.
Cheers, -scott
{{count}} days later:
Scott, thanks for an interesting comment and a great attitude! I suppose langpop probably does tend to attract the sorts of people who want to use it as a club to bash other language users with, but that's really not its purpose.