Elixir vs Erlang: a question of momentum

I fiddled around with Erlang the first time a number of years ago, and have come back to it off and on ever since.  I’m currently using it professionally, and enjoying the experience a lot.  However, Erlang has always struck me as a language that is not particularly “well-rounded”: the best in the business at certain things, and… not always a lot of fun for others.  String handling for instance.  You pick up Ruby or Python or Java, you know they’ll probably be “ok” for anything you throw at them.  Maybe not great, but you’re likely to at least be able to put something together, even if it’s got some flaws.  Erlang has always felt a bit more hit-and-miss – if you’re not operating in its sweet spot, you may be in for a slog.  Not that it’s an unpleasant language.  I’m rather fond of it, actually.  Just that you might end up having to write things yourself that you’d get as easy to install libraries in other systems.

While not a user of it yet, this is one of the reasons why it’s so interesting to watch the Elixir programming language evolve.  Outside of the other languages besides Java targeting the Java Virtual Machine, it’s not every day that we get to see a new language created for a VM.

It’s impressive how much momentum they seem to have.  Sometimes Erlang’s progress feels a bit like trench warfare, where major battles are fought over a few meters.  Elixir, on the other hand, seems to have broken out and is charging ahead fast.  This is of course something of a generalization, and there is plenty of good stuff happening with Erlang too, just that I get a feeling of Elixir racing ahead.

Being, among other things, a “web guy”, this is particularly noticeable to me, and perhaps less so to people using Erlang for what Erlang traditionally has done well.  Erlang has long had some tools for the web, but they’ve never been as easy to use as, say PHP, and they don’t have half the tools available that something like Ruby on Rails has available to it.  It seems that Elixir is cranking out a lot of stuff that Erlang never really got:

  • A web framework, Phoenix that seems to have a lot of work going into it.  Erlang has several web servers, like Yaws, and the excellent Cowboy, as well as some other bits and pieces that work well.   But Erlang has been lacking in the framework space.  You can criticize frameworks, but good ones do save you some time initially, and then get out of your way when you want to do more.  They’re a great starting point for someone who just wants to do some learning by making something that works, and then iterating.  Trying to put together cowboy, poolboy, a circuit breaker, a database driver, erlydtl, and various and sundry other pieces, for someone not well versed in Erlang is a losing proposition.  Their friend, using Elixir, will at least have something up and running in short order.  Also, if putting together the same bunch of components is something a lot of people do, why not do it once and do a good job at it?
  • A package management system, “hex”.  Erlang has rebar, to download and compile source packages, which works ok, but hex seems to do more.  A package system has always seemed around the corner for Erlang, and the Elixir guys just went out and built one.

Those are the two that stick out the most, but there are all kinds of things going on – here are a few random examples:

  • Ecto database driver, with Postgres as the first database integrated (yay!).
  • Ecto migrations.
  • Porcelain, for working with external processes, where Erlang  is somewhat lacking (you need https://github.com/saleyn/erlexec ).

These are things that are available for Erlang, but I’m just impressed at how quickly the Elixir guys are cranking stuff out, even if at the same time I also wonder if for some things, they’re not a bit too ready to create something new when there’s already a perfectly good solution.

Naturally, Elixir is young, and can afford to experiment more in some ways, and perhaps much of this code is not quite “battle tested”, but it’s still nice to see things moving so quickly.

Maybe it’s just the “grass is greener” feeling, but it seems like Elixir has something which Erlang never really managed to acquire: momentum.   It should be very interesting to see how this plays out over the next few years.

The bicycle: a thinking machine

No, not a machine that thinks, but a machine that helps you think.  Me, at least.

With two kids, a job, side projects, and a lot of things going on, it’s easy to put off fun things like riding my bike, but when I do get out, I realize how good it is for me not just physically, but mentally.

  • I’m finally alone – no coworkers, no kids, no wife.  It’s fun to go with friends too, but those solo rides are where thinking happens.
  • No emails, no popups, no phones ringing, no distractions!
  • It gets the blood flowing to my brain in a way that makes me feel more awake, and thinking quicker.
  • It’s mostly smooth and steady so you get into a rhythm that makes it easy to think: the ideal pace is not too hard, nor too slow.

And of course, if you live in the right place, it’s beautiful:


The Cargo Cult Cofounder

Paul Graham is – rightly – regarded as an authority on startups, having done a successful one himself, and having seen hundreds pass through Y Combinator.

So what he says carries a lot of weight.

One thing he wrote, though, seems to have not been completely understood by many people.

Not having a cofounder is a real problem. A startup is too much for one person to bear. And though we differ from other investors on a lot of questions, we all agree on this. All investors, without exception, are more likely to fund you with a cofounder than without.

From http://paulgraham.com/notnot.html

Having read this, a lot of people decided they absolutely, positively must find themselves a cofounder at any cost – because it’s what you have to do.  But from everything I’ve seen and read, that’s backwards: the good cofounders are the ones you come by naturally, the ones you already have a lot of history with.  People you already know you can count on, because you have already.

It’s a little tiny bit like having a baby: long term, things usually go better if you have one with someone you’re really in love with and committed to, rather than the first person you come across who is willing and happens to have a complementary set of reproductive organs.

This is, naturally, something of a catch-22, because by the time you get around to deciding to start the company, you probably already need to have that shared history and background with someone for them to make a good cofounder, but if you don’t have anyone like that available, it’s not like you can create that kind of bond overnight.

The “we need a cofounder at all costs” mentality reminds me a bit of “cargo cult programming“, which in turn takes its names from the cargo cults of the South Pacific: http://en.wikipedia.org/wiki/Cargo_cult who would perform rituals mimicking actions taken by soldiers in the hopes that it would bring more planes and boats full of “cargo”.

As they say in Italian, “meglio soli che male accompagnati”, which means “better to be alone than in bad company”.

Alarming number of spam false positives in Gmail

This morning, I was checking into some unfinished work on my side project, LiberWriter and  noticed that I hadn’t seen several emails.  I checked in Gmail’s spam folder and found around 15 emails that had been dumped there, without anything obviously wrong with them. Several were from customers wondering why they hadn’t heard anything from us.

Worried, I started digging further.  Here’s what I have found so far:

  • An email from the people at work regarding payments to me.  This is important stuff!
  • An important personal email.
  • Emails from JIRA at work.  I get a bunch of these every day; and while they are not high priority, not seeing them is potentially damaging for me and the company.
  • Several emails to the erlang-questions mailing list that were fairly typical and topical.
  • Update: I just found a large (10’s) of emails from the Apache Software Foundation.
  • Update 2: Another personal email, not that important, but not at all spammy.
  • Update 3: Numerous emails from the tcl-core and Postgres ‘general’ mailing lists.

Several of these things were pretty easy to search on, but I am now very worried that there may be many others which I am missing.  I do receive a lot of spam, and in the past, the Gmail team has done an excellent job of filtering things, which is why I have happily recommended it.  Google employs a lot of extremely intelligent and capable people, so the thought that they’re hard at work on the spam problem has always been a good one.

I don’t know what has changed though, but seeing all those emails discarded in spam gave me a bad scare and has got me really worried.  I live in email… and if I can’t trust it, I’m in big trouble.

Ironically, I am a paying customer of Google as of a few days ago, in order to have extra storage space.

Update 4: I got in touch with someone from Google, who said that there was indeed a problem and they are working to push a fix soon.  I’m really glad to hear that!

I Default to Postgres

With a startup or new project of any kind, you’ll face many uncertainties.  One thing I always count on is the Postgres database.

I like to experiment with new things, and have, over the years.  I have a particular weakness for programming languages, but over the years, I’ve always come back to Postgres for data storage.  It’s solid, reliable, flexible, and I know I can take it in whichever direction I need.

I think you have to pick your battles – no one can do it all. You need to strike a careful balance between trying out new stuff, so that you don’t miss out on new and improved ways of doing things, and avoiding churn generated by blindly jumping to new technologies.

I’ve always been happy to try out new programming languages or frameworks, but I like to be able to count on my data, so I’ve always come back to Postgres.

You can do a lot with Postgres, and its capabilities slowly but surely expand, year after year.  One of the nice things is that new capabilities are pretty solid when they come out, rather than feeling like rough concepts waiting to be polished.

Postgres also has a great community, of thoughtful, helpful, talented people, which makes dealing with it that much more pleasant.

Erlang and unreliable resources

Here’s something you should be aware of if you’re using Erlang in a large system.  It’s a pity the core team hasn’t included something like it in the Erlang distribution itself, or event documented; because it’s something I think you’ll eventually need if you build a big enough system.

In a large system, you’ll have components that should be up and running – but at some point might not be.  For instance, a web site with a database.  If you write your Erlang code to just “let it crash” because the web server can’t connect to the database – just like you read in the books – then your whole site will fall over, web server and all.  At this point, you can’t even show your visitors a message explaining that “we’re experiencing problems, please try again later”, because the problem has propagated throughout the system, bringing it all down.  Another scenario might be a machine with a user interface and some fragile hardware.  The machine can’t do its job without the hardware, but if the hardware is broken, the software should not crash completely!  The user interface needs to stay active, letting users know that the machine is broken, and perhaps offering some diagnostics or offering some steps to try and correct the problem.  What these have in common is that there are some components of the system that should always try to be available even if they are not 100% functional, and there are other bits and pieces that may be necessary for the correct functionality of the system, but should not cause it to become entirely unavailable when something goes wrong.

The term for this is “circuit breaker”, because it breaks the chain that is part of a normal OTP system, where enough failures of a worker lead to a supervisor crashing, and its supervisors crashing in turn, after enough failures, and so on up the chain.  One high quality implementation is located here: https://github.com/jlouis/fuse – and politely points to some other, similar implementations.  I’m a bit frustrated that I found this only recently, because it’s a nice bit of code that strikes me as quite likely to be used somewhere withing any sufficiently large Erlang system.

As a footnote, I also created something similar, although it’s not battle tested (I just released it, actually!) and operates at a different level: https://github.com/davidw/hardcore – but it might be useful for some people.

When to use AWS – and when not to

Amazon’s “web services” – AWS for short – have made quite a splash in the developer community.  It’s pretty impressive how much horsepower you can harness in a very short amount of time.

However, it’s not the most economical solution, nor the simplest.  I think many startups are blinded to these facts by dreams of scaling up (“to infinity and beyond!”), when they should be concentrating on growing their business a bit at a time, and would be better off with something else, like Linode (yes folks, that’s a referral link!), which I use, or Digital Ocean, Hetzner, or one of the many others.  If you’re a Silicon Valley company fueled by venture capital and need to get really big really fast, this advice might not apply to you.  Otherwise, read on.

Here’s where I would consider using AWS from the beginning of a new project:

  • You absolutely positively know you’re going to have to scale horizontally from the start.
  • You need massive amounts of storage and want to stay close to Amazon’s S3.  Companies that deal in a lot of images or videos or something might need this.
  • You know you are going to need to start and stop services as demand rises and falls.  This sounds cool, but it’s not easy, so if you aren’t positive you need it, it’s premature optimization and you want to avoid it.  An example of this might be a company that gets requests to process large amounts of data: the request comes in, they start up an EC2 instance, farm out the job to it, and then shut it down when the results are ready.  This isn’t really something you can do in the time frame of an HTTP request though: the time to start a new instance is most likely measured in minutes.

There are probably other cases where some of Amazon’s services make sense, but where I come from, in the land of smaller, bootstrapped companies, worrying about “web scale” is very likely the wrong thing to be doing.  You want to get your infrastructure set up and running, get paying customers, figure out your market, understand which features they need, and so on.  If you’re making money by charging customers for your product, you can probably scale vertically by moving to larger instances, or do simple things like splitting the DB server from the web server, before you have to worry too much about scaling problems.

The actual dollar difference in what you spend is something to keep in mind if you’re watching your expenses carefully, but there’s also a complexity factor: AWS instances can be kind of complex to set up and run in terms of all the bits and pieces and understanding how the fit together.  Something more basic is just a regular old Linux server that you ssh into and run.  It’s pretty straightforward, meaning it’s less of a hassle.  As a small or bootstrapped company, or individual, you want to avoid spending too much time on complex stuff that is not likely to pay off for a long time.

Amazon is a Big Company and is not going to pay much attention to a small fish like you or I – you’ll likely get better support from someone who makes their money helping smaller companies and individuals.

You don’t want the absolute cheapest hosting service, because you do get what you pay for, and what you pay for is not for when things are running smoothly, but when things go bad.  I’ve always had good service and support from Linode, but I’m sure others are good too.  Ask around to see what people are saying.

So: if you’re a small company, don’t know how much you need to scale, or how you might need to scale, do not worry about it or try to “think ahead” too much: save the money and get something better suited to your needs.  When you do need to scale – if you reach that point – you can do so with a much better idea of exactly how to scale your own business in a way that makes sense.