Erlang and unreliable resources

Here’s something you should be aware of if you’re using Erlang in a large system.  It’s a pity the core team hasn’t included something like it in the Erlang distribution itself, or event documented; because it’s something I think you’ll eventually need if you build a big enough system.

In a large system, you’ll have components that should be up and running – but at some point might not be.  For instance, a web site with a database.  If you write your Erlang code to just “let it crash” because the web server can’t connect to the database – just like you read in the books – then your whole site will fall over, web server and all.  At this point, you can’t even show your visitors a message explaining that “we’re experiencing problems, please try again later”, because the problem has propagated throughout the system, bringing it all down.  Another scenario might be a machine with a user interface and some fragile hardware.  The machine can’t do its job without the hardware, but if the hardware is broken, the software should not crash completely!  The user interface needs to stay active, letting users know that the machine is broken, and perhaps offering some diagnostics or offering some steps to try and correct the problem.  What these have in common is that there are some components of the system that should always try to be available even if they are not 100% functional, and there are other bits and pieces that may be necessary for the correct functionality of the system, but should not cause it to become entirely unavailable when something goes wrong.

The term for this is “circuit breaker”, because it breaks the chain that is part of a normal OTP system, where enough failures of a worker lead to a supervisor crashing, and its supervisors crashing in turn, after enough failures, and so on up the chain.  One high quality implementation is located here: – and politely points to some other, similar implementations.  I’m a bit frustrated that I found this only recently, because it’s a nice bit of code that strikes me as quite likely to be used somewhere withing any sufficiently large Erlang system.

As a footnote, I also created something similar, although it’s not battle tested (I just released it, actually!) and operates at a different level: – but it might be useful for some people.

When to use AWS – and when not to

Amazon’s “web services” – AWS for short – have made quite a splash in the developer community.  It’s pretty impressive how much horsepower you can harness in a very short amount of time.

However, it’s not the most economical solution, nor the simplest.  I think many startups are blinded to these facts by dreams of scaling up (“to infinity and beyond!”), when they should be concentrating on growing their business a bit at a time, and would be better off with something else, like Linode (yes folks, that’s a referral link!), which I use, or Digital Ocean, Hetzner, or one of the many others.  If you’re a Silicon Valley company fueled by venture capital and need to get really big really fast, this advice might not apply to you.  Otherwise, read on.

Here’s where I would consider using AWS from the beginning of a new project:

  • You absolutely positively know you’re going to have to scale horizontally from the start.
  • You need massive amounts of storage and want to stay close to Amazon’s S3.  Companies that deal in a lot of images or videos or something might need this.
  • You know you are going to need to start and stop services as demand rises and falls.  This sounds cool, but it’s not easy, so if you aren’t positive you need it, it’s premature optimization and you want to avoid it.  An example of this might be a company that gets requests to process large amounts of data: the request comes in, they start up an EC2 instance, farm out the job to it, and then shut it down when the results are ready.  This isn’t really something you can do in the time frame of an HTTP request though: the time to start a new instance is most likely measured in minutes.

There are probably other cases where some of Amazon’s services make sense, but where I come from, in the land of smaller, bootstrapped companies, worrying about “web scale” is very likely the wrong thing to be doing.  You want to get your infrastructure set up and running, get paying customers, figure out your market, understand which features they need, and so on.  If you’re making money by charging customers for your product, you can probably scale vertically by moving to larger instances, or do simple things like splitting the DB server from the web server, before you have to worry too much about scaling problems.

The actual dollar difference in what you spend is something to keep in mind if you’re watching your expenses carefully, but there’s also a complexity factor: AWS instances can be kind of complex to set up and run in terms of all the bits and pieces and understanding how the fit together.  Something more basic is just a regular old Linux server that you ssh into and run.  It’s pretty straightforward, meaning it’s less of a hassle.  As a small or bootstrapped company, or individual, you want to avoid spending too much time on complex stuff that is not likely to pay off for a long time.

Amazon is a Big Company and is not going to pay much attention to a small fish like you or I – you’ll likely get better support from someone who makes their money helping smaller companies and individuals.

You don’t want the absolute cheapest hosting service, because you do get what you pay for, and what you pay for is not for when things are running smoothly, but when things go bad.  I’ve always had good service and support from Linode, but I’m sure others are good too.  Ask around to see what people are saying.

So: if you’re a small company, don’t know how much you need to scale, or how you might need to scale, do not worry about it or try to “think ahead” too much: save the money and get something better suited to your needs.  When you do need to scale – if you reach that point – you can do so with a much better idea of exactly how to scale your own business in a way that makes sense.

Testing is a Pareto thing

If you work on rocket engine software or some kind of life-critical software, you can ignore this post.

Still here? I wanted to sum up my thoughts on testing, as someone who works on fairly ordinary software from day to day.

I think testing is very beneficial, and software without tests is not much fun to work on, because you’re afraid of breaking it; and maybe not even knowing that you’ve broken it!

However, I also agree with DHH’s post on test driven development: that the rhetoric has gone too far in some circles.

I think it’s best viewed as a graph:

Initially, adding a little bit of testing gives you big benefits: first and foremost because now you have a test framework to hang new tests on, which is a great way to grow it.  As you discover bugs, add tests to ensure they never happen again!  This keeps the test suite growing organically, and gives the tests a sense of purpose, rather than just slogging through the whole codebase, adding tests for everything.

As you add more and more tests though, the returns on time/coverage/effort probably begin to flatten.

This is – very roughly – in line with the Pareto principle, also known as the 80/20 rule.  In the case of testing though, my feeling is that it’s often better to skip that difficult 20% part – unless you have a lot of extra time on your hands.

Like most everything else about software development, it’s driven by the economic necessities of the people and businesses involved, and therefore there are compromises involved.  Aiming for the maximum amount of benefits with the least amount of coverage is the sweet spot for testing.

People, Places and Jobs

Paul Graham has caused a stir with a recent essay that favors immigration of high-skilled programmers:

I don’t think the essay itself was one of Graham’s better efforts, but mostly because it is not bold enough in making the case for immigration. However, the reactions to it on places like Hacker News are dismaying on a variety of levels, both intellectual and moral.


Many of those railing against the idea of letting in more programmers (or workers in general) have a simplified view of the economy, in which there are a fixed number of jobs to go around.  10,272, say.  And if you let in more people, there will be more competition for those jobs, and therefore lower wages.  The people arguing against immigration are afraid that they won’t make as much money.

This is known as the lump of labor fallacy.  It is zero-sum thinking in which for one to gain, another must lose.  It is wrong, with the simplest example being that there are clearly more jobs and more money in the US economy now than 100 years ago.  Currently, there are approximately 320 million people in the United States.  100 years ago, there were roughly 100 million.  Clearly, something happened for the country to have gained both people, jobs, and money: the economy grew.  Graham points out that the economy and business are generally a positive-sum game in one of his earlier essays:

Suppose you own a beat-up old car. Instead of sitting on your butt next summer, you could spend the time restoring your car to pristine condition. In doing so you create wealth. The world is– and you specifically are– one pristine old car the richer. And not just in some metaphorical way. If you sell your car, you’ll get more for it.

In restoring your old car you have made yourself richer. You haven’t made anyone else poorer. So there is obviously not a fixed pie. And in fact, when you look at it this way, you wonder why anyone would think there was.

Immigration is also not a zero-sum game – everyone who comes to a country and works also spends money that then goes back into the local economy, in addition to paying taxes, and most likely contributing in other ways.  Many immigrants go on to create companies themselves, which in turn hire more people.  Companies founded by first or second generation immigrants include Google, Apple, ATT, eBay, IBM, and countless others.  People with the drive to get up and go someplace new tend to be, on average, a bit more entrepreneurial than those who stay put.

Most economic research points to the fact that immigration does not depress wages indicating that the fear of adverse economic effects of immigration is not well-founded.  The article is worth reading carefully, because it goes into the details and links to the actual research.

If you’re a programmer, you’ve probably heard of PHP, Ruby on Rails, Java, C#, C++ and Python, just to grab a few projects at random.  These were developed by people who are or were, at some point, immigrants (they did not necessarily create these things while in the US).  We would all clearly be worse off without these people and their contributions.  Or maybe they would have moved somewhere else.  The sun does not revolve around the United States, and given enough bad policy decisions, people can and will go elsewhere to found their companies.  Looking at open source software shows that Graham is on to something with regards to the distribution of talented programmers: many, if not most open source contributions and projects originated outside the US.  Making it difficult and unpleasant for those people to move to the US does no one any good.

Even more average programmers are a positive thing, though: not all of us are brilliant inventors, and there is plenty of room in the economy for those of us who work on all kinds of projects utilizing the tools created by the most exceptional among us.  Having more people available lets entrepreneurs try more new, inventive and creative businesses.  It’s beneficial for everyone.

If you gave the government the job of trying to determine who the “great” programmers are, and who are merely normal ones, how would that work out in practice?  Keep in mind the immigration folks don’t just deal with programmers, so theoretically they’d have to be responsible for judging “great” people in every field out there, which sounds like a very difficult task under the best of conditions, let alone for harried, overworked bureaucrats.

There are something like 11,000 babies born each and every day in the United States.  With 135,991 H1B visas granted in 2012, that’s only 12 days of babies. Granted, those children won’t compete for jobs right away, but eventually they will.  And yet very, very few people loudly complain about other people having children, because children, just like immigration, do not harm the economy.

Another idea I see often in these discussion is: “we should auction H1B visas to the companies willing to pay the highest wages”.  Auctions can be a sensible way of allocating resources, but with a purely arbitrary number of visa slots to auction off (is there any statistical or research-based reason why the numbers are what they are?), you’re not really improving things a great deal, due to the artificial limit (in programming, we often call numbers like these “magic numbers”, and not in a good way). Because some professions simply pay more than others, you also risk starving some professions of talented immigrants entirely.

A few people make the argument that “brain drain” harms the countries people immigrate from by depriving them of their best and brightest.  In this day and age, however, it’s easy for people to make some money, gain some experience, and then return to their countries of origin, not to mention the huge remittance economy that is a lifeline for many families “back home”.  Everyone is better off.  Some workers are simply not valuable at home, so moving allows them to be more productive and make more money.  Consider a fashion designer from a rural town in Oklahoma: forcing that person to stay where they were born is not going to create a fashion industry in that town, so letting that person go to Milan, Paris or New York is going to make everyone better off.

One sensible point people make in criticizing the H1B program is that it does not offer immigrants a lot of bargaining power.  If they were a bit less beholden to their employers, they could more confidently bargain for higher wages.  It is possible for someone with an H1B to change jobs, but you really need to “grasp the next vine before you let go of the previous one”, or you might get in trouble with your visa status.


People living in a country founded by immigrants trying to keep out other people hoping for a better life for themselves and their families is questionable, at best, and very ugly at its worst.  In reading the reactions to Graham’s essay, I would not ascribe blatant, outright racism to most people commenting, but there’s certainly an ugly undercurrent of “us” and “them”.

My own background certainly colors my views, and makes “us” and “them” far more nebulous concepts than they appear to be for some: I was born in the US, and live in Italy.  My wife is Italian, as are many of my friends.  I work at an Italian company.  I do not see these people as inherently different than people who happen to have been born in the same country I was.  They’re people, just like people anywhere.  If they wanted to go to the US, work, and obey the laws, I don’t see any reason for them not to.  I don’t see these people as a “them”: they’re my friends, family and colleagues.  I can’t support policies that would keep out my Italian friends in order to “protect” people who happen to be born in the same country as me, with whom I may have absolutely nothing in common other than a passport.

If someone can, just by going to a different place, make more money and live a better life, there’s a strong moral argument in favor of letting them do so.  To consign someone to a life less fulfilling than it could be because they happen to be born on the wrong side of a line seems unjust to my way of thinking.

This is especially true because none of us has any say in the matter of where we are born: it’s a genetic lottery that those of us born in wealthy countries were lucky enough to win.  It’s the least we can do to let people from elsewhere have a shot at a better life.  And those from other “1st world” countries?  What’s the harm in letting them in if they follow the rules?  If they don’t like it, they’ll probably go home where there’s more of a social safety net than in the US.

There’s also the matter of freedom: If people agree to abide by the laws of a country, and do so, why should we tell them ‘no’?  Believers in liberty should support others’ liberty to live and work where they wish.

Because of the positive-sum economics of immigration, it doesn’t cost us anything to let people in, and it might make the immigrants’ lives a lot better.  Best of all, this helping hand for our fellow human beings is not an act of charity: it’s through their own work that these people will be better off in addition to contributing to the local economy.  Everyone wins.

I recently read a book first published in the 1960ies, called “Business Adventures” (my review is here), with one quote from the boss of Xerox about unions and their relationship to minorities that struck me as relevant to immigration as well:

For example, we’ve tried, without much fanfare, to equip some Negro youths to take jobs beyond sweeping the floor and so on. The program required complete coöperation from our union, and we got it. But I’ve learned that, in subtle ways, the honeymoon is over. There’s an undercurrent of opposition. Here’s something started, then, that if it grows could confront us with a real business problem. If it becomes a few hundred objectors instead of a few dozen, things might even come to a strike, and in such a case I hope we and the union leadership would stand up and fight. But I don’t really know. You can’t honestly predict what you’d do in a case like that. I think I know what we’d do.”

At one time, it was considered acceptable to keep black people out of ‘good jobs’.  Attempting to keep foreigners out of jobs seems broadly similar to me: it’s immoral and based on bad economics.

If someone is willing to obey the law and work, they ought to be welcome in a country of immigrants.

Further Reading

The optimal number of immigrants

Homelands: The Case for Open Immigration

Perfect software versus economic reality

“Why, oh why, is the state of software so bad, so buggy?” reads a common lament on forums frequented by programmers.  As programmers, we’re a perfectionist bunch, and it bothers those of us who care about quality to see things work poorly.

Why indeed?  The answer is almost entirely dictated by economics.

Imagine that you’re a customer looking to have a custom web site / application built to help manage your small local bookstore.  It’d be bad if the system lost all your data, but other than that, minor bugs are tolerable, and your business just doesn’t make a ton of money in the first place.

You talk to two local shops that might be able to code up what you need:

  • One offers to write code they guarantee to be free of bugs.  It doesn’t have many features, but would probably get the job done.
  • The other company can get it done for a  bit less, with many more features than the first one.

You ask the first vendor about the extra features, and they say they might be able to get to them over time.

Since those extra features would be handy, you opt for the second vendor.  Once in a while, it does have an annoying bug or two, but the programmers do try hard, and manage to fix most of them in a few weeks, and none of them cause you any major problems.

The above anecdote is how much of the market for software works: everyone says they want high quality, defect free code, but how many people are willing to pay for it with either much higher prices, or a reduction in features?  It depends on what the software is used for, but by and large: not many.  They’d rather have the features, or pay less, and tolerate the odd bug here and there.

Everyone’s heard the old adage, “good, fast or cheap, pick any two”, which is very closely related to another triad: “lots of features, few bugs, cheap”.  You simply can’t develop something quickly or on the cheap, and have both lots of features and very few bugs.  You trade off features for time spent on ensuring there are no bugs, and since eliminating bugs is something that seems to asymptotically head for, but never quite achieves perfection, the more time you spend on that, the fewer features you have.

This suggests that if you want to work where people pay a lot of attention to quality, you should work where even small bugs cost a lot of money, or where they put people’s lives in danger.  Those are the sorts of environments where quality will count.


An Erlang Postgres Driver: Refurbishing Open Source

People often wonder how to get involved in open source.  It’s pretty easy, really: there’s more work to be done than people to do it!

It can also be surprisingly easy and pleasant.  Here’s how I dove in, and without spending a lot of time, did something to improve the situation of an Erlang Postgresql driver.

A few months ago, I realized that some code I was using at work, Will Glozer‘s Erlang driver for Postgres, was not really being updated much, and the Github ‘fork’ graphic looked like a bunch of messy spaghetti as everyone forked and patched and hacked, but never contributed anything “back” to any place central.

It was obvious there are some bright people doing good work, so the talent and will (no pun intended!) were not missing, but a central repository was.  Github is nice in a lot of ways, but one thing I think it fails to do in some ways is encourage community, although they have continued to make improvements over the years.  It is a great tool, but open source is about more than source code: it is fundamentally about people interacting in order to produce software.  Forks and pull requests and patches and all that are nice tools, but they can be dispersive, in that they often seem to be small individual efforts that are just thrown out, with little coordination, and little ongoing effort.  That’s not necessarily a bad thing: lots of people get started with little fixes here and there, but you still need a group of people at the heart of a project who consider and decide what to include, when, and why.  Overwhelmingly, the language for that is English, and the place is a mailing list or forum built around the idea of people talking, rather than patch management.

In a moment of pique, I decided that it was a shame to see a small but useful project like epgsql wandering and neglected, so here’s what I did about it.  Most of these steps are things you can do to improve some open source software you use and love but know is suffering from lack of attention.

  • Luckily, the project was in a pretty good place: it does most of what it needs thanks to Will’s original work, and a big rewrite by Anton Lebedevich some time ago.  That’s where the hard work was, and those guys deserve the real credit for the code.  That makes being a caretaker feasible.
  • I wrote an email to the main Erlang mailing list: – it was a bit curt, but generated some interest.  Postgresql is pretty popular among those in the Erlang crowd using relational databases.  They are both projects where robustness and correctness are important.
  • I created an “epgsql” organization on Github so that the code was not merely something that was “davidw”‘s, but had a neutral name.  Hopefully others will continue working on it in the same place if I am no longer able to.
  • I created a Google Group:!forum/epgsql – Google Groups are a great way to create something that’s easy to manage, and very accessible: people who prefer email can interact with it just like a mailing list, and those who are more casually interested, or prefer the web can use it as a forum.  They’re free, very easy to set up, and are reasonably flexible.  For larger projects, perhaps being beholden to Google is not such a great idea, but one of the key points with improving epgsql is that it not interfere with my work by taking much time.
  • I sought out people who had done some interesting work with epgsql and invited them to take part in the group.  There are now several of us with commit privileges.
  • I try and get people to spell out their patches in as much detail as possible, and respond quickly to get some dialog going.  I try and involve everyone in the group in what’s going on, as much as possible (we’re all busy), because I really want this to be a group project, not something that starts and stops with me.

So far so good, I’m happy with the improvements that have been submitted over the past few months.  Now, here’s the thing: I would not consider myself an expert with Erlang, and didn’t know much at all about Postgres’s binary communication protocol, but have managed to pick up enough of the latter that I basically get how things work.  So in order to do a good open source deed, you don’t need to be the best or most knowledgeable person out there, you just need to put in a little bit of time.


MicroConf and the Early Stages of a Bootstrapped Business

The conference for Bootstrappers par excellence, MicroConf, recently sold out their 2014 Las Vegas conference in under a half an hour.  That’s impressive, and having been to the European edition last year, I can say that it makes sense.  It’s the best conference that I’m aware of for people interested in, and working on small(ish) bootstrapped software businesses like my own, LiberWriter.  Demand obviously far exceeds supply, which leads to some obvious options: increase the price, or increase the supply.  Given that the entire thing is organized by just a few guys, who have other things to do with their lives, supply probably can’t go up that much, leaving price.  While I would understand completely if that happens, it’s also kind of unfortunate, because I think it would tend to crowd out those of us who do not have established businesses.

I got to thinking about this in relation to the various hurdles that bootstrapped companies must cross and where things like books and conferences relate to each step along the way.  A good conference is fun for me whatever I get out of it in terms of learning, but if I force myself to think about it in strictly financial terms, the proposition becomes a bit more difficult, given how good material tends to wind up on the internet, for free.  Said another way, the $$$ a conference costs are going to be a larger percentage of what a small business is taking in, and the given time constraints involved in many bootstrapped projects, the knowledge gained is less likely to be deployable in a short time frame.

On the other hand if you have the time and bandwidth to take what you’ve learned and really start going to town with it, then I’m sure MicroConf could pay for itself in a very short time frame.  That would naturally tend to push the conference in the direction of higher prices and an audience composed mostly of people doing their own companies full-time, at minimum.

I was relieved to see that Mike and Rob are keeping “the rest of us” in mind with the conference, though:

we’re trying not to make it unaffordable. We want new entrepreneurs, new founders to be able to come. We also want experienced folks. I find that mix is a positive thing for everybody. –

So once again a big thanks to them, for putting on a great conference, and helping to extend that to those who are still finding their way.

Bootstrapping Hurdles

Lately I’ve been thinking about the various hurdles that a bootstrapped web business has to leap.

When you’re creating something from nothing, there are various barriers you have to get over.  Let’s go through them: I think it’s a helpful way to think about bootstrapped companies in general, but specifically web or app based product companies.  The ‘holy grail’, after all is having a business that provides an income without requiring you to put in an eight hour day.  Now, that income can be a pretty normal income: it doesn’t need to be dot-com “set for the rest of your life” cash; just enough to let you go about doing more things while you have steady money coming in.

First Sale

There’s a lot leading up even to this point, but it’s a pretty clear dividing line.  Have you found a niche, done the marketing research, and then actually put together a product that someone out there on the internet was willing to buy?  Note that for a lot of people, the sensible way to approach this is marketing first: before you write code, attempt to determine if there is a market.  If there’s not… it’s a disappointment, but at least you can give up before you wasted a lot of time building a product that no one wants to buy.  That’s the theory, at least; in practice, things are never quite so cut and dried.  The famous Henry Ford quote is:  “If I had asked people what they wanted, they would have said faster horses.”  In any event though, you’ve found something you think is worthwhile, and someone has given you their money for it!  That’s a pretty big deal, and worth congratulating yourself for, if you’ve reached that point.  One thing I hate about a lot of business books is that they completely gloss over this part of things when you’re just getting started.  But making something that someone will spend money on is nothing to sneeze at.

Sustained Income

Once you’ve sold a few, that probably gets you some feedback that you can utilize your offering, and if things go well, you’ll make more sales, and have a more or less steady rate of cash coming in.  This may bounce around a lot initially, but you start to get an idea of where things are going financially, and how much it costs to provide them and how it impacts your time.  Getting that feedback is important, because most of us don’t get it right the first time.  Charging money for what you’re doing is conveniently a good way to get feedback, though: if you give something away for free, and people don’t like it, they’ll think “ah, oh well, forget it” and move on. If they’ve given you their money, and it’s not working, you are going to be hearing about it!  This is a tricky phase because a lot of us will likely be so euphoric that we actually managed to sell something to someone, that we’ll go implement the first thing anyone mentions.  That’s not necessarily a good idea – some customer suggestions are going to be winners, but not all of them.  Try sitting on them for a while to see what many people request – those are probably worth paying attention to.  The one-off requests can probably be ignored or dealt with on a case-by-case basis depending on how much out of your way you want to go to help out your customers.


Are you making enough to pay all your expenses, like hosting, virtual assistants, and things like that?  For the sake of simplicity, we won’t include your own time in the expenses yet.  If you work with software, it’s likely companies value your time fairly highly, so even for some relatively successful products, reasoning in strictly economic terms, you might be better off not striking out on your own, since at any given point you could conceivably be making more working for someone else.  Even someone like Patrick McKenzie who seems to have been very successful with the products he’s done so far could earn a lot more by only doing consulting for other people.  So ultimately your own time is an investment towards the kind of business you want to have.

Making a Living

Unless you’re young and living with your parents, or have some money set aside and few financial needs, you’re probably doing bootstrapping work on nights and weekends.  That’s where I’m at right now, and … it’s difficult.  The business pulls in money regularly, and ticks over some of the time without my involvement, but it’s not enough to live off of (and right now I have a day job I like a lot too).  I have way more ideas of things to do with it than free time to implement them.  The trick right now is finding things to do to increase money coming in without taking more of my time, and things to do that themselves don’t take a lot of time, since nights and weekends don’t offer a lot of scope for big, involved changes.   Like I said, I also have a contracting thing I really enjoy at the moment, so I’m in no hurry to move on from that, but having the option to be able to work from anywhere, and the freedom a developed business provides, is something I really believe in.

I’m sure there are other big hurdles after these, but if you get to making a living and you’ve built a business that to some degree “runs itself”, you have more free time to either start developing other products or sink some serious development time into your current project to improve it, attract more customers, or simply enjoy the extra freedom.

MicroConf Europe

I don't envy conference organizers these days – most of what's being said can be read the next day, for free, on line, at your own pace, from the comfort of your own home, and without spending a bundle of time and money to sleep in a far away hotel.

Competing with that is not easy, but the guys at MicroConf managed to.  I would sum up the weekend by saying that it was a "very high bandwidth experience".  Every day, from breakfast until I turned in, I was chatting with people or listening to speakers during the conference itself.  That's aproximately  16 hours of being "on", and by the time I got home to Padova, I was exhausted!  But at the end of the day, I felt like it was worth it being there in person, because of all the interaction with other people.  The speakers' talks all ended up on line, more or less, but all the chatting and discussion and getting to know everyone is the human element that is tough to replicate on line, and one of the most important reasons to attend a conference in person.  Prague is also a beautiful city – I wish I had had more time there to check it out.

Here are some highlights and notes, in no particular order:

  • Rob Walling talked about actual, concrete numbers when discussing his current project's revenues.  There's a ton of handwavy stuff out there on the internet, but real numbers are tough to beat.  What makes it especially nice is that they also felt "real": they're good numbers, no doubt about it, but not stratospheric, science fiction numbers that leave you feeling like "ok, whatever, but that's not the planet I live on".  They're numbers that make you think "maybe, if things go well, I could do that too".
  • The number of "I'm from X, but live in Y" people at the conference was high.  Irish but live in Spain, American but live in Japan.  Or maybe just noticeable because I'm in that category myself.  There were people attending from the US, Europe, Japan, South Africa, and even Australia.  Impressive!
  • Almost all of the speakers had very specific, concrete advice that I can and will apply to LiberWriter, time permitting.  I read, and have read, a lot of business books.  Most of them are kind of fluffy, truth be told, in that they've got one decent idea, and a lot of filler to turn what could have been a tight, ten-page article into a book.  This was quite different in that there were a whole lot of tips and tricks being thrown out.
  • Rob's wife Sherry gave a talk about life with an entrepreneur.  Having two kids and a wonderful wife myself, it's a point of view that I was very interested in hearing about.  Judging from the people I chatted with, this was not your typical "startup" conference with a bunch of 20-somethings with no family and no ties – a lot of the other people attending had kids to think about as they launch their ventures.  A question I asked of Rob was how much of a leap he took from consulting to working on his own products, with the answer being that he's actually pretty risk adverse.  No Silicon Valley story about betting the house and everything else on the company – apparently, revenues from the web sites and products were good enough that there wasn't even really a leap to make when he quit consulting.
  • The size of the conference was just right: enough people that I didn't quite manage to meet everyone, but not so many that it was overwhelming.  In downtime between talks, and during dinners, breakfasts, lunch and so on, the speakers were very available to chat with.
  • Patrick McKenzie seems to have stumbled into his life's calling as someone working at the border of software and marketing.  The amount of advice, anecdotes, and data that he was continually spinning off was incredible.  He comes across as being a down-to-earth, approachable, friendly person.
  • Part of the balancing act the organizers have to work with is where people are at: some people had an idea but no concrete business.  Some of us (me) make some money but not too much.  Others have viable businesses that they make enough to live off of, and then there are those who seem pretty much 'set'.  It's difficult to find people to speak to each audience without losing some of the others.
  • The thing I liked the most about a lot of what was discussed was that it seems realistic.  Few people at the conference were from Silicon Valley, and yet… they're successful!  I like hearing about success stories that work out really well for the people involved, but still feel like something attainable.  People should be looking to emulate the successful guys here, not looking at extreme outliers like Bill Gates or Mark Zuckerberg.
  • I'm used to tech conferences, where it's all about the technology.  There was very little actual tech talk at MicroConf – it seems like everyone knows their stuff and was interested in learning about marketing, sales, and so on.

However, since it was a business conference, I also have to put on my cold, hard accountant hat.  Will the conference pay for itself?  Only time will tell.  I learned a variety of interesting and useful things, many of which I think I can put into practice.  The problem is finding the time between consulting work and family, but that was a bottleneck before, too – I had, and have, more things to do than time.  Also, to be very direct about it, how much of what I learned could not have been learned by carefully reading accounts of the conference, slides, and other material published on the internet?  A lot of it.  I'm not sure I would have paid attention to all of it though, so the conference was definitely nice in that it exposed me to some talks and ideas that otherwise I might have brushed off before giving them a chance.  In terms of dollars and cents, I won't be able to say for a while whether it was a sensible investment or not.

Would I go again?  I'd like to – it was a lot of fun and the people were great.

Like I said, it's tough doing conferences because your competition is the internet!

The Software Millionaire Next Door

I've been reading "The Millionaire Next Door" and have so far found it to be a pleasant book with a good message: don't waste your money on silly things and appearance (fancy suits, fancy cars, expensive boats, etc…), save what you do earn consistently and constantly, invest wisely, and so on.   Wikipedia has a good summary:

One of the things I like about it is that it focuses on "ordinary" wealthy people, those with a million or more in the bank, but not the Warren Buffets or Bill Gates types that are extreme statistical outliers.  There are plenty of people in the US who have done well by themselves by slowly but surely putting together enough money to be financially independent, without, however, being in the spotlight.   As the book says, these are the kind of people who maybe own a local chain of businesses doing something fairly ordinary, but doing it well enough to succeed.  They may very well not live in a fancy house, nor drive an expensive car, or otherwise outwardly draw much attention to themselves.

The world of software does not revolve around "dressing for success" (you noticed?), but we do tend to focus on the "big winners".  Gates, Jobs, Zuckerberg, Larry & Sergey, Larry Ellison, and so on are the stars of the show.  Of course, the economics of software being what they are, instances of winner-take-all markets with one big fish and a lot of also-rans are not uncommon.   However, that is not the only story, and I think it'd be interesting to know more about those in our industry who have accumulated significant wealth, yet are not the guys with more money than they could possibly ever spend on things that aren't, say, country-sized chunks of real-estate.

I'm guessing they'd fall into these categories:

  • Highly paid workers who have consistently saved over the years.  There are examples in the aforementioned book about people with relatively low salaries who happened to be very frugal and invest well (and have had some luck in their investments too).  These people would probably tend to be older, as it takes a while to save up that kind of money, and since this industry is so young with so much turnover, I would not think there would be a lot of people out there like this, but who knows, maybe there are a bunch of IBMers with this kind of story.
  • Those who got in on the right IPO, like Google or Facebook or something like that.  These events not only generate billions for those at the top of the heap, but for the right person at the right place at the right time, can mean significant wealth even without being in the upper echelons of the company.  My suspicion is that this kind of IPO, where everyone cashes out, is not common enough to have a lot of people in this category, but who knows, maybe it adds up over the years.
  • Those who own or started software firms that do something that's not very visible, but nonetheless dominates some particular niche.  This is where I'd guess most of them would be, but I certainly have no data or even anecdotes to back this up.

It'd be very interesting to gather some actual data on this, although I'm not in a position to do so myself – I wouldn't even really know where to start. 

As I age, I think the third category has begun to seem appealing in many ways – I'm simply not cut out for the Big Company life, and I'm not interested in living in Silicon Valley and going "all in" on the latest startup – I already did that, and while it was fun and I don't regret it, it's not the kind of thing I'd want to do now that I'm married and have kids.   Incidentally, this more relaxed, under the radar approach is exactly what is expoused in one of my favorite books of the past few years, Start Small, Stay Small.

Edit : I finished reading the book and reviewed it here: