First Code Mesh in London, then Built Stuff in Vilnius: two great events during the first half of December 2013, where I spoke, and despite that, nothing bad happened. Indeed, meeting so many amazing and stimulating people didn't just take the discussions out of the box. It shredded the box, burnt the remains, and chucked the ashes into the toilet. There were so many amazing little stories that came out of these two trips. I want to capture some of them and share them with you.
FoldUnfold
Table of Contents
A Tale of Two Cities
Meetings are Global Mutable State
Growing by Learning
Sex in Title - Cheap Trick or Valid Hook?
Your Weakness is Your Public API
The Human Protocol Theory
Breaking the Code of Conduct
Women are Big Data, Men are Actors
HumanCoin — The Social Currency
Vilnius Got Talent
Conversation on a Plane
Summing Up
A Tale of Two Cities
On 3rd December, my birthday, I took the Eurostar to Erlang Solution's offices in London, and there gave a 4-hour workshop on ZeroMQ to a group of ten or so. We went from basics (what problems ZeroMQ solves) to highly technical detail (how the ZeroMQ CURVE security mechanism works). With a smart, inquiring audience, even half a day is too short.
That evening I spoke with the quietly brilliant Martin Thompson, who I'd just crossed paths with on Twitter. Over beer and food at the speakers' dinner, we talked about many things and then he spoke about Conway's Law, which states, roughly, that the things an organization produces will reflect how the organization communicates.
The next morning I listened to the also quietly brilliant Joe Armstrong explaining the principles of Erlang. He spoke of large scale systems and how to build them, and as my brain said "yes, yes, this is right, we know this," a vision came to me.
A large scale system is one that stretches across time and space. A large organization similarly, stretches across time and space. Conway's Law ties the two. How we organize defines how we think collectively, and thus what we make collectively. I've written often, how we organize is more important than who we are. Smart people in bad structures produce trash. Ordinary people, well organized, can produce precious gems.
I've also written that to build software, first build a community. In the ZeroMQ project that was always my prime focus. Community before code. Code emerges from our organization, i.e. our relationships.
My brain on fire, I went to the ZeroMQ talk I'd promised to give. "I've no slides," I said to the audience, which was true, because more and more I'm speaking without slides. "I promised to talk about ZeroMQ," I continued, "but I'm not going to. If you want to learn what that is, go buy this book," and I showed the O'Reilly book that the lovely Josette Garcia — at the O'Reilly table in the main hall — had loaned me.
We sold all the ZeroMQ books she'd brought. As for my talk, I delivered it rapidly and without questions, as we were running late. What I said basically was the following. I've expanded and reordered my actual presentation.
Meetings are Global Mutable State
I think Conway's Law is really about self-similarity. Software is essentially about people, and the larger-scale we make it, the more that is visible. One program reflects how one person thinks. A large-scale application reflects how many people think together.
An old joke, the epitome of Conway's Law, goes, "why did the compiler have four passes?" Answer: "because the team had five people. One had to be project leader."
Let me propose, I told the audience, that to build a successful large-scale distributed software system, you must build a successful large-scale distributed organization. Sounds plausible, right? Let's work through what this means. Clearly there are degrees of success. Degrees of cost. The more efficient, the cheaper. Hence our successful open source communities like ZeroMQ show the state of the art.
To start with, I'll classify successful large systems into two distinct and mutually incompatible types. First, the hierarchy. Second, the network. A large-scale hierarchical system (LSHS) is defined by command and control from top to bottom. It is largely synchronous, homogeneous, expensive, inefficient, stupid in the technical sense (it makes large mistakes based on poor understanding of reality). A large-scale network system (LSNS) is defined by flow of events in multiple directions. It is largely asynchronous, heterogeneous, efficient, and smart in that it makes accurate decisions rapidly.
Conway's Law says, a LSHS organization like Microsoft will inevitably build LSHS software like Windows. The mapping is not so much an analogy as a law of self-similarity. "If you're using ZeroMQ in a hierarchical organization, you're wasting your time," I told the slightly shocked audience. Yet it seems true, and a good explanation for why so many large organizations cannot get a handle on ZeroMQ and technologies like it.
There are many expressions of a LSNS. There's software (an application). There's people (an organization). There's buildings (a city). There's animals and plants (a jungle). Let me compare the LSNS of software and of people.
The components of a LSNS develop independently in space and time. That means it must be, and here I respectfully divert from Joe Armstrong's vision, free of language or platform constraints. It must be naturally and successfully heterogeneous. To force SAP across "the Enterprise" is like standardizing on German. Not only does that reduce diversity and competition of thought, it is expense at no profit. Languages do not define interoperability. So, what does?
When you develop components or teams independently across space and time, how do you connect those teams? The answer is two-fold. First, you define verifiable contracts for every single necessary interaction. Second, you implement those contracts in a lazy, event-driven, asynchronous, message-passing way. It's how we built the Internet.
Again, these are not hard requirements. You can do things stupidly and inefficiently, as the LSHS proves, if you are willing to pay the price. Unfortunately, structural inefficiency isn't like driving the long way. There's no scenery to admire, just long-lasting low-level pain and stress.
The LSNS is based on heterogeneous message passing of contract-based events. In software, I can't see a better way than ZeroMQ, which is precisely focused on this — and hence the relevance of my material to the talk's title, "Building Large-Scale Systems with ZeroMQ." For organizations, it's harder to generalize. For software development at least, GitHub seems to work as a message-passing system, with pull requests as events, and something like our C4.1 process as contract.
An LSNS spans space and time and is essentially concurrent. You can turn that around and prove, with science, that an LSNS is the only way to build a 100% scalable concurrent system. This is Amdahl's Law, another of Martin's Thompson's gifts from the previous day. In a software system, concurrency means exploiting threads, processes, cores, boxes without any penalty. In an organization, that scalability applies to brains.
A LSHS tries to scale by scaling its "global mutable state," in other words data that everyone agrees on. It is a natural pattern for one, or two people. I don't send myself emails. As you add more people, however, the cost of changing shared state increases horribly. The only way to scale is to in fact ban the concept of global mutable state, and switch to a top-down hierarchical state. That leads to amazingly slow and inaccurate decision making, where essential knowledge can take so long to filter back up to the top that the whole LSHS is dead before a necessary decision can be taken. Bye-bye Nokia!
In software, we have synchronization points called locks or mutexes, leading to complex, fragile designs that break under any kind of stress. In business, we have meetings and conference calls, which are absolutely analogous. To build a LSNS you eliminate all synchronization points.
This is, parenthetically, why improving the meeting format is as pointless as writing a faster lock mechanism. Don't improve meetings. Design them away. For sure, meetups like the conference are fun and useful to exchange ideas, yet these are not state sharing mechanisms, they are high-speed data distribution. The notable thing about the best meetups is how little we agree to do together afterwards, except meet again and drink more beer.
So I've covered concurrency, events and messages, contracts, time and space. There are a few more interesting properties. A LSNS is location and grain independent. That means a task could be done by a thread, a process, or an entire cloud. Or, a person, a team, or another entire LSNS. It is, and must be, invisible to anyone and anything on the other side of the contract. Do you know what is going on behind the page at google.com?
Growing by Learning
I hope that gave you some material to think about during your next conference call or daily stand-up. Agile is a kind of a LSNS-wannabe process for the LSHS. A real LSNS process doesn't have meetings.
The LSHS process consists of building the organization by hiring smart people, putting them into teams, assigning them budget and tasks, and coordinating the lot with layers of management.
The LSNS process consists of growing by discovering valuable problems and organizing around them. The scale of the problem defines the scale of the system that handles it. If there is a documented process, it is one of "growth by learning." Thus a state-of-the-art LSNS like the ZeroMQ community operates by discovering and solving interesting problems, and growing around those.
When we forked the ZeroMQ project in 2009, it was precisely over the desire of some people to build a LSHS, and others to build a LSNS. You can look at any system — human or software — and see right away whether it will survive in the long term or not. Is it designed to grow into a LSNS by learning, or is it built to execute a mission, as a LSHS? The latter is doomed, particularly as the knowledge of how to build a successful competing LSNS grows. Blackberry could survive when its only competitors were other LSHS's.
In software, a LSNS doesn't just grow by learning, it actually builds its software by learning. Once you recognize that learning is the key mechanism, you can optimize for that. You need, for example, broad inputs with voting mechanisms to detect bad data and mistaken assumptions. That means diversity, not in the nastily fake US sense, but in the true sense. A LSNS depends on people of all backgrounds, ages, and educations, as well as genders. To try to standardize on brains ("they should all be good programmers") is detrimental to the outcome just as much as trying to standardize on technology ("they should all use F#"), because it will sabotage the learning process.
If we operate a LSNS by learning, the latency is critical. I've written before of "change latency," as the time it takes for an organization to respond to a new market challenge. Years, months, weeks, days, or hours? Clearly there's a difference. In software, we get lower message latency by cutting large messages ("here are the closing stock prices for the NYSE") into many smaller messages ("BTX is now at USD 639.12"). In an organization we get lower latency by slicing problems and tasks into smaller pieces.
My goal with the ZeroMQ process was to bring the change latency down to minutes where possible. That meant merging patches to master, without code review (a form of global lock!), and fixing bad patches with further patches. If you do code reviews, you must make them non-blocking, thus optional. Even waiting for a complex regression test will add to change latency, and often I'll merge other peoples' changes without waiting for Travis CI to confirm that they were "OK."
Sex in Title - Cheap Trick or Valid Hook?
At Code Mesh I met many interesting people, not least my friend Garrett Smith, and the folks from Erlang Solutions who were organizing the event. I learned some useful if not very polite Romanian words and got a chance to practice my favorite language, Polish.
I got a tweet from Greg Young, the guy behind Build Stuff, which was running the next week (from 10 December) in Vilnius, Lithuania. "We're in the hotel bar, come find us!" he said. I was talking with Amanda Laucher, who I'd just discovered (much like one might discover a new force of nature), so we both headed off to the bar. It slowly dawned on me that Amanda was also going to Vilnius. "I'm also going to be there!" I gushed, excited. "Yes, I know," she replied. "Wow! What a coincidence," I said. She gave me a slightly quizzical look, as if I was being deliberately stupid. Presumably there are people who can accurately map the flows of conference speakers from one event to the next. When we don't see the overall picture, little details can look like astonishing coincidences.
I'd never met Greg before. He had pinged me after Devops Days in London, asking if I'd do a keynote for him. This would be the opening slot on Monday morning. Nice! I sent him the abstract of my Culture and Empire book, and he said that was great, which surprised me. After NDC Oslo, where two people came to my talk, I'm a bit cynical about the whole conference circuit.
The one thing I'd learned from NDC apart from "when you go to Norway, take your beer with you" was, the talk of the title matters. In fact you can call it anything, if you can excite the audience no matter who they are. Since I don't talk on niche topics, after NDC I'd decided to always put sex in the title. Cheap trick or valid hook, it doesn't matter if no-one comes to see you. Since NDC I've also stopped using slides entirely. I made really nice slides for NDC Oslo, as in some of my best ever. Eff that. A proper performer doesn't use PowerPoint.
Meeting Greg in person was like being hit in the face by a fresh breeze. For some reason I started venting about NDC and he said, "yes, last year we called our conference 'We Actually Build Stuff', exactly in response to events like that." He then asked me what I'd disliked specifically. Oh my god, I said, that opening keynote. A guy called Joel Spolsky who claimed to be talking about community — one of my favorite topics — but instead talked about "gameification," a cheap and nasty trick where you give people worthless points for being your temporary workforce.
I believe, and hold true, that if a community about ZeroMQ is all about creating value together, and sharing the profits widely, the communities on sites like StackOverflow are about creating value for one firm by getting masses of people to work for free.
Not only is the whole concept a distasteful form of pig lipstick — like Agile, a smear of color on an otherwise unkissable beast — the keynote was delivered in a flat, boring monotone with slides. What a waste of a thousand people's time and attention! And Greg nodded and turned his laptop to me, showing one of his articles, saying exactly the same.
I am a bit odd. I will rarely if ever answer a question on StackOverflow (SO). If I do its because I didn't have any other easy way of answering the question. This is entirely by design.
OMG #awesome. I'm already feeling the love in me! A community is so defined by the character of its founder. Meeting Greg, I realized I was going to enjoy Vilnius. I didn't yet realize quite how rich that week would be.
Your Weakness is Your Public API
There's no real sign, as you fly into Vilnius airport, what awaits you. I've sometimes said that the quality of life in any country is directly proportional with the cost/benefit ratio of its Internet. One could also say, "beer." Certainly it started well, as I arrived in the hotel (thank you, Greg, for putting all the speakers in one hotel), and stumbled into Rob Ashton, Sebastien Lambla, and some others at the small and friendly bar.
If you know me at all, you know I enjoy drinking, partly as a narcotic that helps switch off an over-active brain, partly as a collective lubricant to help quieter people open up. I dislike the side-effects of alcohol, in myself and other people, such as loss of executive control, emotionality, and plain toxicity. My best way to manage these is to drink only beer, eat before and during drinking, and have a very poor memory of what was said or done. Also, before sleeping, drink a lot of water.
For some reason, people find me a good listener (though it seems to me that I never shut up), and I got this statement from more than one (sober) person: "I want to fix my flaws and become a better person." It's an implicit request for "points for improvement." However, it seems to me, after understanding the LSHS vs. LSNS dichotomy, that this search for personal excellence is based on the perverse lie of self-perfectiblity.
For sure, there are many people who find themselves isolated and alone, and blame it on their weaknesses or flaws. Their self-criticism makes them hesitant in company, and unwilling to take social risks. Randomly, because I'd been thinking of systems of people, it struck me that the reality is just opposed.
"Perfection precludes participation," I wrote years ago when designing community processes. Make a perfect piece of software, and no-one will join you to improve it. It's obvious why: we're drawn to areas where we can add value. That means areas of weakness or incompleteness. Ah! If this applies to software systems, so it applies to human systems.
Conway's Law! It's not our strengths that make us attractive to others, it's our weaknesses and vulnerabilities. In some cases that can be quite nasty, even predatory. In the general case, however, it's healthy. If you are a good illustrator and I am a good writer, we can make a better book together. However if I'm also a good illustrator, you can't work with me.
I developed this sixty-second instant coaching session: embrace your weaknesses and express them happily to others, for those are your public APIs. That is how the right people will find you and plug into to you, building relationships that make both of you happy. Don't fix what makes you interesting! It has to work both ways, I guess, you have to both accept weaknesses in yourself, and in others. We're so indoctrinated, from youth, to hide our flaws and hate them in others, that this may be hard.
Let me take that one step further. Advertising our strengths is frankly dishonest. Instead, on dating sites and CVs, we should advertise our weaknesses and needs. My name is Pieter, and I'm terrible at birthdays and names. I'm arrogant and opinionated and all my life I've obsessively developed theories of the world. I am a walking Code of Conduct violation, and perversely proud of all that. The best thing I can say about myself is that I accept all my flaws without question.
The Human Protocol Theory
One of my theories, dating back some years, is the Theory of Human Protocols. It states that humans speak native protocols which underly all higher languages. Further, that these protocols are constant across all cultures, thus biological, and heavily biased by gender and by age.
The study of communications models is one of my passions, mostly because it produces some of the best comedy of errors. There's a pattern I noticed, which annoyed me at first because I could not understand it, nor participate, and which later started to make sense, and appear more widely, as I decoded it. This pattern is very specific: two women exchanging information. Now you may respond that any two people talking are "exchanging information," and ask why I'm picking out women.
If you're a fan of wishy-washy generalizations that hide all detail, or easily offended by reverse engineering of the human mind, skip this section. If you enjoy looking beneath the surface to decode deeper reality, continue reading.
Here are some of the observations I've made, repeatedly, both first hand and by asking hundreds if not thousands of people over the years. You tell me if I'm wrong (you can mentally insert the words "generally", "often", or "mostly" before any absolute statement if it makes you happier):
When two women, who know each other, meet after some separation, they spend a measurable time talking to each other. The time they spend is proportional to the time of separation and perhaps their degree of closeness.
When two men similarly meet, they perform a contact ritual — handshake, high-five, hug, embarrassed cheek peck, back slap — and a brief exchange of "everything good?" with the only acceptable response, "yeah, great, and you?" The two men will then proceed rapidly to food, drink, bitching about their women (or lack thereof), and plans of action.
While women — in general — seem more comfortable with a two-way dialog heavy in acknowledgments and confirmations, men seem more comfortable with a one-way broadcast with lightweight confirmations. "So we invade Greece. You good with that?" "Yeah, sure, sounds fine."
Women, meeting and talking, are collecting information about people and events. When a man meets another man, and then relates the meeting to his female partner, she will often ask something like, "Did you ask about X," to which the man will reply, "No," and she will then say, "Did you ask about Y then?" and he will again reply "No," leaving both of them puzzled and mildly irritated.
If a man interrupts — politely or rudely — two women who are in the middle of such a talk, they will both suspend the session, take care of whatever the man is talking about, and then resume their session.
When two women have finished such a session, they change their focus from each other to other people. I've observed three-way sessions but they are rare, and more often a third woman will drift away and focus on other people.
That's the basic data and it's consistent across culture and space and time, as far as I can see. It's not a complete data set, by far. There are clearly other cases, such as women who are more comfortable talking to men than to other women, and men who are happier talking to women than to men.
The best theory I can find to explain this is that we carry a gender-oriented set of tools, which I classify as "protocols," that interconnect with other people in specific ways. These protocols are biological, not learned. That's clear because culture has no impact on these patterns. As with all our mental tools, we can assume that they sharpen with use, and develop during key phases of our lives.
Let me caricature one of the main male protocols. It consists of one guy, usually an older male, standing up and talking loudly to others. The others listen while he tells a story. He's making promises and plans, trying to convince the group to follow him. Some of the group interrupt, asking questions to test him. So long as his story is interesting, and can't be falsified, heads nod, and if he can make a convincing argument why the whole group will profit, there's a good chance they will follow him.
"Let's all go for pizza! It's two-for-one Tuesday and I know the owner so he'll give us free drinks."
Notice the pattern: it's a one-way flow of promises and plausible data. The whole package hangs together. The speaker gets his slot, makes his case, and the audience either accepts, or rejects the whole package. There's no need for a preexisting relationship, and the outcome of a successful pizza expedition is some hierarchical karma.
Now to sketch the main female protocol. There is a relationship, one-to-one, built up over time through the trade of information about people and events. The more valuable the information a peer provides, the more valuable the peer. A peer that gives only worthless data, or wrong data, is considered low value. As individuals collect data from others, or by observation, they sort it, quantify it, and trade it on.
For the male brain, or the female brain that doesn't speak this protocol well, the data may look like banal chat, gossip even, yet it's the act of trading that creates the relationship as much as the information being traded. (I'll open this particular box in the next story.)
Trading is replication of sorts, a resynchronization that takes place after a period of disconnection. There's an opening handshake, exchange of credentials perhaps, and some mood setting. Shall we sit here, or there. Perhaps somewhere quieter. According to the time apart, and presumably many other factors, trading goes slower or faster, and covers more or less territory. It ends with a closing handshake, and then it's over.
"Come here, I'll tell you what happened at yesterday's party. Yes, that's right. How did you know? No, both of them. Yes, both! Well, it's a long story. How's your father doing?"
What I enjoy about the theory of human protocols is that it explains many uncomfortable and strange social situations. I'll explore those elsewhere — there's a book I want to write on this. Understanding the protocols lets us understand many pain points, like why so many teenage girls end up in multi-year arguments with their fathers, and potentially fix them.
Breaking the Code of Conduct
Let me take one of those pain points, and propose a solution, based on the theory. This pain point is the Code of Conduct, the Diversity issue, the Wage Gap, Glass Ceiling, what have you. It is the difficulty we have, even in our enlightened digital societies, to value people for their qualities rather than their gender.
It seems perverse that in our digital society, where we are freer than ever to work where we like, with whom we like, on what we like, that our communities are more gender biased than ever. The most egalitarian societies, in the gender sense, are totalitarian states. This is surely a sign that we're doing something profoundly wrong when it comes to large-scale organization. The long-standing accusation of sexism may be accurate data yet it's done nothing to improve things, and instead widens the disagreeable, and enduring, split between the genders.
Rather than seeking the answer in labeling and regulation, let's look at the question in terms of individuals and the protocols they speak best and are happiest with. With best fit also comes value. If the only sport is football, the natural swimmers are always going to be at a disadvantage.
Here's an observation I've often made that no-one has managed to explain: in my open source communities, we see very few women, if any. I'm talking specifically about ZeroMQ-style projects which are entirely self-organizing, with little or no enterprise contributors, very few meetups, little forward planning. I've worked with people for years who I never met and did not know what they looked like. The AUTHORS list for ZeroMQ has a hundred or so people, and one woman (Tamara Kustarova) who was an employee in the original team.
How to explain this? It cannot be sexism, neither overt nor discrete. We are an extraordinarily happy and open community. You'll search in vain for any discussion on our email lists that rejects, excludes, or offends. Yet women do not join. If they come as ZeroMQ users, they do not stay as active contributors. You may say, men are more willing to take the risk and public step of contributing to open source. Yet most of our contributors are simply people using ZeroMQ for work and submitting patches they wanted to make.
The answer is, I'll claim, that we're a community of footballers, not swimmers. Only a certain amount of force can get footballers and swimmers to play in the same team, and the results are usually miserable. Does this mean football is better than swimming? Of course not. Does it mean swimmers are less valuable than footballers? No way. Do footballers hate the swimmers in some sense? Nopes.
Male-dominated organizations, I believe, are based on male protocols; designed by men for men; comfortable and effective for the male mind. Women come along, see patterns of communication they feel awkward with, and leave. The better it works for one gender, the more it excludes the other.
I've worked as the only male in an all-female environment, and it is strange. It is as if I'm the stupidest one in the room: tolerated and liked, yet unable to catch what is going on. It's much like being the only English speaker in a roomful of people speaking French. Even learning French and speaking it well, I'll always be a second-class French speaker.
Much the same applies to the school. I'd never have endorsed single-gender classes before, yet understanding that girls and boys learn differently, and that existing classrooms are largely designed for boys, makes me think my daughter should study in female-biased classes. I see our work as, ideally, an endless learning process.
Women will never reach parity in the workplace by emulating men. My proposal for solving the gender issue in organizations is thus to create gender-biased teams which define their preferred ways of working together, and then connect using the LSNS model of organization. Each team is free to develop its own value and share that among its participants. Rather than demanding a fair and equal wage, everyone should be free to earn it.
A LSNS works by measurable contracts. This lets us exclude bias by design. Either a team can meet the contract, or it cannot. How the team organizes, let alone their gender, age, or skin color, is surely irrelevant, and it's indiscreet and rude to ask.
Women are Big Data, Men are Actors
One outcome of the Theory of Human Protocols is that we can see the global network of female brains as a form of massively parallel Big Data. Interestingly enough, it has all the properties of a perfect LSNS: fully concurrent, decentralized, real-time, event-driven, message-based, lock-free, etc.
A communications product like ZeroMQ actually implements, as far as I can tell, male protocols. These are simple protocols that work best when nodes have no state, and are best at doing one task at a time. Just like men, perhaps.
I'd like to actually document the main female protocols, and implement them in software, and use that to create a LSNS that solves the big data problem. I'd expect that alone it would not work, it would have to bridge to male protocols like publish-subscribe and push-pull to get any kind of accuracy.
HumanCoin — The Social Currency
Pavlo Baron, pleasant and easy-going, paid for our taxi ride to the conference venue, and I bought him a tea and a bar of chocolate. "Why did you buy that for me?" he asked. I searched for an answer, and then took BitCoin as an analogy.
Before there was money, I said, we traded. It wasn't a barter system — one cow for fifty chickens — that's a silly notion. It was an asynchronous exchange of gifts and favors over time, the only way to invest an excess. Today my family is hungry, and you have too many eggs. Tomorrow, your chickens died, but my cow has lots of milk.
Any system has to be robust against cheats, however, so we track these givings and receivings, and maintain a mental balance for every different individual we know. This tally isn't just a single figure, rather, it is a list of transactions back in time. The value of a gift may age — the chicken you gave me last year is less important than the chicken we shared for lunch today. Or it may not — you saved my life ten years ago, and I still owe you.
Arguably, the very essence of a relationship between two people is this two-way accounting. A deep relationship has many transactions — the smaller and more subtle the better — and takes significant mental effort to calculate. That constant recalculation of the transaction chain generates our social currency. "He's a great guy!" really means, "There was a mutually profitable trade of things that seem very valuable to me."
The trade isn't just in chickens and chocolates, indeed the really interesting asset is knowledge. If I tell you "buy BitCoin," and you act on it, that can be worth much more than a cow. Men are OK at trading knowledge. Women however, are the absolute experts in this (as per my previous story). I'd say, my brain has a single-core Atom that does most things in software, while most women I know have 24 high speed GPUs with hardcoded silicon.
Perhaps that extra space men don't dedicate to processing social knowledge gets remapped to other kinds of memory, which is why I can design a thousand-node image processing cloud, yet cannot easily remember the birthdays of my own children.
I'd enjoy seeing social network applications that implement HumanCoin or at least parts of it. Twitter, please track the people I've retweeted the most, and show their tweets in a larger font.
Once we see HumanCoin as the natural social currency, we understand for instance the significance of Dunbar's Number, which defines the limit of our social networks. That limit is imposed by the capacity of our brains to process transactions. It's an average, or perhaps a maximum. Many people cannot maintain more than a few or a dozen relationships. Some people can't maintain any at all, which shows as a severe personality disorder.
And finally we see the role of, and need for, liquid currency. Irrespective of what rulers have used it for, it is essentially an answer of how to trade with strangers, while remaining asynchronous and scalable. Cows for chickens is impossibly synchronous (think for a second about the logistics). Selling a cow for money, then buying chickens at a later date is simple.
Money has the additional benefit (or evil) of wiping the social ledger. If a someone gives me a lift and I give them money, there's no further work required for either of us. If I don't pay anything, I'm in debt, and we've established a relationship. Perhaps this is why a gift of money is so delicate. It in fact only works reliably with children, who we know are incapable of mining HumanCoins anyhow.
Vilnius Got Talent
Build Stuff was a great event set in a pool of people that amplified it to "astonishing." I met so many interesting and interested people that my poor memory lost track. Amanda, Clemens, Dan, Freek, Jake, James, Joe, Johannes, John, Jonas, João, Jérémie, Karina, Kristoffer, Mark, Martin, Morten, Omer, Patrick, Paul, Pavlo, Rob, Sam, Sebastien, Seif, Tomas, Vitaly, to cite a few first names. I already miss you all!
One night at perhaps 2am, Sam Aaron and I walked around the old town, led by a young and beautiful Russian woman called Zhanna. She took us, dancing and talking incessantly, along the old streets, walls, and fortifications. We were skeptical, and Sam was cold, yet we followed. It had been raining lightly on an ice-cold ground, so the streets and pavements were covered in a thin sheet of crackling ice. The night was dry and crisp and utterly empty of people, an ineffable pastiche of old and new under the yellow sodium lighting. The next day the ice had melted, and we shook the feeling off, like a dream.
I've organized a few conferences and attended many, and want to take a moment to imagine the perfect event. Code Mesh was great, and Build Stuff came close to perfect on many levels (beyond the superficial), and left many people in a similar and precious state of dream shock, I think. It's only when we escape the imaginary confines of our "real life" that our minds open to new realities.
Let me list the actual problems I've observed in many events, before coming to my proposal:
Audience and speakers dispersing after the talks. This typically happens when there are many locals, in a large city - London, Berlin, etc. The goal of the day should be to stimulate and drive discussions during the dark hours.
Boring presenters who talk for too long, sending the audience to sleep. My belief is: if you can read it on the Internet, it's not worth presenting. Speakers should be presenters, either by force, selection, or gentle education. Talks should be limited in time: ten minutes is good, 30 is already a small workshop. It's far better for one person to give three 10-minute talks than a single half-hour one.
Weak speaking structure. Every speaker should be introduced by a host who warms the audience up before handing the stage over; every talk should be brutally timed; every speaker should hand the stage back to the host afterwards; every talk should have space for questions and discussions.
Death by slide show. Really, we have to learn to get off slides. I don't care how beautiful the photos or elegant the layout. The goal of the speaker should be to tell a story, engage the audience, and make them think. I'd think it's worth having a pre-conference event just to practice and learn to to do this. The argument that "it's great for people who weren't there" is useless. My perfect event is only for those who attend. The rest can read a blog post.
Predetermined program. We need surprise to stay awake, and the audience should really not know what the next talk will be. Perhaps they know the speaker. Perhaps they have some idea of the general area. The talk itself should be a surprise. That means killing the fixed program and moving to a different format. I'm fully in favor of inviting specific speakers. "Come to my event and surprise the audience," would be a great brief.
Bursts of traffic for catering. It's a poor sign when there are queues for food and coffee. We talk of building asynchronous large-scale systems in conferences, and then create massive deadlocks for refreshments. Drinks and food should be delivered constantly to participants, morphing from caffeine to alcohol by evening.
Participants looking for rooms. Literally 5-10 minutes per hour is spent just running around rooms. Again, not a good sign. It doesn't scale, creates delays, and leads to situations like large rooms half empty, and small rooms overflowing.
365 days of change latency. If the audience makes a realization — we need more talks on crowdfunding — it takes a full year to respond. This is not good. A good event should react like a LSNS, in minutes or at most hours, to new demands. Indeed the whole conference should be built on the principle of "do by learning".
OK, let me mix these together into something I call "What The Conf?" The goal is to surprise, entertain, and educate the participants, to break boundaries of space, time, role, and normality. The conference should solve the problem of "I need a few good days of insanely productive chatter with interesting people," and it should do it cheaply, and minimally.
The conference takes place in a big club in a small cheap distant city. There must be cheap and plentiful beer, decent food, and a majority of visitors over locals among the participants, so that people don't all disappear home after hours. It should ideally be in a small city that's walkable, so groups spread out and coalesce easily. For instance, the Comedy Club in Vilnius seems a perfect fit.
The event runs for two to three days. It's cheap though not free: the event must cover its costs. There are tracks that run in serial from a single main podium. The audience basically finds comfortable chairs and tables, and sits in groups. They don't have to move except to refill their drinks or grab some food from the buffet. The speakers do all the running.
Anyone can propose a track, if they can recruit a handful of speakers. The presentation of corporate languages or products is reserved for the fixed "Hail Corporate" track that runs right after lunch. The "Bullshit" track, with such topics as, "Agile, the Way Forward for Linux? How Daily Standups Will Revolutionize Kernel Development," runs concurrently, with speakers randomly intermixed.
By evening, we enter the "Comedy" track, where speakers talk about catastrophes. We need to speak about our failures as much, or more, as our successes. The Comedy track coincides with a smooth move from herbal teas and water to light beer.
Speakers get 10 minutes and have to speak without slides. If they don't know their stuff well enough to talk about it, they perhaps shouldn't be there. The audience will be very tolerant of speakers, even shy ones. There is a lot of applause, heckling, laughing, and shouting. There is a time keeper who raises a green card at 5 minutes, an orange card at 8 minutes, and a red card at 10 minutes.
There is also a panel of judges who may come with points for improvement handed on slips of paper to the speaker afterwards. Every speaker should feel they learned something useful, when they step down. It might be, talk slower, or use more body language, or don't say "Uhm" so often. The judges are the hosts and take turns introducing speakers and guiding them off the stage afterwards.
The audience gets slips of paper and if there is a speaker they like a lot, they note those names and pass to the judges. The judges can randomly invite good speakers back for more.
Behind the speaker is a screen that shows the #whattheconf IRC chat channel (white text on black background, large font to be readable from the back of the room). On this channel, all participants can comment on the talk, post URLs, and so on. Questions are all asked via IRC.
There's a live video stream of the whole event, so this is how the public interacts. The judges and audience see the screen; the speaker doesn't, unless they turn. The live video may be cut with subtitles, off-stage discussions, adverts, whatever that technology allows. That's entertainment!
The day runs without breaks. People can come and go at will, yet there are no breaks. Without slides, there are no technical issues. As the host guides the previous speaker off the stage, "A huge round of applause for Jeremy's inspiring tale of learning Vim," the host also introduces the next speaker, "and let's welcome Pavel, from Torun, who will explain how to build million-user web sites in PHP. This should be a good laugh!"
Speakers wait in a queue, seated near the stage, so there's a smooth flow. For the most part, anyone can join the queue, even a speaker who's just finished. If there are no speakers, the judges slash hosts will animate the audience for a while, then call a break.
Separately, there's a demo room, with wired Internet and tables and space for quiet demos and walk-throughs. We don't do demos or code on the main stage, unless someone wants to post code snippets to the IRC channel.
Coffee and WiFi: these have to work perfectly and copiously. These were the two great things about NDC Oslo. Espresso everywhere, and the best WiFi ever built. For coffee, there are no baristas. There are a range of good espresso machines; some work on capsules and some with ground coffee. There are no grinders, too noisy. Making a great espresso for another person is part of the bonding process (as I explained in HumanCoin).
There's a bar, and from five PM or so, it serves beer in bottles. Code Mesh did work hard on their beer, and it was worth it. The beer can continue until late. There is more food, and perhaps one or two comedians to shift gears. The venue stays open until midnight, maybe with speakers randomly through the night. There is music, dancing, and a lot of mixing. Sure, he may be an ace Clojure coder, the question is, can he twerk?
Food… food should be a diverse and endless buffet with ample choice for vegetarians as well as carnivores. The food has to be light, or people will fall asleep (which is actually fine, the chairs will be comfy.) Salads of every style and concoction, cheeses, cured meats, breads of all types, miso, fried rice, small dumplings, nachos, chicken bits, hot sauce, cold sauce, vegetable samosas, beef skewers, peanuts, and so on.
In terms of scope for a conference, "we actually build stuff" is pretty damned awesome. However I think the key is the speakers: they must all be people with war stories, in any domain that touches on our industry. To filter out the dispassionate speakers, the conference will cover hotel costs but not travel. One of the best talks at Build Stuff was from a guy building rockets. We need more of those kinds of stories.
Conversation on a Plane
I was lucky enough to meet Mathias Verraes in the airport and as we were on the same flight to Brussels, we sat together. On the flight to Vilnius, he'd sat behind me and was reading the ZeroMQ book, which surprised us both when we found out.
Mathias kicked off the discussion by talking about some ideas for improving meetings. I said something like, it's ironic to see people make sensible improvements to something that be totally scrapped. Meetings are global mutable state, toxic to concurrency and scaling and change latency. Mathias was taken aback and asked me to explain, and thus we talked for two hours.
At a certain point in this discussion, which I really wish we'd recorded, Mathias said, "but surely there are exceptions," to my statement that a LSNS was the best way to organize. I thought about it, then decided, no. This isn't about subjective organization design. This is a universal truth, which we can approach closer and closer. Amdahl's Law.
It's an interesting and perhaps disturbing concept, that the perfect model for a large-scale system is built into the fabric of the Universe. As humans, we're discovering it, not inventing it. This model applies to large-scale systems at many scales. It may even define how we evolved, as a species, a fundamental property as solid as gravity or time.
The Large Scale Network System is still an emerging truth. We have at least captured some of its essential properties: event-driven message passing, fully concurrency, no global mutable state, heterogeneous, fully asynchronous, learning-propelled, accurate, efficient, and low-latency. We are still in the dark about how to do big data properly. Amazon still recommends me to buy my own book. I've suggested one direction to explore for this.
Summing Up
My takeaway lesson from Code Mesh and Build Stuff is that the best conversations cut through narrow vertical topics, and that there are some utterly fascinating people around. I'd be hard-pressed to find a single person who I didn't learn something from in Vilnius. Yet not a single of those many chats was about specific languages, platforms, even tools. I think this reflects in my lengthy report, which I hope made some sense to you.
Thanks again to the folk at Erlang Factory for organizing Code Mesh and giving me a stage; thanks also to Greg and Neringa for making Build Stuff happen. It was a beautiful two weeks that turned the end of year "god I hate Winter" mood into a "wow…!" feeling of joy.
by pieterh