Some guys at Stanford invited me to speak at their EE Computer Systems Colloquium last week. Pretty cool, eh? It was quite an honor. I wound up giving a talk on dynamic languages: the tools, the performance, the history, the religion, everything. It was a lot of fun, and it went over surprisingly well, all things considered.
They've uploaded the video of my talk, but since it's a full hour, I figured I'd transcribe it for those of you who want to just skim it.
This is the first time I've transcribed a talk. It's tricky to decide how faithful to be to my spoken wording and phrasing. I've opted to try to make it very faithful, with only minor smoothing.
Unfortunately I wound up using continuation-passing style for many of my arguments: I'd occasionally get started on some train of thought, get sidetracked, and return to it two or three times in the talk before I finally completed it. However, I've left my rambling as-is, modulo a few editor's notes, additions and corrections in [brackets].
I didn't transcribe Andy's introduction, as it seems immodest to do so. It was funny, though.
Technical corrections are welcome. I'm sure I misspoke, oversimplified, over-generalized and even got a few things flat-out wrong. I think the overall message will survive any technical errors on my part.
The talk...
Thank you everybody! So the sound guys told me that because of a sound glitch in the recording, my normally deep and manly voice, that you can all hear, is going to come through the recording as this sort of whiny, high-pitched geek, but I assure you that's not what I actually sound like.
So I'm going to be talking about dynamic languages. I assume that you're all dynamic language interest... that you've got an interest, because there's a dude down the hall talking about Scala, which is you know, this very strongly typed JVM language (a bunch of you get up and walk over there – exactly.) So you know, presumably all the people who are like really fanatical about strong typing, who would potentially maybe get a little offended about some of the sort of colorful comments I might inadvertently make during this talk — which, by the way, are my own opinions and not Google's — well, we'll assume they're all over there.
All right. I assume you all looked through the slides already, so I don't need to spend a whole lot of time with them. I'll go into major rant-mode here at the end. My goal is... for you guys to come away with, sort of a couple of new pictures in your mind, thinking about how languages have evolved over the last 20 years, where they're going, what we can do to fix them, that kind of thing.
Does anyone here know how to use a Mac? It's showing me this weird, uh... thing... OK. All right. Here goes.
So!
Popular opinion of dynamic languages: slooooow! They're always talking about how Python is really slow, right? Python is, what, like 10x to 100x slower? And they have bad tools.
And also there's this sort of, kind of difficult-to-refute one, that says at millions of lines of code, they're maintenance nightmares, right? Because they don't have static types. That one, uh, unfortunately we're not going to be able to talk much about, because not many people have millions-of-lines code bases for us to look at — because dynamic languages wind up with small code bases. But I'll talk a little bit about it.
So first of all, one of my compatriots here, who's an actual smart person, like probably everybody in this room, you're all waaay smarter than me — I got invited here for the booger jokes, all right? – he's a languages guy, and he said: "You know, you can't talk about dynamic languages without precisely defining what you mean."
So I'm going to precisely define it. Dynamic languages are, by definition... Perl, Python, Ruby, JavaScript, Lua, Tcl... all right? [(laughter)] It's the working set of languages that people dismiss today as "dynamic languages." I'll also include Smalltalk, Lisp, Self, Prolog, some of our stars, you know, from the 70s and 80s that, uh, well they'll come up here today too.
I'm deliberately not going down the path of "well, some static languages have dynamic features, and some dynamic languages have static types", because first of all it's this neverending pit of, you know, argument, and second of all, as you're going to see, it's completely irrelevant to my talk. The two... sort of qualities that people associate with "dynamic": one would be sort of... runtime features, starting with eval, and the other would be the lack of type tags, the lack of required type tags, or even just escapes in your type system. These things work together to produce the tools problems and the performance problems, ok? And I'll talk about them, and how they're going to be fixed.
All right!
I just talked about that [slide].
So! Uh... yeah, that's right, I'm at Stanford! Forgot about that. So I've been interviewing for about 20 years, at a whole bunch of companies, and yeah, Stan– every school has this sort of profile, right? You know, the candidates come out with these ideals that their profs have instilled in them. And Stanford has a really interesting one, by and large: that their undergrads and their grad students come out, and they believe that C and C++ are the fabric with which God wove the Universe. OK? And they truly [think]: what is it with all these other languages?
Whereas like MIT and Berkeley, they come out, and they're like "languages, languages, languages!" and you're like, uh, dude, you actually have to use C and C++, and they're like "oh." So it's funny, the kinds of profiles that come out. But this one [first slide bullet point], I mean, it's kind of a funny thing to say, because the guy's a Ph.D., and he's just discovered Turing's thesis. Of course all you need is C or C++. All you need is a Turing machine, right? You know?
What we're talking about here is fundamentally a very personal, a very political, sort of a, it's almost a fashion statement about who you are, what kind of language you pick. So, you know... unfortunately we could talk, I mean I've got 2 hours of ranting in me about this topic, but I'm gonna have to, like, kinda like narrow it down to... we're gonna talk about dynamic languages because people are out there today using them. They're getting stuff done, and it works. All right? And they really do have performance and tools issues.
But they're getting resolved in really interesting ways. And I'm hoping that those of you who are either going out into the industry to start making big things happen, OR, you're researchers, who are going to be publishing the next decade's worth of papers on programming language design, will take some interesting directional lessons out of this talk. We'll see.
All right. So why are dynamic languages slow? Uh, we all know they're slow because... they're dynamic! Because, ah, the dynamic features defeat the compiler. Compilers are this really well understood, you know, really really thoroughly researched... everybody knows THIS [brandish the Dragon Book], right?
Compilers! The Dragon Book! From your school! OK? It's a great book. Although interestingly, heh, it's funny: if you implement everything in this book, what you wind up with is a really naïve compiler. It's really advanced a long way since... [the book was written] and they know that.
Dynamic languages are slow because all the tricks that compilers can do to try to guess how to generate efficient machine code get completely thrown out the window. Here's one example. C is really fast, because among other things, the compiler can inline function calls. It's gonna use some heuristics, so it doesn't get too much code bloat, but if it sees a function call, it can inline it: it patches it right in, ok, because it knows the address at link time.
C++ — you've got your virtual method dispatch, which is what C++ you know, sort of evangelists, that's the first thing they go after, like in an interview, "tell me how a virtual method table works!" Right? Out of all the features in C++, they care a lot about that one, because it's the one they have to pay for at run time, and it drives them nuts! It drives them nuts because the compiler doesn't know, at run time, the receiver's type.
If you call foo.bar(), foo could be some class that C++ knows about, or it could be some class that got loaded in afterwards. And so it winds up — this polymorphism winds up meaning the compiler can compile both the caller and the callee, but it can't compile them together. So you get all the overhead of a function call. Plus, you know, the method lookup. Which is more than just the instructions involved. You're also blowing your instruction cache, and you're messing with all these, potentially, code optimizations that could be happening if it were one basic-block fall-through.
All right. Please – feel free to stop me or ask questions if I say something that's unclear. I know, just looking around the room, that most of you probably know this stuff better than I do.
So! The last [bullet point] is really interesting. Because nobody has tried, for this latest crop of languages, to optimize them. They're scripting languages, right? They were actually designed to either script some host environment like a browser, or to script Unix. I mean the goal was to perform these sort of I/O-bound computations; there was no point in making them fast. Except when people started trying to build larger and larger systems with them: that's when speed really started becoming an issue.
OK. So obviously there's a bunch of ways you can speed up a dynamic language. The number one thing you can do, is you can write a better program. The algorithm, you know, is gonna trump any of the stuff you're doing at the VM – you can optimize the hell out of Bubble Sort, but...
Native threads would be really nice. Perl, Python, Ruby, JavaScript, Lua... none of them has a usable concurrency option right now. None of them. I mean, they kinda have them, but they're like, Buyer Beware! Don't ever use this on a machine with more than one processor. Or more than one thread. And then you're OK. It's just, you know...
So actually, this is funny, because, all right, show of hands here. We've all heard this for fifteen years now – is it true? Is Java as fast as C++? Who says yes? All right... we've got a small number of hands... so I assume the rest of you are like, don't know, or it doesn't matter, or "No."
[Audience member: "We read your slides."] You read my slides. OK. I don't know... I can't remember what I put in my slides.
But it's interesting because C++ is obviously faster for, you know, the short-running [programs], but Java cheated very recently. With multicore! This is actually becoming a huge thorn in the side of all the C++ programmers, including my colleagues at Google, who've written vast amounts of C++ code that doesn't take advantage of multicore. And so the extent to which the cores, you know, the processors become parallel, C++ is gonna fall behind.
Now obviously threads don't scale that well either, right? So the Java people have got a leg up for a while, because you can use ten threads or a hundred threads, but you're not going to use a million threads! It's not going to be Erlang on you all of the sudden. So obviously a better concurrency option – and that's a huge rat's nest that I'm not going to go into right now – but it's gonna be the right way to go.
But for now, Java programs are getting amazing throughput because they can parallelize and they can take advantage of it. They cheated! Right? But threads aside, the JVM has gotten really really fast, and at Google it's now widely admitted on the Java side that Java's just as fast as C++. [(laughter)]
So! It's interesting, because every once in a while, a C++ programmer, you know, they flip: they go over to the Dark Side. I've seen it happen to some of the most brilliant C++ hackers, I mean they're computer scientists, but they're also C++ to the core. And all of a sudden they're stuck with some, you know, lame JavaScript they had to do as an adjunct to this beautiful backend system they wrote. And they futz around with it for a while, and then all of a sudden this sort of light bulb goes off, and they're like "Hey, what's up with this? This is way more productive, you know, and it doesn't seem to be as slow as I'd sort of envisioned it to be."
And then they maybe do some build scripting in Python, and then all of a sudden they come over to my desk and they ask: "Hey! Can any of these be fast?" Ha, ha, ha! I mean, these are the same people that, you know, a year ago I'd talk to them and I'd say "why not use... anything but C++? Why not use D? Why not use Objective-C? Why not use anything but C++? Right?
Because we all know that C++ has some very serious problems, that organizations, you know, put hundreds of staff years into fixing. Portability across compiler upgrades, across platforms, I mean the list goes on and on and on. C++ is like an evolutionary sort of dead-end. But, you know, it's fast, right?
And so you ask them, why not use, like, D? Or Objective-C. And they say, "well, what if there's a garbage collection pause?"
Oooh! [I mock shudder] You know, garbage collection – first of all, generational garbage collectors don't have pauses anymore, but second of all, they're kind of missing the point that they're still running on an operating system that has to do things like process scheduling and memory management. There are pauses. It's not as if you're running DOS! I hope. OK?
And so, you know, their whole argument is based on these fallacious, you know, sort of almost pseudo-religious... and often it's the case that they're actually based on things that used to be true, but they're not really true anymore, and we're gonna get to some of the interesting ones here.
But mostly what we're going to be talking about today is the compilers themselves. Because they're getting really, really smart.
All right, so first of all I've gotta give a nod to these languages... which nobody uses. OK? Common Lisp has a bunch of really high-quality compilers. And when they say they achieve, you know, "C-like speed", you've gotta understand, you know, I mean, there's more to it than just "does this benchmark match this benchmark?"
Everybody knows it's an ROI [calculation]. It's a tradeoff where you're saying: is it sufficiently fast now that the extra hardware cost for it being 10 or 20 percent slower (or even 2x slower), you know, is outweighed by the productivity gains we get from having dynamic features and expressive languages. That's of course the rational approach that everyone takes, right?
No! Lisp has all those parentheses. Of course nobody's gonna look at it. I mean, it's ridiculous how people think about these things.
But with that said, these were actually very good languages. And let me tell you something that's NOT in the slides, for all those of you who read them in advance, OK? This is my probably completely wrong... it's certainly over-generalized, but it's a partly true take on what happened to languages and language research and language implementations over the last, say 30 years.
There was a period where they were kind of neck and neck, dynamic and static, you know, there were Fortran and Lisp, you know, and then there was a period where dynamic languages really flourished. They really took off. I mean, I'm talking about the research papers, you can look: there's paper after paper, proofs...
And implementations! StrongTalk was really interesting. They added a static type system, an optional static type system on top of Smalltalk that sped it up like 20x, or maybe it was 12x. But, you know, this is a prototype compiler that never even made it into production. You've gotta understand that when a researcher does a prototype, right, that comes within, you know, fifty percent of the speed gains you can achieve from a production compiler... because they haven't done a tenth, a hundredth of the optimizations that you could do if you were in the industry cranking interns through the problem, right?
I mean HotSpot's VM, it's got like ten years of Sun's implementation into not one, but two compilers in HotSpot, which is a problem they're trying to address. So we're talking about, you know, a 12x gain really translates to something a lot larger than that when you put it into practice.
In case I forget to mention it, all these compiler optimizations I'm talking about, I do mean all of them, are composable. Which is really important. It's not like you have to choose this way or you have to choose that way. They're composable, which means they actually reinforce each other. So God only knows how fast these things can get.
This is the only interesting... this is actually the only, I would say, probably original, sort of compelling thought for this talk today. I really – I started to believe this about a week ago. All right? Because it's an urban legend [that they change every decade]. You know how there's Moore's Law, and there are all these conjectures in our industry that involve, you know, how things work. And one of them is that languages get replaced every ten years.
Because that's what was happening up until like 1995. But the barriers to adoption are really high. One that I didn't put on the slide here, I mean obviously there's the marketing, you know, and there's the open-source code base, and there are legacy code bases.
There's also, there are also a lot more programmers, I mean many more, orders of magnitude more, around the world today than there were in 1995. Remember, the dot-com boom made everybody go: "Oooh, I wanna be in Computer Science, right? Or I just wanna learn Python and go hack." OK? Either way. (The Python hackers probably made a lot more money.)
But what we wound up with was a bunch of entry-level programmers all around the world who know one language, whichever one it is, and they don't want to switch. Switching languages: the second one is your hardest. Because the first one was hard, and you think the second one's going to be that bad, and that you wasted the entire investment you put into learning the first one.
So, by and large, programmers – you know, the rank-and-file – they pretty much pick a language and they stay with it for their entire career. And that is why we've got this situation where now, this... See, there's plenty of great languages out there today. OK?
I mean obviously you can start with Squeak, sort of the latest Smalltalk fork, and it's beautiful. Or you can talk about various Lisp implementations out there that are smokin' fast, or they're smokin' good. Or in one or two cases, both.
But also there's, like, the Boo language, the io language, there's the Scala language, you know, I mean there's Nice, and Pizza, have you guys heard about these ones? I mean there's a bunch of good languages out there, right? Some of them are really good dynamically typed languages. Some of them are, you know, strongly [statically] typed. And some are hybrids, which I personally really like.
And nobody's using any of them!
Now, I mean, Scala might have a chance. There's a guy giving a talk right down the hall about it, the inventor of – one of the inventors of Scala. And I think it's a great language and I wish him all the success in the world. Because it would be nice to have, you know, it would be nice to have that as an alternative to Java.
But when you're out in the industry, you can't. You get lynched for trying to use a language that the other engineers don't know. Trust me. I've tried it. I don't know how many of you guys here have actually been out in the industry, but I was talking about this with my intern. I was, and I think you [(point to audience member)] said this in the beginning: this is 80% politics and 20% technology, right? You know.
And [my intern] is, like, "well I understand the argument" and I'm like "No, no, no! You've never been in a company where there's an engineer with a Computer Science degree and ten years of experience, an architect, who's in your face screaming at you, with spittle flying on you, because you suggested using, you know... D. Or Haskell. Or Lisp, or Erlang, or take your pick."
In fact, I'll tell you a funny story. So this... at Google, when I first got there, I was all idealistic. I'm like, wow, well Google hires all these great computer scientists, and so they must all be completely language-agnostic, and ha, ha, little do I know... So I'm up there, and I'm like, we've got this product, this totally research-y prototype type thing, we don't know. We want to put some quick-turnaround kind of work into it.
But Google is really good at building infrastructure for scaling. And I mean scaling to, you know, how many gazillion transactions per second or queries per second, you know, whatever. They scale like nobody's business, but their "Hello, World" takes three days to get through. At least it did when I first got to Google. They were not built for rapid prototyping, OK?
So that means when you try to do what Eric Schmidt talks about and try to generate luck, by having a whole bunch of initiatives, some of which will get lucky, right? Everybody's stuck trying to scale it from the ground up. And that was unacceptable to me, so I tried to... I made the famously, horribly, career-shatteringly bad mistake of trying to use Ruby at Google, for this project.
And I became, very quickly, I mean almost overnight, the Most Hated Person At Google. And, uh, and I'd have arguments with people about it, and they'd be like Nooooooo, WHAT IF... And ultimately, you know, ultimately they actually convinced me that they were right, in the sense that there actually were a few things. There were some taxes that I was imposing on the systems people, where they were gonna have to have some maintenance issues that they wouldn't have [otherwise had]. Those reasons I thought were good ones.
But when I was going through this debate, I actually talked to our VP Alan Eustace, who came up to a visit to Kirkland. And I was like, "Alan!" (after his talk) "Let's say, hypothetically, we've got this team who are really smart people..."
And I point to my friend Barry [pretending it's him], and I'm like: "Let's say they want to do something in a programming language that's not one of the supported Google languages. You know, like what if they wanted to use, you know, Haskell?"
What I really wanted to do at the time was use Lisp, actually, but I didn't say it. And [Alan] goes, "Well!" He says, "Well... how would you feel if there was a team out there who said they were gonna use... LISP!" [(laughter)]
He'd pulled his ace out of his [sleeve], and brandished it at me, and I went: "that's what I wanted to use." And he goes, "Oh." [(turning away quickly)] And that was the end of the conversation. [(laughter)]
But you know, ultimately, and it comes up all the time, I mean we've got a bunch of famous Lisp people, and (obviously) famous Python people, and you know, famous language people inside of Google, and of course they'd like to do some experimentation. But, you know, Google's all about getting stuff done.
So that brings us full circle back to the point of this topic, which is: the languages we have today, sorted by popularity at this instant, are probably going to stay about that popular for the next ten years.
Sad, isn't it? Very, very sad. But that's the way it is.
So how do we fix them?
How – how am I doing for time? Probably done, huh? Fifteen minutes? [(audience member: no, more than that)] OK, good.
So! I'm gonna talk a little bit about tools, because one interesting thing I noticed when I was putting this thing together, right, was that the ways you solve tools problems for dynamic languages are very similar to the way you solve perf problems. OK? And I'm not going to try to keep you guessing or anything. I'll tell you what the sort of... kernel of the idea is here.
It's that... the notion of "static" versus "dynamic", where you kind of have to do all these optimizations and all these computations statically, on a language, is very old-fashioned. OK? And increasingly it's becoming obvious to everybody, you know, even the C++ crowd, that you get a lot better information at run-time. *Much* better information.
In particular, let me come back to my inlining example. Java inlines polymorphic methods! Now the simplest way to do it was actually invented here at Stanford by Googler Urs Hoelzle, who's, you know, like VP and Fellow there, and it's called, it's now called Polymorphic Inline Caching. He called it, uh, type-feedback compilation, I believe is what he called it. Great paper. And it scared everybody, apparently. The rumors on the mailing lists were that people were terrified of it, I mean it seems too hard. And if you look at it now, you're like, dang, that was a good idea.
All it is, I mean, I told you the compiler doesn't know the receiver type, right? But the thing is, in computing, I mean, heuristics work pretty well. The whole 80/20 rule and the Power Law apply pretty much unilaterally across the board. So you can make assumptions like: the first time through a loop, if a particular variable is a specific instance of a type, then it's probably going to be [the same type] on the remaining iterations of the loop. OK?
So what he [Urs] does, is he has these counters at hot spots in the code, in the VM. And they come in and they check the types of the arguments [or operands]. And they say, all right, it looks like a bunch of them appear to be class B, where we thought it might be class A.
So what we're gonna do is generate this fall-through code that says, all right, if it's a B – so they have to put the guard instruction in there; it has to be correct: it has to handle the case where they're wrong, OK? But they can make the guard instruction very, very fast, effectively one instruction, depending on how you do it. You can compare the address of the intended method, or you can maybe do a type-tag comparison. There are different ways to do it, but it's fast, and more importantly, if it's right, which it is 80-90% of the time, it falls through [i.e., inlines the method for that type - Ed.], which means you maintain your processor pipeline and all that stuff.
So it means they have predicted the type of the receiver. They've successfully inlined that. I mean, you can do a whole bunch of branching, and they actually found out through some experimentation that you only need to do 2 to 4 of these, right, before the gain completely tails off. So you don't have to generate too much of this. And they've expanded on this idea now, for the last ten years.
Getting back to my point about what's happening [over the past 30 years], there was an AI winter. You all remember the AI winter, right? Where, like, investors were pumping millions of dollars into Smalltalk and Lisp companies who were promising they'd cure world hunger, cure cancer, and everything?
And unfortunately they were using determinism!
They're using heuristics, OK, but you know... before I came to Google, you know, I was really fascinated by something Peter Norvig was saying. He was saying that they don't do natural language processing deterministically any more. You know, like maybe, conceivably, speculating here, Microsoft Word's grammar checker does it, where you'd have a Chomsky grammar, right? And you're actually going in and you're doing something like a compiler does, trying to derive the sentence structure. And you know, whatever your output is, whether it's translation or grammar checking or whatever...
None of that worked! It all became way too computationally expensive, plus the languages kept changing, and the idioms and all that. Instead, [Peter was saying] they do it all probablistically.
Now historically, every time you came along, and you just obsoleted a decade of research by saying, "Well, we're just gonna kind of wing it, probabilistically" — and you know, Peter Norvig was saying they get these big data sets of documents that have been translated, in a whole bunch of different languages, and they run a bunch of machine learning over it, and they can actually match your sentence in there to one with a high probability of it being this translation.
And it's usually right! It certainly works a lot better than deterministic methods, and it's computationally a lot cheaper.
OK, so whenever you do that, it makes people MAD.
Their first instinct is to say "nuh-UUUUUUH!!!!" Right? I'm serious! I'm serious. It happened when John von Neumann [and others] introduced Monte Carlo methods. Everyone was like "arrgggggh", but eventually they come around to it. They go "yeah, I guess you're right; I'll go back and hit the math books again."
It's happening in programming languages today. I mean, as we speak. I mean, there's a paper I'm gonna tell you about, from October, and it's basically coming along and... it's not really machine learning, but you're gonna see it's the same kind of [data-driven] thing, right? It's this "winging it" approach that's actually much cheaper to compute. And it has much better results, because the runtime has all the information.
So let me just finish the tools really quick.
And I'm not talking to you guys; I'm talking to the people in the screen [i.e. watching the recording] – all these conversations I've had with people who say: "No type tags means no information!" I mean, effectively that's what they're saying.
I mean...
What's foo? It's a function. How did I know that? [(laughter)] What's bar? What's x? You know, it's a composite type. It's an Object. It has two fields that are strings. Call it a record, call it a tuple, call it whatever you want: we know what it is.
The syntax of a language, unless it's Scheme, gives you a lot of clues about the semantics, right? That's actually the one place, maybe, where lots of syntax actually wins out [over Scheme]. I just thought of that. Huh.
OK, so... then you get into dynamic languages. This [code] is all JavaScript. This is actually something I'm working on right now. I'm trying to build this JavaScript code graph, and you actually have to know all these tricks. And of course it's undecidable, right, I mean this is, you know, somebody could be defining a function at the console, and I'm not gonna be able to find that.
So at some point you've gotta kind of draw the line. What you do is, you look at your corpus, your code base, and see what are the common idioms that people are using. In JavaScript, you've got a couple of big standard libraries that everybody seems to be including these days, and they all have their slightly different ways of doing function definitions. Some of them use Object literals; some of them use the horrible with statement, you know, that JavaScript people hate.
But your compiler can figure all these out. And I was actually going through this Dragon Book, because they can even handle aliasing, right? Your IDE for JavaScript, if I say "var x = some object", and you know...
Did I handle this here [in the slides]?
Yeah, right here! And I say, foo is an object, x is foo, and I have an alias now. The algorithm for doing this is right here in the Dragon Book. It's data-flow analysis. Now they use it for compiler optimization to do, you know, live variable analysis, register allocation, dead-code elimination, you know, the list kind of goes on. It's a very useful technique. You build this big code graph of basic blocks...
So it's actually one of the few static-analysis that's actually carrying over in this new dynamic world where we have all this extra information. But you can actually use it in JavaScript to figure out function declarations that didn't actually get declared until way later in the code.
Another big point that people miss is that the Java IDEs, you know, that are supposedly always right? They're wrong. If you miss one time, you're wrong. Right? In Java Reflection, obviously, the IDE has no information about what's going on in that string, by definition. It's a string: it's quoted; it's opaque.
And so they always wave their hands and say "Ohhhhh, you can't do Rename Method!"
Even though Rename Method came from the Smalltalk environment, of course, right? And you say, "It came from the Smalltalk environment, so yes, you can do Rename Method in dynamic languages."
And they say "NO! Because it'll miss sometimes!"
To which, I say to you people in the screen, you'd be astonished at how often the Java IDEs miss. They miss every single instance of a method name that shows up in an XML configuration file, in a reflection layer, in a database persistence layer where you're matching column names to fields in your classes. Every time you've deployed some code to some people out in the field...
Rename Method only works in a small set of cases. These Refactoring tools that, really, they're acting are like the Holy Grail, you can do ALL of that in dynamic languages. That's the proof, right? [I.e., static langs miss as often as dynamic – Ed.]
It's not even a very interesting topic, except that I just run across it all the time. Because you ask people, "hey, you say that you're ten times as productive in Python as in your other language... why aren't you using Python?"
Slow? Admittedly, well, we'll get to that.
And tools. Admittedly. But I think what's happened here is Java has kind of shown the new crop of programmers what Smalltalk showed us back in the 80s, which is that IDEs can work and they can be beautiful.
And more importantly – and this isn't in the slides either, for those of you who cheated – they have to be tied to the runtime. They complain, you know, the Java people are like "Well you have to have all the code loaded into the IDE. That's not scalable, it's not flexible, they can't simulate the program just to be able to get it correct."
And yet: any sufficiently large Java or C++ system has health checks, monitoring, it opens sockets with listeners so you can ping it programmatically; you can get, you know, debuggers, you can get remote debuggers attached to it; it's got logging, it's got profiling... it's got this long list of things that you need because the static type system failed.
OK... Why did we have the static type system in the first place?
Let me tell you guys a story that, even if you know all this stuff, is still going to shock you. I credit Bob Jervis for sharing this with me (the guy who wrote Turbo C.)
So javac, the Java compiler: what does it do? Well, it generates bytecode, does some optimizations presumably, and maybe tells you some errors. And then you ship it off to the JVM. And what happens to that bytecode? First thing that happens is they build a tree out of it, because the bytecode verifier has to go in and make sure you're not doing anything [illegal]. And of course you can't do it from a stream of bytes: it has to build a usable representation. So it effectively rebuilds the source code that you went to all that effort to put into bytecode.
But that's not the end of it, because maybe javac did some optimizations, using the old Dragon Book. Maybe it did some constant propagation, maybe it did some loop unrolling, whatever.
The next thing that happens in the JVM is the JIT undoes all the optimizations! Why? So it can do better ones because it has runtime information.
So it undoes all the work that javac did, except maybe tell you that you had a parse error.
And the weird thing is, Java keeps piling... I'm getting into rant-mode here, I can tell. We're never going to make it to the end of these slides. Java keeps piling syntax on, you know, but it's not making the language more expressive. What they're doing is they're adding red tape and bureacracy for stuff you could do back in Java 1.0.
In Java 1.0, when you pulled a String out of a Hashtable you had to cast it as a String, which was really stupid because you said
You know, it's like... if you had to pick a syntax [for casting], you should at least pick one that specifies what you think it's supposed to be, not what it's becoming – obviously becoming – on the left side, right?
And everybody was like, "I don't like casting! I don't like casting!" So what did they do? What they could have done is they could have said, "All right, you don't have to cast anymore. We know what kind of variable you're trying to put it in. We'll cast it, and [maybe] you'll get a ClassCastException."
Instead, they introduced generics, right, which is this huge, massive, category-theoretic type system that they brought in, where you have to under[stand] – to actually use it you have to know the difference between covariant and contravariant return [and argument] types, and you have to understand why every single mathematical... [I tail off in strangled frustration...]
And then what happens on mailing lists is users say: "So I'm trying to do X." And they say: "WELL, for the following category-theoretic reasons ...there's no way to do it." And they go: "Oh! Oh. Then I'm gonna go use JavaScript, then." Right?
I mean, it's like, what the hell did this type system do for Java? It introduced inertia and complexity to everybody who's writing tools, to everybody who's writing compilers, to everybody who's writing runtimes, and to everybody who's writing code. And it didn't make the language more expressive.
So what's happening? Java 7 is happening. And I encourage you all to go look at that train wreck, because oh my God. Oh, God. I didn't sleep last night. I'm all wired right now because I looked at Java 7 last night. And it was a mistake. [(laughter)] Ohhh...
OK. So! Moving right back along to our simple dynamic languages, the lesson is: it's not actually harder to build these tools [for dynamic languages]. It's different. And nobody's done the work yet, although people are starting to. And actually IntelliJ is a company with this IDEA [IDE], and they... my friends show off the JavaScript tool, you know, and it's like, man! They should do one for Python, and they should do one for every single dynamic language out there, because they kick butt at it. I'm sure they did all this stuff and more than I'm talking about here.
All right. Now we can talk about perf. This is the Crown Jewels of the talk. Yeah. So... unfortunately I have to make the disclaimer that everybody thinks about performance wrong, except for you guys 'cuz you all know, right? But seriously, I mean, you know, you understand, I started out of school... *sigh*
OK: I went to the University of Washington and [then] I got hired by this company called Geoworks, doing assembly-language programming, and I did it for five years. To us, the Geoworkers, we wrote a whole operating system, the libraries, drivers, apps, you know: a desktop operating system in assembly. 8086 assembly! It wasn't even good assembly! We had four registers! [Plus the] si [register] if you counted, you know, if you counted 386, right? It was horrible.
I mean, actually we kind of liked it. It was Object-Oriented Assembly. It's amazing what you can talk yourself into liking, which is the real irony of all this. And to us, C++ was the ultimate in Roman decadence. I mean, it was equivalent to going and vomiting so you could eat more. They had IF! We had jump CX zero! Right? They had "Objects". Well we did too, but I mean they had syntax for it, right? I mean it was all just such weeniness. And we knew that we could outperform any compiler out there because at the time, we could!
So what happened? Well, they went bankrupt. Why? Now I'm probably disagreeing – I know for a fact that I'm disagreeing with every Geoworker out there. I'm the only one that holds this belief. But it's because we wrote fifteen million lines of 8086 assembly language. We had really good tools, world class tools: trust me, you need 'em. But at some point, man...
The problem is, picture an ant walking across your garage floor, trying to make a straight line of it. It ain't gonna make a straight line. And you know this because you have perspective. You can see the ant walking around, going hee hee hee, look at him locally optimize for that rock, and now he's going off this way, right?
This is what we were, when we were writing this giant assembly-language system. Because what happened was, Microsoft eventually released a platform for mobile devices that was much faster than ours. OK? And I started going in with my debugger, going, what? What is up with this? This rendering is just really slow, it's like sluggish, you know. And I went in and found out that some title bar was getting rendered 140 times every time you refreshed the screen. It wasn't just the title bar. Everything was getting called multiple times.
Because we couldn't see how the system worked anymore!
Small systems are not only easier to optimize, they're possible to optimize. And I mean globally optimize.
So when we talk about performance, it's all crap. The most important thing is that you have a small system. And then the performance will just fall out of it naturally.
That said, all else being equal, let's just pretend that Java can make small systems. Heh, that's a real stretch, I know. Let's talk about actual optimization.
And by the way, here are some real examples, sort of like the Geoworks one, where a slower language wound up with a faster system. It's not just me. I've seen it all over the place. Do you know why this one happened? Why was the Ruby on Rails faster than Struts? This started one of the internet's largest flamewars since Richard Stallman dissed Tcl back in the 80s, you know. You guys remember that? [(laughter)]
I mean, the Java people went nuts, I mean really really nuts, I mean like angry Orcs, they were just like AAAaaaaauuuugh, they did NOT want to hear it. OK? It was because they were serializing everything to and from XML because Java can't do declarations. That's why. That's the reason. I mean, stupid reasons, but performance comes from some strange places.
That said, OK, disclaimers out of the way...
Yeah yeah, people are using them.
Um, yeah. So JavaScript. JavaScript has been really interesting to me lately, because JavaScript actually does care about performance. They're the first of the modern dynamic languages where performance has become an issue not just for the industry at large, but also increasingly for academia.
Why JavaScript? Well, it was Ajax. See, what happened was... Lemme tell ya how it was supposed to be. JavaScript was going away. It doesn't matter whether you were Sun or Microsoft or anybody, right? JavaScript was going away, and it was gonna get replaced with... heh. Whatever your favorite language was.
I mean, it wasn't actually the same for everybody. It might have been C#, it might have been Java, it might have been some new language, but it was going to be a modern language. A fast language. It was gonna be a scalable language, in the sense of large-scale engineering. Building desktop apps. That's the way it was gonna be.
The way it's really gonna be, is JavaScript is gonna become one of the smokin'-est fast languages out there. And I mean smokin' fast.
Now it's not the only one that's making this claim. There's actually a lot of other... you guys know about PyPy? Python in Python? Those crack fiends say they can get C-like performance. Come on... COME ON! They... I mean, seriously! That's what they say.
Here's the deal, right? They're saying it because they're throwing all the old assumptions out. They can get this performance by using these techniques here, fundamentally. But if nobody believes them, then even when they achieve this performance it's not gonna matter because still nobody's gonna believe them, so all of this stuff we're talking about is a little bit moot.
Nevertheless, I'm going to tell you about some of the stuff that I know about that's going on in JavaScript.
So type inference. You can do type inference. Except that it's lame, because it doesn't handle weird dynamic features like upconverting integers to Doubles when they overflow. Which JavaScript does, interestingly enough, which is I guess better behavior than... I mean, it still overflows eventually, right?
We overflowed a long at Google once. Nobody thought that was possible, but it actually happened. I'll tell you about that later if you want to know.
So... oh yeah, I already talked about Polymorphic Inline Caches. Great! I already talked about a lot of this stuff.
This one's really cool. This is a trick that somebody came up with, that you can actually – there's a paper on it, where you can actually figure out the actual types of any data object in any dynamic language: figure it out the first time through by using this double virtual method lookup. They've boxed these things. And then you just expect it to be the same the rest of the time through [the loop], and so all this stuff about having a type-tag saying this is an int – which might not actually be technically correct, if you're going to overflow into a Double, right? Or maybe you're using an int but what you're really using is a byte's worth of it, you know. The runtime can actually figure things out around bounds that are undecidable at compile time.
So that's a cool one.
This is the really cool one. This is the really, really cool one. Trace trees. This is a paper that came out in October. This is the one, actually... I'll be honest with you, I actually have two optimizations that couldn't go into this talk that are even cooler than this because they haven't published yet. And I didn't want to let the cat out of the bag before they published. So this is actually just the tip of the iceberg.
But trace trees, it's a really simple idea. What you do is your runtime, your VM, you know, it's interpreting instructions and can count them. Well, it can also record them! So any time it hits, basically, a branch backwards, which usually means it's going to the beginning of a loop, which usually means it's going to be a hot spot, especially if you're putting a counter there... Obviously [in] the inner loops, the hot spots will get the highest counts, and they get triggered at a certain level.
It turns on a recorder. That's all it does. It starts recording instructions. It doesn't care about loop boundaries. It doesn't care about methods. It doesn't care about modules. It just cares about "What are you executing?"
And it records these tree – well actually, traces, until they get back to that point. And it uses some heuristics to throw stuff away if it goes too long or whatever. But it records right through methods. And instead of setting up the activation, it just inlines it as it goes. Inline, inline, inline, right? So they're big traces, but they're known to be hot spots.
And even here in the Dragon Book, Aho, Sethi and Ullman, they say, you know, one of the most important things a compiler can do is try to identify what the hot spots are going to be so it can make them efficient. Because who cares if you're optimizing the function that gets executed once at startup, right?
So these traces wind up being trees, because what can happen is, they branch any time an operand is a different type. That's how they handle the overflow to Double: there'll be a branch. They wind up with these trees. They've still got a few little technical issues like, for example, growing exponentially on the Game of Life. There's a blog about it, um... I'm sorry, I've completely forgotten his name [Andreas Gal], but I will blog this. And the guy that's doing these trace trees, he got feedback saying that they've got exponential growth.
So they came up with this novel way of folding the trace trees, right, so there are code paths that are almost identical and they can share, right?
It's all the same kind of stuff they were doing with these [Dragon Book] data structures back when they were building static compilers. We are at the very beginning of this research! What has happened is, we've gone from Dynamic [to] AI Winter... dynamic research stopped, and anybody who was doing it was sort of anathema in the whole academic [community]... worldwide across all the universities. There were a couple of holdouts. [Dan Friedman and] Matthias Felleisen, right, the Little Schemer guys, right? Holding out hope.
And everybody else went and chased static. And they've been doing it like crazy. And they've, in my opinion, reached the theoretical bounds of what they can deliver, and it has FAILED. These static type systems, they're WRONG. Wrong in the sense that when you try to do something, and they say: No, category theory doesn't allow that, because it's not elegant... Hey man: who's wrong? The person who's trying to write the program, or the type system?
And some of the type errors you see in these Hindley-Milner type [systems], or any type system, like "expected (int * int * int)", you know, a tuple, and "but got (int * int * int)", you know [(clapping my hands to my head)] it's pretty bad, right? I mean, they've, I think they've failed. Which is why they're not getting adopted.
Now of course that's really controversial. There are probably a bunch of type-systems researchers here who are really mad, but...
What's happening is: as of this Ajax revolution, the industry shifted to trying to optimize JavaScript. And that has triggered what is going to be a landslide of research in optimizing dynamic languages.
So these tricks I'm telling you about, they're just the beginning of it. And if we come out of this talk with one thing, it's that it's cool to optimize dynamic languages again! "Cool" in the sense of getting venture funding, right? You know, and research grants... "Cool" in the sense of making meaningful differences to all those people writing Super Mario clones in JavaScript.
You know. It's cool.
And so I encourage you, if you're a language-savvy kind of person, to jump in and try to help. Me, I'm probably going to be doing grunt implementations, since I'm not that smart.
And I don't even need to talk about this [last optimization — Escape Analysis], since you already knew it.
All right! So that's it. That's my talk. CPUs... you get all the good information about how a program is running at run time. And this has huge implications for the tools and for the performance. It's going to change the way we work. It's eventually – God, I hope sooner rather than later – going to obsolete C++ finally.
It's going to be a lot of work, right?
<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://4.bp.blogspot.com/__0laCHgF5uc/SCfgacOp-