Continuing our QCon London speaker series, this time round, we spoke to Dave Pollak – best known as the creator of Scala web application framework Lift. As well as discussing what lies ahead for the popular expressive technology, in this interview we talk about the perils of forking, Docker FUD, and why he believes Scala never quite reached critical mass.
Voxxed: Can you outline what you spoke about at this event?
Pollak: I talked about securing PaaS using Docker and Weave. Docker, pretty much everyone knows about, but Weave is probably a little bit less well known. It’s a virtual networking layer, so you can create virtual LANs between your Docker containers. Building a PaaS, there are a lot of security concerns because you, in effect, have a multi-tenant application, but you want each of the tenants to think that they have their own building. There are lots of ways to secure that. One of the ways is what Amazon does, which is to give everyone a virtual machine. You don’t see cross-tenant communication at Amazon. There are occasional issues with their underlying machine infrastructure, but they fix those. But if you want to do it more cost effectively, not have to spend tens or hundreds of dollars per virtual machine, using Docker makes a lot more sense. You can also spin out Docker instances radically more quickly.
You touch on security – a lot of people are criticising Docker in this respect. What’s your take on the issue?
That’s their argument this week? My opinion is very much my opinion on Linux security – it’s mostly good enough, and having people focus a lot of attention on things tends to get others to fix them. There have been rumblings, but I have not seen any exploitable vulnerabilities in any Docker containers…There will be vulnerabilities everywhere we have these complex systems.
The fact that people are looking at it, and the fact that people are tunnelling into it and beating it with a sledgehammer means that at least the surface layer is going to be dusted off. Having Rocket is good, because competition is good, but I don’t see any material vulnerability design flaws. Anything in implementation, we patch…we go.
Whenever there are rumblings in a community, you’ll have stuff that people seize on. I look at it as strategic versus tactical. You can fix tactics…rolling back a strategy is much more difficult. So, I don’t see anything strategically failed in the way LXE containers could be done. Could Docker be done better? Well, yeah, though with the value of hindsight, almost anything could be better.
Speaking of language wars…you were a very early adopter of Scala. Do you still feel the way you did about the language as you did back in 2006, when it was touted as a ‘Java killer’?
I think Scala is going to be a very effective systems language. I don’t mean that in the way that C is – but a language that a small group of people use to write very powerful infrastructure. So if you look at the number of projects in the Apache foundation that are Scala based, that number is going up faster than the number of projects that are in Apache. But almost all of those projects – you know, Spark being the ‘poster-child’ of projects in Apache land – all the projects that were written in Scala, all have Java APIs.
You even see Typesafe moving in that direction. Akka has had Java APIs from the beginning. Typesafe is moving a lot to support Java 8 streams. So what you’re seeing is people writing the core infrastructure using Scala, because it allows you to be more expressive, but not forcing people to change the way they develop software, simply publishing nice Java APIs on top of the Scala code.
You can see that Scala has traction and is gaining traction in certain quarters. Many years ago, I realised that Scala was never going to be a Java alternative. It doesn’t mean that I don’t use it every day in my work. But trying to get – what is it, ten million Java developers? – to switch, is a Herculean task, and I don’t think that Scala is superior enough to get ten million developers to switch.
The people who are writing fintech software in London – Scala is a better alternative for doing that. People who are writing yet another forms based ERP application – those people, they know Java. They know that it’s verbose and it tells a story. There’s one set of patterns that everyone has adopted for Java, and so in that case, Java really is superior. Not because it’s more efficient, not because they’re going to get their job done faster, but because there’s momentum, and that counts for a lot. Herd education, herd understanding counts for a lot. Like the Docker phenomenon that we’ve seen. So you have that critical mass.
The JVM was something that reached critical mass. iOS was something that reached critical mass. Scala, no it didn’t, but I don’t think it’s something that’s going to go away. It’s a really, really powerful tool for certain use cases. I think it’s going to be a really great asset for people building complex libraries like Spark – so I don’t see it as an either-or, but I do see it as something where, are you going to get a million developers? Probably not. Is Scala still going to be around in twenty years? Probably.
On to the longevity of Scala…what are your feelings on the renewal of the Reactive Manifesto?
I’ve never touched the Reactive Manifesto. That’s Typesafe and Jonas’ thing. I’ve been using Actors in Lift since before it was popular. Actors are reactive, and in programming this makes a lot of sense, because having stuff that’s tightly coupled leads to a lot of failure modes, where if you design things that are loosely coupled, it leads to fewer failure modes. Taking it and making it a marketing manifesto rather than a technical way of doing things…it’s not a train I want to jump on. They’ve spent a lot of time promoting it…but it ain’t my thing.
You don’t need to know the Manifesto to have good practice, and I think that one of the blessings – and curses – of Scala is that there are twenty five ways of doing almost anything. There are no real patterns in Scala – or there are many patterns in Scala. Being able to have people that pick and choose their patterns for the specific application rather than saying, “OK, I have my cookie cutter, and I’m going to apply it,” that makes more sense to me. That also requires a lot more work, and it makes maintenance a lot more difficult because unless you have people who have your coding style, it makes it more difficult for them to maintain the code.
There are some really interesting, super-academic Scala forks going on…
Oh fork, are we going to talk about this?
We are! In light of the Node.js/ io.js split…do you ever feel that there could ever be a split this big in the Scala community?
The Scala community never reached critical mass, and people can, and do, point at me as the first guy to split off and throw my finger at the community. I didn’t fork Scala, but Lift is not going to be part of a mainstream effort. There are a lot of personal reasons I have for that.
I deeply understand the motivation and the frustration of the people who are forked, but on the other hand, I really in my heart of hearts, I wish Paul Phillips could find a way back. [Editor’s note: See this thread for the low-down on Paul Phillips’ issues with Scala].
In my experience, he’s been the single biggest contributor of Scala as a commercially viable thing, both technically, and his understanding of how people use and write software. So the Policy split just breaks my heart.
With all that being said, it ain’t going to happen, and I’m not sure what diluting the codebase, diluting the energy, the momentum, really does, other than to make people feel good. Being one of those people who did something because it felt good, and not necessarily right for the language, I have sympathy there.
There are also the control and the community management aspects to it all. And if you see how many really great projects that were abandoned – probably ten times more great things on GitHub than have been popularised – but it takes a particular balance. Richard Stallman hit that balance. Linus Torvalds, for all his faults, did it too. Sun managed to do something right with Java. But if you take a look at what Microsoft did with .Net, they’re still struggling with it. It’s got all the hallmarks of something that should be wildly successful, and it’s not. And it’s got nothing to do with open source. It’s got everything to do with community management. It’s really hard to know what that balance is, and how to strike it.
What’s on the roadmap ahead for Lift?
We’ve done a lot of stuff with Lift to break down the barriers between address spaces. Do events propagate information across address spaces in a way that’s invisible to the developer? And you might call that reactive, because they’re asynchronous events, or you might not – I don’t know.
But basically, being able to materialise stuff between the server and the browser without the developer really thinking about how to materialise it. One of the things that we did in Lift 3.0 was to add in client-side actors so that basically you can send a message to a server side proxy and that message gets serialised as JSON and sent to the browser and then bubbled up and sent to the actor in the browser. Or you can send a message to an actor in a browser – or a proxy in a browser – and the same thing happens.
You have the event based reactive thing that goes across the address spaces. One of the things that I’m actually going to be presenting at QCon China is a Clojure Om app running in the browser, through Lift, using Lift’s communications pipeline from a Scala app that is sourcing data from a Clojure app…because I can. So being able to interface with a variety of client libraries, melting the address space boundaries in a way that does not turn it into remote procedure calls…
Since about four months into the project, Lift has had particularly good support for bidirectional communication for between the browser and the server. The developer doesn’t have to worry about stuff like mobile client disconnects because in Lift-land we take care of that for you, but we also give you the knobs to deal with it.
With Lift 3.0 we’ve done a lot of focus on really streaming data between address spaces so that the developer has a more logic focused, application focused view of the world, rather than a plumbing focused view of the world.
The post Dave Pollak on the Lift Latest, and Scala as a Superior Systems Language appeared first on Voxxed.