In this post we interview Gabriel Hurley, Manager of Nebula’s Platform Team. Gabriel talks about what they do not teach you at school, community-based development and using GitHub as your resume. He also talks about how keeping it simple is not always sustainable, things to consider when building scalable systems and repeatability.
Gabriel has a wealth of experience from working at Nebula and NASA, so he understands what it takes to deliver and maintain systems at scale.
Q: Can you tell us a little about your background as a developer?
I am currently the Manager for Nebula's Platform Team, but I've arrived there by a pretty diverse path. Previously at Nebula I was the UX Team's Technical Lead; I built large swaths of the web interface for the Nebula One. I was also the Project Technical Lead for the OpenStack Dashboard project (Horizon), and greatly shaped its current incarnation. Prior to that I was a Core Committer for the open source Python web framework Django, and worked as a developer in the office of the CTO for IT at NASA. So far seems pretty standard, right? Well, before that I ran a small creative design agency, and before that I was in Sales/Marketing at UC Berkeley. The last programming class I took was during my freshman year of High School. I'm almost entirely self-taught, and my skillset ranges from programming to film and photography to audio engineering to graphic design, marketing, and more. When it comes to development, though, I'm a firm believer in open source, a firm believer in great user experience, and Python and Javascript are my favorite languages.
Q: What do they not teach you at school about software development?
How about everything! As I mentioned, I'm almost entirely self-taught. In fact, the last programming class I took probably did more harm than good. What I found instead is that the best way to learn any skill is to have a project you really want to accomplish using that skill. A friend had a cool idea for a web app, and after giving it a go in PHP we discovered Python/Django and that was the turning point in my career. Django has a fantastic community of brilliant developers, and the core devs are some of the best people to learn from. One of the things I love about open source is that you can start using a project, and as you become more familiar with it you start to find bugs so you go and you report those bugs. Then maybe you decide you want to try fixing a bug. Before you know it you're a part of that community making real contributions, and the world can see the work you do. Your Github account will get you more jobs than your resume as a developer these days. Learn from others and share what you do!
Q: What are the craziest “Dev Done Wrong” things that you have seen during your career?
There's a quote in the Django community which is usually attributed to Simon Willison that goes "Do the simplest thing that could possibly work." In general, this is good advice. However, it's also the root of all evil when it comes to sustainable development practices. I'd bet that many of us have, at some point in our career, FTP'd or SSH'd to a live website and changed some files on the server. This is fast, and simple, and it gets the job done. Heck, I know folks that have managed the websites of successful companies that way. And that's fine until you want to scale, or until there's an outage, or a catastrophic data loss.
Okay, that's a bit contrived, but it illustrates the point that simplicity is relative to the tools you have available. If that developer had been handed a github repository and a Heroku account ready-to-go what was "simplest" would've been a different game. If the company had their own development/deployment workflow established that embraced principles like automation and distributed storage and backups and provided these to their developers... again, "simple" all depends where you're starting from. So what keeps us from getting there?
Almost every big enterprise company, government agency, and any organization of sufficient scale has huge problems with not just legacy IT but "shadow" IT. I'm talking about servers sitting under people's desks being the sole backup of mission-critical data. Developers so fed up with procurement that they pay for AWS on their own credit cards. Back at NASA the first thing that was explained to me was that I may have access to data covered by the International Traffic in Arms Regulation (ITAR) and that if I let that outside of NASA's network I would likely be headed for prison. These restrictions and regulations are very real, and shadow IT poses very real risks to both companies and individuals without them even thinking about it. All because the tools and structures that are in place are too painful and people are drawn to what is simple.
Q: How can PaaS assist in doing dev right?
Platform-as-a-Service (PaaS) is a key piece in doing things both repeatably and painlessly. Most PaaSes have a wealth of service templates ready-to-go. Databases like MySQL or Postgres, memcached, load balancers, message queues, all the things you need to build dynamic, scalable applications. The setup time is seconds, not hours, days or weeks. This means you can spend your time working on your application code, and you don't artificially rule out good solutions because setting them up would be too painful.
Repeatability is a game-changer. When you have a script or a template that can restore your entire application to a clean state in minutes or seconds it opens up worlds of possibility. The entire architecture changes. It's not about keeping your one server up every millisecond of every day and being on pager-duty in case it goes down. You build systems that can take care of themselves.
Netflix's (former) Chief Architect Adrian Cockcroft describes the goal of the modern application developer as being to "construct a highly agile and highly available service from ephemeral and assumed broken components." The implication of that is huge. When you start with the (quite realistical) assumption that at any moment any part of your system could cease functioning you have to build your application in a very different way. Ephemeral also means that when something goes wrong, you just shoot that piece and your automation builds another one for you. Everything goes back to working, just like that. The core of this approach ultimately lies in distributed data stores (like Cassandra) and very finely decoupled service components to achieve high availability. When your database goes away, does your website stop working? When your workers stop handling your background task queue should everything come to a crashing halt? No! Each piece should be fault-tolerant at least to the point where it can handle a small amount of downtime for its connected components.
PaaS offerings are great for this because they encourage thinking about each component as a discrete piece, and many of them offer built-in SLA management... keeping services alive, respawning them if they die, perhaps even scaling them based on certain thresholds. These are all pieces you no longer have to worry about. Moreover, PaaSes all use some form of template or DSL to define exactly how your service or application should be deployed each time, meaning that by even using a PaaS you've already ensured repeatability for your deployment.
Q: What can we expect from the upcoming joint Stackato and Nebula One webinar?
The joint webinar has a very practical story that's loosely based on a request I got while I worked at NASA: The CIO walks in one day and says "we've invested millions in 'cloud' and 'big data' and nobody is using our datasets! You've got 24 hours to build a site that publishes our datasets internally or you're fired! Oh, and by the way, the data is restricted and this project is secret... everything you do has to stay inside our firewall!"
Okay, so the "24 hours or you're fired" part is made up, but the point is that this problem can be solved well within that timeframe if you've got the right tools. I'm going to demonstrate how I used a private cloud (a Nebula One system) and a PaaS (ActiveState Stackato) to build a scalable web service that can do continuous deployment and communicate with the cloud's APIs in no time flat, all without any data ever leaving the company's datacenter. I literally gave myself that challenge last week (hint: I succeeded), and the core of the webinar is me walking through exactly what I did, recorded straight off my screen as I did it. The best part is that it's only a few dozen lines of my own code, and involves a full tutorial on setting up ActiveState Stackato on Nebula in the process. If my fictional company had already offered both a Nebula One system and Stackato for self-service use by anyone at the company, I could've gone from idea to finished in half an hour, first try. It's just that powerful.
And that, in itself, is the core of what Nebula and ActiveState are doing: we're empowering developers with self-service tools that let them get their jobs done both quickly and well by making modern best practices "the simplest thing that could possibly work."
Conclusion
Thanks Gabriel!
You can meet Gabriel and ask him questions about running Stackato on Nebula One in tomorrow’s webinar. Modern Application Development Using Stackato and Nebula (January 14, 2014 at 10am PST).
Image courtesy of thirteenofclubs@flickr