Welcome to This Week in Modern Software, or TWiMS, our weekly analysis of the most interesting and most important news, stories, and events in the world of modern software and analytics.
This week, our top story takes us to Austin, Texas, for the annual extravaganza known as SXSW.
TWiMS Top Story:
At SXSW, a Shift From Apps to a Tech Lifestyle—The New York Times
What it’s about: South by Southwest (SXSW) returned to Austin this week, and added a new feather to its cap: presenters included President Barack Obama and First Lady Michelle Obama (don’t miss The Verge’s delightful feature about her mastery of social media). The president used the conference to once again beseech the tech world to bring technology tools and data to bear on the country’s most pressing issues. But he also weighed in on encryption in an onstage conversation with Texas Tribune Editor in Chief Evan Smith. “If you can’t crack that [device] at all, if government can’t get in, everybody’s walking around with a Swiss bank account in their pocket,” Obama said, according to PCWorld’s recap. Aside from the Obamas’ star power, though, what was the real takeaway from the event? Not much, according to The Verge’s Casey Newton, who bemoaned a lack of breakout tech. The New York Times’ Farhad Manjoo had a slightly different take: SXSW 2016 wasn’t about any particular device or app, Manjoo said. Rather, it was the moment that tech transcended hardware and software to become part of the cultural fabric, influencing everything from what we eat to what we wear.
Why you should care: If Manjoo’s thesis bears true, that makes techies into newly powerful arbiters of mainstream culture. Consider Go Cubes chewable coffee (tag line: “The future of coffee is here”). Manjoo writes of the product’s maker Nootrobox: “Though Nootrobox wants to reach a mainstream audience, the company sees techies as an influential beachhead—win the engineers today and you’ll get the rest of the world tomorrow.” That is, if the rest of the world can afford to keep up with the famously well-compensated tech crowd. For that matter, do they really want to? “To put it plainly, techies can be a bit strange,” Manjoo notes. Newton, on the other hand, taps into an issue that often surfaces when an annual event gets saddled with sky-high expectations: true innovation often takes a lot of time. Big breakthroughs in AI, for example, are likely still many years away, but that doesn’t suit the impatience of the gotta-have-it-now crowds standing in long lines at “South By.”
Further reading:
Obama on Encryption: ‘It’s Fetishizing Our Phones Above Every Other Value’—PCWorld
Why There Was No Breakout Technology at SXSW This Year—The Verge
@MichelleObama: An Exclusive Look at How the First Lady Mastered Social Media—The Verge
11 Things We Learned at SXSW 2016—CNET
Word ‘Innovate’ Said 650,000 Times at SXSW So Far—The Onion
Is SXSW Keeping It Real?—Fast Company
Facebook, Google, and WhatsApp Plan to Increase Encryption of User Data—The Guardian
What it’s about: In the latest chapter of Apple’s showdown with the FBI over data encryption, The Guardian reports this week that some of the biggest names in tech are well underway with plans to expand encryption of user data in their products and services. Facebook, Google, WhatsApp, and Snapchat are all cited as examples. Facebook-owned WhatsApp, for instance, will expand its existing encryption features to include voice calls. Google, meanwhile, appears to be giving renewed attention to a long-standing encrypted email project called End to End, according to The Guardian. The U.K. newspaper reports that, in a recent company town hall meeting, an engineer asked Google security and privacy exec Gerhard Eshelbeck why the company was not doing more to support encrypted communications. “[Eshelbeck] countered the company increasingly was putting effort behind such projects,” writes Danny Yadron. “Some Google employees are discussing whether the technology behind End to End can be applied to other products, though no final determinations have been made.”
Why you should care: Apple is set to appear in federal court later this month to continue its dispute with the FBI over unlocking an iPhone used by the San Bernardino shooters, so expect this story—and the heated opinions on it—to continue evolving. At face value, the companies named in The Guardian’s story obviously appear to be siding with Apple. But the piece also serves as a reminder that the encryption and privacy debate can be enormously complicated. Encrypting Gmail, for instance, would certainly complicate Google’s ad business on that service, Yadron reports. This sounds a lot like a real-world manifestation of the Berkman Center’s recent report on the possibility of criminals “going dark” on the Web or on wireless networks—and why that is unlikely to happen on a massive scale. Meanwhile, The Wall Street Journal reports that Google faces similar challenges encrypting Android devices, including potential performance problems on the many lower-end phones running the OS. As always, stay tuned for more developments (and be sure to watch this hilarious video of John Oliver’s segment on the topic.)
Further reading:
Source: Members of Congress Dismayed by FBI Director’s Lack of Tech Knowledge—Fast Company
Last Week Tonight With John Oliver: Encryption—HBO (via YouTube)
Google Faces Challenges in Encrypting Android Phones—The Wall Street Journal
10 Lessons From 10 Years of Amazon Web Services—All Things Distributed
What it’s about: Amazon Web Services turned 10 years old this week, and CTO Werner Vogels took to his All Things Distributed blog to reflect on some of the most important lessons learned along the way. It’s not surprising that there’d be plenty to look back on: In the decade since AWS first launched Amazon S3, the company has grown astronomically. AWS reported $2.4 billion in revenue in the final quarter of 2015, a 69% jump from the previous year. The company raked in a staggering $7.88 billion last year—again, this is a business that’s just 10 years old—and its cloud footprint now includes 190 countries. Vogels’ takeaways are essential reading for modern software pros in general, and especially anyone building and managing applications in the cloud.
Why you should care: There’s plenty to consider here, but we’ll point out some of our favorites, beginning with number one on Vogels’ list: “Build evolvable systems.” The cloud era required new approaches to IT, and AWS recognized that very early on. “Almost from day one, we knew that the software we were building would not be the software that would be running a year later,” Vogels writes. Similarly, he notes that modern software in the cloud requires treating failure as a given—not to foster defeatism, but to best monitor for and minimize the impact of failures. “We needed to build systems that embrace failure as a natural occurrence even if we did not know what the failure might be,” Vogels writes. “We’ve developed the fundamental skill of managing the ‘blast radius’ of a failure occurrence such that the overall health of the system can be maintained.”
Another crucial lesson, this one for API-driven software: When you build an ecosystem around your APIs, they’re “forever.” AWS can’t tinker with its APIs because they’ll disrupt their customers operations. “We knew that designing APIs was a very important task as we’d only have one chance to get it right,” Vogels says. Here’s to AWS and the cloud’s next 10 years—and the lessons we’ll all learn along the way.
Further reading:
A Decade of Innovation—Perspectives
The Amazon Tax—Stratechery
Ten Years in the AWS Cloud—AWS Official Blog
The Epic Story of Dropbox’s Exodus From the Amazon Cloud Empire—WIRED
What it’s about: Meanwhile, this week offers up another kind of AWS story: WIRED’s deep dive on why and how Dropbox has spent two-and-a-half years (and counting) moving itself off Amazon’s cloud, even as so many other companies are moving onto Amazon and other cloud platforms. For much of its existence, Dropbox stored billions of its customers’ files on AWS machines. Now, 90% of those files are stored internally: “Dropbox built its own vast computer network and shifted its service onto a new breed of machines designed by its own engineers, all orchestrated by a software system built by its own programmers with a brand new programming language.”
Why you should care: This is not necessarily a harbinger of doom for AWS. Instead, it is a measure of cloud computing’s tremendous power in the modern software era: Only the very largest operations could consider such a move, and even then they’ll still be relying on cloud architecture principles. In fact, Google employee #8 and former U.C. Berkeley professor Urs Hölzle tells Wired author Cade Metz that most organizations shouldn’t even consider a similar strategy: “The right answer is to actually not do this yourself.” Dropbox’s combination of engineering talent (including veterans of firms like Facebook and Google that have built similarly massive-scale internal systems), high-flying financial valuation, and huge user base made it possible to build its own version of the Amazon S3 in-house, which Dropbox calls “Magic Pocket.” The company also built custom hardware for Magic Pocket to run on, machines known as “Diskotech”; each box can hold up to a petabyte of data. “Just 50 of these machines could store everything human beings have ever written,” Metz writes.
It’s an impressive engineering and coding accomplishment, but one that comes with considerable risk: A slowdown in Dropbox’s growth could turn this internal cloud transformation into an expensive albatross. Moreover, it turns Dropbox into a more direct competitor to the likes of AWS and Google—not exactly meek foes. Metz notes that the engineering buzz that accompanies this kind of project can lead to massive savings for a company. But beware: “It can lead to what those in the Valley call Not Invented Here Syndrome, where companies start building all sorts of new stuff just because they’re intent on building all sorts of new stuff,” Metz writes.
Further reading:
Dropbox to Wean Off of AWS, Eyes Hybrid Approach, Its Own Infrastructure—ZDNet
The Cloud’s Missing Link: Monitoring and Management—InfoWorld
What it’s about: Cloud Technology Partners consultant David S. Linthicum’s latest column for InfoWorld calls attention to one of the critical blind spots in too many cloud environments: monitoring and management. Linthicum points out some logical reasons why: There’s a lot to think about and manage in the cloud, from costs to security and plenty more. But as cloud increasingly becomes the default setting, and the size and complexity of cloud-based production environments continues to grow, proper monitoring and management become increasingly crucial. That’s true for identifying and mitigating real-time issues—Linthicum offers the example of database performance lagging behind application requirements. But companies that aren’t using good monitoring and management tools are missing out on an even bigger opportunity, according to Linthicum: The ability to track and analyze trends over time.
Why you should care: Seeing the big picture can help cloud pros identify issues before they occur, among other benefits. “This means gathering many data points over time and drawing conclusions as to what they mean,” Linthicum writes. “For example, spotting accelerated use of storage services would suggest there could be imminent performance and capacity problems. From there, proactive corrective action could be taken, whether automated or manual.” Linthicum also shares his three fundamental criteria for selecting a monitoring and management tool, including: “Make sure your tool can gather system data over time, and then make sense of the data,” Linthicum advises. “In other words, it should have data analytics capabilities.”
Microsoft’s New Software Turns Minecraft Into a Testing Ground for AI—The Verge
What it’s about: When Microsoft announced in 2014 its plans to acquire Minecraft maker Mojang for $2.5 billion, more than a handful of people asked, “Why?” Money was one answer. But it’s now apparent that the Minecraft deal wasn’t strictly about dollars and cents, nor was it a straight-up games play. This week, Microsoft unveiled new open source software called AIX, which will enable AI researchers to turn Minecraft into a testing and development tool. (AIX is currently in private beta; Microsoft expects it to become generally available under an open source license this summer.) “Minecraft is the perfect platform for this kind of research because it’s this very open world,” says AIX creator and Microsoft researcher Katja Hofmann in a feature on Microsoft’s blog.
Why you should care: In spite of recent achievements and breakthroughs, AI still isn’t all that smart today. As Robert Schapire, a principal researcher at Microsoft Research, says in the Microsoft blog post: “The things that seem really easy for us are actually the things that are really difficult for an artificial intelligence.” The AIX project and the unstructured environment of Minecraft could be a significant step in teaching AI to actually learn. As James Vincent writes in The Verge: “Unsupervised learning of the sort that AIX fosters is the next big step for artificial intelligence…. Tools like AIX will help create better methods for AI to teach themselves, and, subsequently, start to learn what the whole world is like, not just a small slice of it.”
Further reading:
Microsoft Will Open Source AI Development Platform Based on Minecraft—PCWorld
Microsoft AIX: Using Minecraft to Build Better AI—InformationWeek
Project AIX: Using Minecraft to Build More Intelligent Technology—Microsoft Blog
Want to suggest something that we should cover in the next edition of TWiMS? Email us at blog@newrelic.com.
Tune In to the Future
Can’t get enough modern software news and commentary? Be sure to check out our Modern Software Podcast. New Relic Editor-in-Chief Fredric Paul and guests discuss the most important things happening in the world of software analytics, cloud computing, application monitoring, development methodologies, programming languages, and more. Listen to episode 9 or subscribe on iTunes.
Tech lifestyle, birthday cake, and pixel figure images courtesy of Shutterstock.com.