The Internet Archive’s historical search engine, the “Wayback Machine,” grows by half a billion pages a week.
On this episode of Recode Decode, hosted by Kara Swisher, entrepreneur, activist and founder of the Internet Archive Brewster Kahle discussed the growth of the open internet and the importance of having a history of the internet available to everyone.
You can read some of the highlights from the interview at that link, or listen to it in the audio player above. Below, we’ve posted a lightly edited complete transcript of their conversation.
If you like this, be sure to subscribe to Recode Decode on iTunes, Google Play Music, TuneIn and Stitcher.
Kara Swisher: Today in the red chair I have Brewster Kahle, and I’m thrilled to have him. I’ve known him for so long, since the beginnings of the internets. He’s an entrepreneur, an activist and the founder of the Internet Archive, which just turned 20. In 1996, he also co-founded Alexa Internet, which was sold to Amazon in 1999, way before we all started saying “Alexa” every day. And if that wasn’t enough, he’s also a member of the Internet Hall of Fame. Brewster, welcome to Recode Decode.
Brewster Kahle: Great to be here.
We have known each other before that.
Maybe 30 years.
Let’s talk about where we met. We met at AOL, when you sold your company to AOL.
Yeah, that was the early days.
Give people some real history here because these young people, they don’t know anything.
Well, one of the ideas of this internet was to try to build the library of everything. How do we do that? And just, okay, yeah, we actually could, technically, but there’s a bunch of pieces missing. One, we had to get publishing online, so a bunch of us worked on that before there was the World Wide Web.
You had come from where?
I started a company called Thinking Machines, or helped start a company. It was a supercomputer company, and we made it so we could build computers that could search everything. We did that, and then we found that we needed something to distribute it over, so there was this ARPANET idea. We basically put the first search engines on the internet. It was called WAIS, wide area information services. It was the first publishing system.
Wide area information services. That sure trips off your tongue, doesn’t it?
Yeah, I know. Never name your thing an acronym. The idea was to make everyone into a publisher, and this was kind of a radical idea in 1989. It came along, and it came before Gopher, and before the World Wide Web. The web being open and really easy, also, to publish into, made it so that the search and publishing all merged into the World Wide Web. I helped put a bunch of folks online, like Wall Street Journal, New York Times, Encyclopedia Britannica, to try to anchor the open world so it wasn’t a closed world, like what AOL was doing.
Which you sold your company to.
Yes.
So, you started WAIS. Did you have venture capital funding for that?
No.
No, you just started?
Yeah. Bootstrap startup. In those days, venture capital was not directed well.
Right, right. You started this, you were living where?
Boston, and then I moved out to San Francisco because the thought was that San Francisco was going to be where the internet was going to happen. In 1989, moved out, did a project with Apple, KPMG Peat Marwick, Dow Jones, Thinking Machines, to go and build a system so you could turn to your computer to ask questions. This was kind of a radical idea.
I remember.
The idea that you could then ask that of databases that were very far away.
Right, and without technical people between you and the information.
Yes, you would just ask natural-language questions.
Why was that such a big deal? Because now it’s like breathing, essentially. What did it take to make that cognitive leap for people like you?
It required thinking that computers were gonna be smart, that they were gonna be able to interact on our terms. Also, that we were gonna have these computers that weren’t just gonna be latched to the mainframe in your company, but they were gonna reach out onto the network. That was something we could see coming because of my background at MIT and ARPANET and the like. The next thing was to ask questions of multiple places at once. You could ask questions of lots of places.
Because the answers were out there.
The answers were out there. That we hadn’t seen put back after WAIS basically lost out to the World Wide Web. We ended up going to centralized places like Google or AltaVista or Go or something.
Before we get to that, so you started WAIS. You sold it to AOL, and the hope was that you would create an internet, correct?
Yes.
That was the idea?
The wonderful thing about AOL at that time is they had a business model that paid authors. They took 15 percent of the money that they got paid by subscribers, and put it upstream in royalties system. I like the royalties system rather than ads.
Because they brought people into the service and made it interesting, and therefore, they owed them money.
Yes.
That was the idea.
That was the idea. It’s how books worked. But it’s not how magazines and TV work. They work with ads, and I didn’t like ads. It’s a winner-takes-all system, it makes for monopolies, but you can have a royalty-based system. You can have a distributed system of lots of different authors, lots of different venues for getting things out there. There’s no necessary contraction point.
You sold it to AOL. You just said very quickly, but it’s a big deal, you lost out to the internet. What does that mean?
WAIS lost out to the web. WAIS was an open system for going and doing client-server interactions over the internet, and the web was just better. It was easier to play with. It had some failings to it, but the wonder of it is it works so well and it’s so easy to use.
Especially when they had the browser and then the linking.
So Mosaic, which was the first modern web browser before Netscape, Mosaic was called Mosaic because it was made up of the web, WAIS, Gopher and FTP. There were these different systems at that time until the dominant one won out, which was the World Wide Web.
What did you think when that was happening? You had this company, you sold it to AOL, you worked there for a short time.
Yes. Well, AOL changed track. Instead of going and thinking, “Well, how do we go and reimburse and encourage authors and publishers?” They said maybe they should charge for people to get to their eyeballs.
Access.
Remember those terrible days of, “Let’s talk about eyeballs?” Sticky eyeballs. What we really want is sticky eyeballs. It was a particularly dark period of internet development. It really turned to being an ad-based system. At that point, I knew that wasn’t really the place to be, so I left to go ...
It’s sort of Mafioso-like, too, in a lot of ways, like shake ’em down.
It didn’t have to be that way.
No, it didn’t.
I think AOL was dealing with hyper growth at that point. They were just trying to keep the thing together at all. We got publishing online by around 1994, 1995. That phase was going well. I could go back to the real dream of building the library of everything.
Where did you come up with that dream, just the idea? Alexandria’s where Alexa came from?
Yes, Alexa’s named after the library of Alexandria, so it’s meant to be. That was in 1980. I’m the last phase of the optimistic, utopian ’60s thing. 1980, walking back and forth across the Charles River, trying to figure out how do you use technology to build something really pretty great?
The idea of putting everything online, making a library of everything, was actually promised for a long time. “Oh, we’re gonna have the Library of Congress on our desk!” or something like that. I said, “How hard could that be?” I joined the AI Lab at MIT to develop chips so we could build this, and Danny Hillis had an idea of how to build a supercomputer that could actually scale to size of the library of everything.
The idea is, the Library of Alexandria was destroyed and it had all the knowledge of the previous world in it.
Yes, the Library of Alexandria 300 B.C. is best known for not being here anymore. That it’s just completely destroyed. If they’d made another copy and put it in China or in India, we’d have the other works of Aristotle, the other plays of Euripides.
Instead of destroyed in a fire.
Not just one. It really wasn’t a fire, that’s the myth. It basically decayed because people didn’t want universal knowledge anymore. There were alternative ideas with the Christians, that basically went and said, “Well, this is the way of truth.” There’s a wonderful movie called “Agora” about the late-stage Library of Alexandria that shows how times change, and the idea of having universal access was a Greek ideal, not what we had for the Middle Ages.
Your concept here was the same thing. Universal knowledge of everything, everywhere.
Yes.
Huge idea.
It’s a huge idea, but it’s actually coming about. In fact, if you ask most people, “Isn’t it all there now? Isn’t that what Google is? Haven’t they digitized all the books?” And the answer is no, we haven’t, but we’re starting to get people excited that you can turn to your machine and not just ask questions and have it be based on Wikipedia, but based on the depth of all of the libraries and all of what’s been done.
Let’s talk first about what happened to you. You went and you did Alexa and sold it to Amazon. What was the thinking behind Alexa? Now the Echo in Alexa device?
They used the name for that. Alexa internet still exists as a company that was to try to catalog the web. It archived the whole web and it used ...
It’s trying to be a competitor to Google at the time, as I recall.
It wasn’t a search engine. It answered questions in a different way. It was a new way. It was based on how people moved through the net. It turned out that the search engine guys actually scaled much better than I thought they would.
Alexa Internet is now used mostly as sort of a metrics company, helping debug their ... It was bought by Amazon in 1999 and we did something interesting. We started a nonprofit at exactly the same day. There’s a contract buried in the soul of Alexa Internet to donate a copy of everything it collects to the Internet Archive with a six-month time delay.
When the acquisition was going through ... Actually, Jeff Bezos said, “Why don’t we buy this?” I was like, “Eh, I tried that with AOL, didn’t work out all that well.” He said, “Well, how do we make this work?” I said, “Well, let’s run it as a separate company.” We were true to that. Now 17 years later it’s still run as a separate company.
Also there’s this funny thing. It’s a contract to donate everything to a nonprofit. He said, “Well, six months. I’m not sure about the time, but I’m okay with that.” And he’s been true to that. Every day, Alexa Internet donates a copy of the web collection that it’s been building forever to the Internet Archive. In 2002 I moved to the Internet Archive, and the idea was to try to build the library for real.
To build a library of everything for real?
Yes.
We’re going to talk about that in the next section. This thing that you’ve been working on forever was everything. What I want to do is just get ... Before we break ... Everything? What do you mean by everything?
Everything ever published by humankind. If you take all of the books, all the music, all the video, all the webpages, all the software. If you count it up it actually turns out to be quite manageable. The Library of Congress is the largest library by far. It’s got about 28 million books. A book is about a megabyte. 28 million megabytes is about 28 terabytes. That fits on four hard drives. You could buy it at Best Buy for less than a month’s rent. At least storing it, if you had it in Microsoft Word form, is completely manageable by current computer standards. We knew this was coming.
Except they’re in books. They’re on paper.
We have to go and photograph them. Get the words out of them. OCR them. Then also work with the copyright issues.
We’re gonna talk about that and more when we get back with Brewster Kahle who created the Internet Archive which is the archive of everything. We’ll talk about what that means going forward.
[ad]
We’re here with Brewster Kahle of the Internet Archive. He’s also a longtime entrepreneur and worked in lots of different companies. Created them. Sold one to AOL, one to Amazon. Since then, for the last 20 years he’s been working on the Internet Archive. It’s the chronicle of everything. The digital chronicle of everything, correct? Talk about what’s happened in the 20 years and how much have you saved there and what your goals are, going forward.
We started by archiving the World Wide Web that was ephemeral of media. A webpage lasts about 100 days before it’s either changed or deleted. That’s the average life of a webpage. The web is not a reliable structure. We said, “Okay, why don’t we just archive it?” We made this thing called the Wayback Machine. It’s free. You can go to archive.org and type in a URL.
I have done it many times
There’s a new feature that we came out with just two weeks ago. You can search it. You can go and search. If you go to archive.org and you go to the Wayback Machine page. Everybody does it, but you have to click now to go and say ‘search’. Then you can say, for instance, “Trump for president.” If you do that you can get to see the 2008 Trump website even though it’s been offline since 2007. Or the 2012 presidential Trump website also pops up. Or Hillary Clinton’s ... When she was in the Senate. You can search to find the websites that you’re looking for and then find your way into it.
You can’t find the page with the search engine. We’re not that good. We get to the website. So there’s the Wayback Machine that we’ve been building and it’s been growing at about a half a billion pages every week. It is huge. We work at the Library of Congress. There are about 1,000 librarians that help curate the collection. It’s a record of our time.
Why? Why do you want to keep this? Why do we have an old “Hillary Clinton for Senate” page?
If we allow those who control the present to control the past then they control the future. That’s George Orwell. We need to know what came before. Because the web flips on and off, actually most of the best of the web is already off the web. Whole newspapers go away. Countries blink on and off. If we want to know what happened 10 years ago, 20 years ago, it’s often the only record. Now we’re trying to get it so it’s woven back into the web. We’ve gone through ... There’s a robot some volunteers made to go on all of the Wikipedia outbound links that now are broken. If they’re in the Wayback Machine they’re fixed.
I see.
There are now one million links in Wikipedia that now are good again.
It resides where? Speaking of making copies in India and China. Where does it all reside? And how big is it at this point?
The Wayback Machine, or sort of all of the Internet Archive actually, lives in a couple of places. There’s a copy in San Francisco. We bought a church. We converted a church and made it into a library. There’s a copy also in Richmond, California. A partial copy in Alexandria, Egypt, really at the new Library of Alexandria. A partial copy in Amsterdam. The idea is to have multiple copies in multiple places.
Constantly updated.
We do pretty well with most of them. We’re trying to now expand it further. With the change in the administration, it really sort of is a wake-up call to go and say, “Gosh, it’s really time to make sure that there are copies in other places.” We’re working with others to make this happen.
What do you expect people to do with that information? The Internet Archive. Just to have it. Just to have a digital footprint of ourselves.
Let’s weave it into society so that basically you have a ... It’s as easy to search the library as it is to search the current web. Would that be great? We get about three to four million users every day. We’re about the 300th most popular website, according to Alexa Internet. It’s popular. A lot of people want older materials.
What do you think are the most important materials? Are there any or is it just what people want? Just what people request, or are there critical things that you think need to stay current?
What we try to do is if people saw it before they can see it again. If you can’t refer back to things, if you can’t quote what happened before, then you can’t do compare and contrast. Kids in college, that’s what thinking critically is. You have to be able to quote what happened, compare and contrast. There’s certain things like the web that could just be taken away from you. Then you would have no way of being able to refer to it.
We’re going to talk about that later. It’s interesting. I was just talking to someone about how people can still lie about it and continue to do so even with those examples on the web. Which is interesting. Donald Trump yesterday literally told three Twitter lies that were easily provable. It was fascinating. I literally got in an argument. They’re like, “Well, it’s a he-said/she-said.” I’m like, “No, no, no.”
No, no, no, no. Here it is. This brings me to another collection. We started collecting television in the year 2000 because nobody else was doing it at scale.
Google had started to do that.
Did they?
Yeah. They had a secret room of televisions. I visited once. They were recording everything.
Well, I don’t know. This is the only one that’s at this scale. We’ve been recording 60 channels 24 hours a day from 25 countries. Around this election we went and recorded four channels from each of 20 battleground areas. The idea was to then find all of the political ads and then say, “When did they run? How often did they run?” Expose them to fact-checkers like Politifact that then went and did fact-checking. We made this available to journalists mostly. Or bloggers. To really understand. That’s politicaladarchive.org. We haven’t gone further than that. We took the debates and did audio fingerprinting on it and found weird quotes from that appear in other stations post debate.
So, what popped?
What popped and for whom. What popped for Fox News was completely different than what popped for MSNBC. What you see out of this is what we know anecdotally, that people live in very cloistered separate universes, our little echo chambers within themselves, so that they don’t even see the world the same way. The world, if they put on media glasses, they see a completely different universe.
How do you solve that by collecting everything? Because even if you’re collecting it and it’s right there, they don’t want to believe it.
Some may. All we can do is to collect it and make it easily available. What’s exciting to me is that not only can we now see that there are these separate worlds, but also we can see across universes. We can see across these collections. Hopefully we start to build mechanisms for those that want to be elucidated. Everybody says they do but often they just want their biases reconfirmed. For those that want to be able to see the broader picture, let’s make it easy to. Because right now if you have time-based media, even this podcast, it just sort of flows over. How do you go and make it a referenceable easy resource?
This is a perfect example.
How to we turn media into data?
You do the web, you’re doing television, you’re doing books, which is hard. Correct? Still as hard as ever to do books?
It’s a pain in the neck. We do about 1,000 books every day.
How many books are there?
28 million is the library of congress. Maybe 100 million total. If you take a Yale, a Princeton or a Boston public library, it’s about 10 million. We’ve digitized 2.5 million books. Google has as well, but they’re locked up. They locked up the public domain, which we think of as a sin. The library sort of made elite services around it just for themselves. We would like this to be available to everybody.
Right. These things are out of copyright? All the books that are ...
Most of them are out of copyright, but we’ve also digitized 100,000 in copyright machines.
What are the issues now? Google got into trouble over that. Over what they were doing, largely because they were doing it for their own benefit. Do you imagine you’ll be able to scan all of the copyrighted books? And where are we in the copyright law?
We’re doing very well. Starting in 2010, so six years ago, we started digitizing at scale in-copyright non-rights-cleared books. Basically all of it. Made it available to the blind and dyslexic because you could, you should. We did. And we’re also making it available via lending. If you go to openlibrary.org — this is fun. I hope that you do — go to openlibrary.org and go to the little carousel of borrowable books. You can check out a book and then you’re the only one that can read it until you either return it or two weeks later. This is emulating, basically, the restrictions of a library. It’s just a library. We’ve been doing this and it’s been working out great.
I see. You have copyrighted material available.
Libraries only. Most of what’s in a library is copyrighted materials. We’ve always worked with publishers. What we do is buy and lend. We buy what we can in books. We buy what we can and then we scan what we have to. Right now, actually in books, if we’re trying to buy books the way we used to buy books, but you’re trying to buy them in e-form, they put funny restrictions on them. We hope they ease up kind of like the music guys did.
Where are books going, from your perspective? Your wise friends of bookbinding and printing.
Yes. The San Francisco Center for the Book where they make books with lead and with sewing and all of this. Books play a very special role in our society. It’s the long-form way of expressing an idea. It’s not just an interview. It’s not just a speech. It’s a long-form discourse. We want to see that continue. The long-form narrative. Whether it’s actually going to be on dead trees or on electronics or both, I don’t really care as long as we have ways of expressing these long-form ideas. Some people don’t really need to be paid because they’re paid by the academics or whatever. Others do want to get paid, so we need a mechanism of paying people. Buying what we can in the ebook world so that they are owned by libraries ... three to four billion dollars from libraries go to authors and publishers. Let’s keep that going. But right now it’s stumbling.
It is. Absolutely. You’ve got books. Video?
Video, the way we’ve been doing video is people can upload to the archive.org.
This is non-TV video.
Yeah. If you go to archive.org there’s an upload button. Thousands of people a day upload things. We also archive the popular things on YouTube. If they’re referred to on Twitter then we archive the YouTube videos.
What about all of social media? You’re not archiving that.
Social media is ... They make it difficult for us. Facebook makes it the hardest. Twitter is pretty easy, but we’re not at scale. Around the election we made sure that we had all of the election-related tweets that we could. Hundreds of millions of them.
Why do they make it difficult?
I think because they own these platforms and they want to own and control what happens.
Where does it go?
It goes into their databases. To Twitter’s credit, when Alex McGilvery was there, he arranged so that the copy went to the Library of Congress. I don’t know much of what’s happened with that yet, but at least it’s there. We’d like to see more archive-aware organizations like Alexa that donated a copy as time went on. We’d like to see that happen more.
That they are chronicling what has happened so they’re searchable? Although there’s privacy issues because a lot of it’s personal. Not Twitter, that would be more public, correct?
In general, the public materials ... You still have to have mechanisms of redaction. We do that with robot exclusions or people complain.
With all the stuff going on, it’s all going on on social media now. A lot of the information.
A lot of it’s there. We’re trying to adapt as best we can to these platforms. That’s a big shift of how the internet ... We thought it was going to be a very ...
Easy thing.
Easy, democratic. Anybody can go and be a player. Now we have these large platforms that really control our lives.
What about Snapchat and things like that?
Snapchat goes away, so I would say Instagram is of the sort meant to be preserved.
Which you don’t have access to currently?
It’s tricky. We get what we can.
How do you get that?
By writing special crawls. For those librarians ... We have about 1,000 librarians that subscribe to archiveit.org and they go and tell us what should be archived to be put in particular collections.
Things get missed, then.
Things get missed, so we have robots that try to sweep everything up that we can. Things that are linked to. We only have the parts of Flickr that people link to, for instance. Things do get missed.
What should happen then? Should these companies just say, “We’re going to preserve this forever in a nonprofit place,” or not? I don’t see them doing that. I don’t see them doing it at all.
I would hope that they’d donate things to the Internet Archive. It hasn’t been happening much.
How do you go to them when you ... Do go to Mark Zuckerberg and say, “You have the history of the world here.” Current world.
Those conversations have happened. Not with Mark Zuckerberg. Love to. Some say, “Yeah, we can do this,” or “We can’t do this,” or they say, “It’s legal issues or proprietary issues.” It’s sort of a collision between culture and capitalism.
Who wins there?
Right now I’d say capitalism is winning. It always has. We have to adapt to as much as we can.
What else do you think you need to preserve? VR as we start to move?
VR, 3-D scans. But actually we have to do a better job of preserving the web. The web was not designed to be preserved. We’ve been working with some very forward-thinking people on building a decentralized web.
Explain that first. Super geeky.
Here’s the problem with the web — this is so cool, it’s worth it. The internet is decentralized in the sense that you can kind of nuke any part of it and it still works. That was its original design. The World Wide Web isn’t that way. You go and knock out any particular piece of hardware, it goes away.
Can we make a reliable web that’s served from many different places, kind of like how the Amazon cloud works, but for everybody? The answer is yes, you can. You can make kind of a pure to pure distribution structure, such that the web becomes reliable. Another is that we can make it private so that there’s reader privacy. Edward Snowden has brought to light some really difficult architectural problems of the current World Wide Web. The GCHQ, the secret service of the British, watched everybody using WikiLeaks, and then offered all of those IP addresses, which are personally identifiable in the large part, to the NSA. The NSA had conversations about using that as a means to go and ... monitor people at an enhanced level that those are now suspects.
Libraries have long had history with people being rounded up for what they’ve read and bad things happening to them. We have an interest in trying to make it so that there’s reader privacy. If we can build a decentralized web ...
They couldn’t find you.
That basically don’t track all the readers. It’s like when we were growing up and you could read a book and nobody knew you were reading that book unless somebody saw you reading that book. It’s not part of government databases and corporate databases that you’re reading that book. Can we make it reliable so that the decentralized web is still information, but it’s a mechanism that makes it so that the web also would be archivable such as the active working natures of websites. Not just what they looked like, but how they worked, could be kept alive as people retire, die, want to work on a different project.
Right. That’s fascinating. We’re talking with Brewster Kahle, who is the creator of the Internet Archive and a number of other companies. When we get back we’ll talk more about where Silicon Valley in technology is going.
[ad]
We’re here with Brewster Kahle of the Internet Archive. Brewster and I have gone way way back. We’re super old, I guess. We met back at AOL in the early ’90s when he sold his company there. I talked to him a lot when he sold his other company to Amazon. Since then, at the Internet Archive. Internet Archive is having its 20th anniversary, which means the web is kind of old in a lot of ways.
Let’s talk a little bit about ... You were talking a little bit about the decentralized web and the ability to preserve all kinds of new forms of digital media, which is where everybody transacts now pretty much. I think there was a really interesting study recently that half of all Americans get their news from Facebook. I know, right? You want to just cry.
Well, it’s filtered.
It’s filtered and it’s fake, some of it. One percent, according to Mark Zuckerberg, but that’s 1 percent too much for my tastes.
Talk a little bit about where it’s going. Like how memories are preserved, where you think we’re going from a digital innovation point of view in this area.
We’re evermore connected. I think, if you ask younger people what their most valuable possession is, or what would they run back into the house if there were on fire, it would be their phone. We’re very connected and very interdependent. It’s been interesting. We’ve had a period of trust that is going on where people are trusting putting their thoughts ...
Everything.
Everything online. And making a lot of it very public. If it’s not completely public it’s public to these large corporations. We’ve had this trusting period and I sure hope that this continues. Even before some of the political changes there were corporate changes where these large platform companies that controlled [so much] exerted a certain level of ownership over the sharing that we’ve done. Who we’re connected to. What we’re talking about. What we buy. What we’re reading. All of those have been put in place of large corporations, and people are having some struggles with this. With this new administration that has been openly saying that neutrality is not something they are in favor of, journalists are hated and I guess jokingly were threatened with being killed.
They weren’t joking.
I sure hope they were joking. We have a real change in what’s going on. We have at least a fire drill to go and say, “What kind of world do we want to live in?” In a world that’s got a lot of different points of view going on. Also, development in the last 20 years is the Chinese firewall. There’s a different world that you see inside China of what the world looks like because it’s filtered by their government. Things blink on and off. I think the Internet Archive is now blocked in Turkey.
That guy. Jesus.
We’re seeing a shift. It’s not fun and games anymore, I guess would be the way to look at it. It’s not like, “Oh, isn’t this a cool new widget?” It’s how we understand our world. It’s how we interact with each other and it’s all quite controllable.
It’s always been controlled, but it’s always been a free-wheeling thing until recently. You see things like that happen in China, in Turkey obviously. Many many of the authoritarian governments understand how powerful this information is and the access to information is for people, and want to limit it. Especially negative information or critical information about the regimes which they deem inaccurate even if it’s accurate. The U.S. has been at the forefront of not being like that. Although the spying by the NSA has disturbed everyone, it’s been relatively free-wheeling in terms of what’s available and what people can access and say. Do you see that changing? And what does that mean?
It can change. A lot of what’s built the World Wide Web in the open information infrastructure that we understand is just actually embedded in a few laws that basically make it so it can be user originated content that can be then hosted on other companies’ websites without having them have liability. As long as, “Hey, we’re in copyright infringement,” if they took it down quickly then that was good enough. Those laws are all quite changeable. They’re not universal.
The United States, as you point out, has been actually freer than most in terms of offering access to information. It doesn’t have to be this way. Openness is not the default in terms of how if you look back in time, or even around the world. We need to show how openness works better. That you end up with companies that thrive better. That you have, with standards, you have more competition that builds more jobs. That openness is a win.
That’s a reasonable argument, Brewster. I mean, a lot of the control of information or the flooding of information with false information or slightly skewed information. I think we’ve gotten into a situation where ... I mean, just this recent controversy on Facebook fake news. How do you look at that? Facebook’s argument is, “Hey, we’re a platform. We’re not responsible.” I think they’re fully responsible for a lot of that. And, “Oh, it doesn’t have impact.” Well of course it has impact. Then it becomes a, “This is their point of view, this is your point of view,” when actually there is a reality of what happens.
These enormous players have absolute roles and responsibilities in how things work. The medium is the message. We are living in Marshall McLuhan land where the medium forms how we think about things. The way the web is coded determines a lot of how we live our lives online. It’s now not just the web, it’s how Facebook is coded, how Google is coded.
Who codes it.
Who codes it. Who controls this. Can we make multiples of these? I guess theoretically we can. Winner takes all is such an important thing in this networked world. It’s not important ... It seems to happen so often because one player gets a bit of a lead. They make some more money that allows them to continue the R&D much more quickly and you lose out, so the Yahoos disappear — effectively disappear — and now there’s Bing and there’s Google.
Really, there’s just Google.
Really, there’s just Google. Or there’s just Facebook, or there’s just Twitter. We have some challenges. The wonder is that this technology is very available. It’s pretty amazing. It’s pretty startling how good it actually is at answering questions. Yet its ability to control what we think is absolute.
What should these companies do? Their go-to thing is, “It’s a platform, we’re not responsible.” I’m responsible as a media person if I get something wrong, but they’re not responsible. What do you think? You were saying that they do have responsibilities, but what are the responsibilities that you think should be? That they should either take up or be put upon them?
We used to have antitrust laws in the United States that had teeth to try to stop monopolies from happening. Not just make it so that consumers have cheaper products, but actually that there was competition. We made it so that there wasn’t lock-in. That’s in large part gone in this country. Looking for that is challenging. Sometimes I find some of the open systems to be some of the most interesting things going on. I think it’s interesting when Linux can go and be an operating system that competes with the very biggest companies. IBM is now supporting and has moved over to Linux. What does that mean? That the open world can work. It does require a large amount of investment of time and effort and tolerance. Sometimes the open systems don’t work as well.
Maybe we should be spending more time and effort within academia, within government, within our general lives to go and invest in things that aren’t going to end up putting us in a cage that we’re going to regret later. Is that a bit of a plea and a cry? Yes, it is. But Wikipedia is the No. 5 most popular website of all websites. That’s pretty stunning. For the good world. EFF is probably the most influential IP law firm in the Silicon Valley, yet they give away all of their time. Can you make it possible for this open world, the free world, as Richard Stallman would put it, works? And the answer is yes.
Who backs that? Why would Mark Zuckerberg pay for that, for example? Or why would Larry Page pay for that? The only person I, for example, I’ve heard talking about open AI of significance is Elon Musk. The rest of them present AI as a happy shiny future where everything will be delivered to us correctly by smarter machines. He’s worried about it, for example. And I know a lot of tech people like to say, “Don’t worry about it,” but to me when a very small group of people are designing AI right now, and it’s typically white men, and they’re programming it, you have to wonder who is going to control the rest of it. Who gets other control.
Well, my schooling was in the AI lab with Danny Hillis and Marvin Minsky. Yeah, those were heady days of the ’80s when we knew we were basically bringing up our new overlord. If you’re gonna bring up your new overlords then have them read good books. What I think we got wrong back then is we sort of thought of it more like “The Terminator” where there’s sort of this Skynet that separates off and then comes back and battles us. Actually I think it’s much more of a blur between people and computers and the networks.
Let me go off the rails just for a minute. When I really thought about sort of ... If we want to understand what AIs are going to look like, I think the proto AI that we have are corporations. Corporations are sort of these funny little beasts. They’re not small. I guess they’re not little beasts, but they’re strange. It takes special training to have humans be able to fit within them. They’re made out of humans mostly but they’re more and more being built out of computers. They’re controlled by not very many people, and more and more the emphasis is on computers. Corporations are interesting as a way of trying to understand how AI’s will think, because they operate under very simple rules like, “Make more money,” and, “Who cares about blank blank blank blank blank blank.” That sort of simplified world is more likely to be what we see for the first, I don’t know. Now we’re sort of in the area of AI. For the next 50 years we may see very simplified goal structures. Very powerfully put in place by leverage. By being embedded as corporate entities, or maybe as militaries. Militaries also have very simple success metrics and they’ll be basically machines as quickly as they can make them.
Right, exactly. Who has the responsibility? Should there be an open AI so we all have those AI structures? Elon talked about attaching their own networks to our own heads to be smart like them. Where do you see it going?
I guess I wake up on two sides of the bed sometimes. We will be in a zoo soon ...
House cats. He said they’ll treat us like house cats. Give us some food and then we’ll ...
There’s a question I haven’t really gotten a good answer for, which is: If you wake up in the morning, how do you know if you’re in a zoo? How do we know? This is sort of the question. Are we there now? Probably not. But how do you know when we’ve tripped over into something like that? We’re building these very very powerful machines. I don’t see it slowing down. In machines, technology is really the extension of the arms of humans. It makes it so that people can do more. That usually means that a few people can do a lot more. Most people won’t be able to do much.
Let’s finish talking about that. Because this election sort of brings that back. A lot of the anger that Trump tapped into was manufacturing. People who lost their jobs, people that aren’t trained for the future. Really, a lot of this future that is not coming back and is actually going to get worse for everybody else. Evermore people are going to have their jobs replaced. You want to talk about manufacturing. Let’s move into office stuff. Let’s move into stuff that we didn’t think could be replaced easily. What does that set us up for as a society? We don’t have enough people trained as entrepreneurs. We don’t have enough people that do what needs to be done. Do we even need the people? You get to that point where is there even a necessity for that many people to have jobs?
The nature of jobs just seems to already be changing. Just look around. How many of us are really farming food anymore? That’s all been organized. At least in the United States. How many people are manufacturing things in the United States? That’s not happening very much anymore. Driverless cars. The nature of work looks like it’s irrevocably changing. What are we going to do? We do need a mechanism of having it so that people have a minimum standard of living. Otherwise people will revolt. Historically, if you have just too high of income disparity people are driven out and away in a way that is usually quite ... It doesn’t end well.
No.
You need a mechanism of, whether it’s the Hebrews have a mechanism of jubilee, the Babylonians has wiping the slate clean. Quite literally all debts were written down, so you just wiped them clean when there was a new monarch. You have to have some mechanism of equalizing. Is technology going to do it? It really depends on who’s building it and for what. We could end up in a sort of entertainment paradise of everybody sort of straps on VR and sort of takes their new legalized drug of choice and sort of blanks out. Wow, what a nasty world that would be.
I really kind of like the American project. The experiment of, “Hey, let’s go try out a bunch of different things and allow people to build things that they would like to have built.” Why don’t we enable and empower more people in that way? Fortunately the technologies can work that way, but we need people to think ethically. As Larry Lessig put it, some rights reserved. Even though you could assert more power over people, you might be able to do something more, don’t.
Alexa Internet, personal example. Alexa Internet was trying to understand the web by archiving all of it. We also archived how people moved around on the web. We anonymized it so we couldn’t figure out who was who. We found we didn’t need IP addresses. The idea of IP addresses, which are one per computer, is too much information. Even though we could collect it, we decided not to. It turned out it never hurt us. I hope that more and more people that are in these companies go and say, “You know? Maybe we don’t need those IP addresses. Maybe that’s toxic waste.” It’s just something that if we don’t collect it in the first place, we’re pulling back from, “Well maybe we’ll use it later because we can.” It turns into, “because we should.”
I think we’ve got a short time frame, sort of an alarm bell going off, because we don’t know how things are going to change. What’s going to happen in the new administration. If it’s not this administration, maybe it’s the next one after that. Maybe we could use this as our time in the tech community to look at ourselves and say, “What does it really mean to be a good citizen?” Being a good citizen participating in the corporate world.
Do you think we will?
Some and some.
On that note. I do not think so, Brewster, but I like that you do. I like the hopefulness. I’m less hopeful. “Because we can” is something I think they do. Because they can. It’s sort of like that with everything. Destroys societies.
One of the antidotes towards some of the toxic nature that we’ve built up in some of our especially tech companies ... We used to turn to unions for a while and that has largely been hollowed out. Any trust was very difficult to put in place, was torn down during the ’80s, but remade into something else. I actually kind of like the open source world and the companion organizations which are nonprofits. The nonprofits in the United States I think are one of the best inventions the United States has ever made. They’re kind of like secular churches. They’re there for the public good. They get a tax donation. There’s certain things that you have to do. You have to be mission oriented, your tax filings are all public on GuideStar even if you’re small. You know what all the top people are paid. You have to be there for public good.
Where I think this 501(C)(3) structure can be torn apart, and it looks like it’s in the process of that, by political abuse of it as a money laundering system. I think we can build a declaration of how a certain new kind of organization works. How transparent it is, that it gives things away. That can be very successful and spread from country to country. I think that that idea ... because we’ve seen open source software work well ... We’ve seen Wikipedia, Mozilla, Internet Archive, Electronic Frontier Foundation, Public Library of Science. These are infrastructure-type companies, but they’re not-for-profits. Is there a mechanism of building a new generation of organization that doesn’t have at its core a, “Because you can destroy as much as you can. Grow or die. Rip people to shreds because you’re allowed to, therefore you’re supposed to.” Maybe there’s something more we can grow into.
Well, Brewster, on that note, I love that 20 years later you’re still as hopeful as ever. Which is nice.
Thank you, Kara.
Brewster, it was great talking to you. Thanks for coming by.
Subscribe to the Recode Newsletter
Sign up for our Recode Daily newsletter to get the top tech and business news stories delivered to your inbox.