KEY POINTS
Getting streaming video from its source to the end user isn't a simple proposition. Video is too data intensive to flow automatically in a glitch-free manner from a video company's servers to an end user watching a movie on an Internet-connected TV. Content providers — everyone from Hulu and Netflix to a mid-sized publisher of Web video — face an array of options and several layers of middle-men to get their content to audiences. This report will explain the different components and choices, while demystifying the entire ecosystem.
Transit providers like Cogent, Level 3, and XO Communications are the basic pipelines for moving video around. Transit companies connect content providers to the "last-mile ISPs" — the Internet Service Providers that control the connections and lines that lead directly into consumer homes.
Content Distribution Networks, or CDNs, such as Akamai and LimeLight are the specialists in making sure that the content providers are getting the best video performance at the best price. CDNs overlap with the transit providers (Level 3 provides both services), but they are distinct in the sense that they don't just get the video to a destination. They also provide added value, including consulting services, customized video routing, and servers that cache and dole out video at optimal times. The CDNs collectively account for nearly three-fifths of consumer Internet traffic in the U.S. in 2014, according to DeepFields.
Transit and CDN prices are negotiated on an ad-hoc basis, based on a number of factors, including traffic volume, time of use, and other determinants. But throughput (bandwidth/speed) is the key factor driving traffic prices. Most CDN and transit contracts include service license agreements (SLA) that guarantee a level of service quality to the content providers.
In an effort to gain more control over the quality and cost of moving video around, many content providers have built out their own CDN networks. Examples include Netflix OpenConnect and Google's (YouTube) Global Cache.
The relationship between content providers, CDNs, transit providers, and the ISPs are governed by "peering agreements," which is just another way of referring to the contracts that stipulate the volume of traffic that each player is entitled to at different prices, including free and paid tiers.
Both content providers and ISPs have misrepresented the economics and mechanics of video streaming in order to advance their agendas in the context of the debates over fuzzy concepts like "net neutrality," and "Internet freedom," and we try to boil down the disputes in an even-handed, no-nonsense manner.
The disputes over video streaming fees, and the recent peering agreement between Comcast and Netflix, boil down to who should pay for the exploding popularity of streaming video services like Netflix, and at what quality of service. The different players have self-interested interpretations of how pricing should work.
Introduction
Netflix's recent peering agreement with Comcast — the contract they signed that governs how much Netflix has to pay Comcast to carry its content — ignited a firestorm of debate about the relationship between video companies and the Internet Service Providers (ISPs), which operate the "last mile," or the final segment of the Internet that connects with end users.
Content providers contend that it's the responsibility of these ISPs to handle the cost of the increased traffic from their subscribers' growing demand for streaming video. The ISPs, on the other hand, believe that content providers should shoulder some of the costs triggered by their bandwidth-intensive content.
Customers have grown frustrated, as they watch the speed of their video streams slow to a crawl on some ISPs
Unfortunately, a number of media outlets have misunderstood how peering agreements, such as the Netflix-Comcast deal, relate to the wider debate over "net neutrality," the concept that all Internet content should receive equal treatment from regulators and Internet service providers. In other words, ISPs shouldn't choke or censor certain forms of content in order to advance their own agendas.
To make matters worse, all sides in the debate over the cost of delivering video — transit providers, CDNs, ISPs, and content providers — are guilty of distorting the terms of the debate. The content providers tend to hide behind the shield of net neutrality (while cynically demanding that everyone else in the ecosystem shoulder the cost of delivering their video).
Transit providers sometimes accuse ISPs of "unfairly" charging them for access, and ISPs whine that they are unable to make money, squeezed by penny-pinching consumers on one end, and on the other by data-hogging video content providers, CDNs, and transit providers. In fact, even after signing the deal with Comcast, Netflix CEO Reed Hastings published an open letter basically blasting ISPs as stiflers of net neutrality.
Broadly, we think that net neutrality shouldn't necessarily be taken to mean that different players in the video ecosystem can't ask other actors to pay for the privilege of using the infrastructure to deliver heavy video files.
We agree with streaming media analyst Dan Rayburn, who believes that Netflix's peering agreement with Comcast actually has little to do with net neutrality.
"The bottom line is that all three companies involved — the transit provider [Cogent], the content owner, and the ISP, — all three are to blame, because all three of them need to work together to come up with a business model that works in fair and reasonable terms," Rayburn said in an exclusive interview with BI Intelligence. Rayburn continued, "we’ve seen companies have problems like this in the past, and they’ve worked through them."
But before making sense of this thorny debate, and the economics underlying it, one has to understand how the video ecosystem works, and how video streams get from a specific content service to the end user.
Click here to download the charts and data associated with this report in Excel »
Click here to download the report as a PDF »
Internet Infrastructure And Video
Streaming video services like Netflix, YouTube, and Hulu have exploded as a share of total Internet traffic over the last few years, putting a heavy strain on existing Internet infrastructure.
Netflix alone accounts for nearly one-third of peak Internet traffic (defined as download traffic in the evening hours) in the U.S., according to the network intelligence firm DeepFields.
All this streaming video traffic must navigate a complex web of "network operators," or operators of Internet infrastructure such as traffic exchange nodes and fiber-optic cables. These operators include the array of players such as the transit providers, CDNs, and ISPs we described above.
One way to understand the relationship between these players is to think about how large they are and how much geographic reach they have.
For example, a small ISP that operates in a small metro area — say, Youngstown, Ohio — does not have much reach. They need to connect with larger network operators to bring their customers a full array of content, and Internet services — including the data-intensive video streams from Netflix.
So, network operators are classified on a three-tier system based on how expansive their reach is. Many of the most prominent players in the American streaming video ecosystem — like Comcast, Cogent and Level 3 — are Tier 1 players:
Tier 1 – Network operators that can access the entire Internet without transit or peering interconnections.
Tier 2 – Network operators that can access most of the Internet without transit or peering interconnections.
Tier 3 – Network operators that require transit or peering interconnections to access the Internet.
These categories help explain how important the Tier 1 players are.
Both sides of the video streaming equation rely on these Tier 1 operators.
For example, a Tier 2 ISP might establish a peering relationship with a Tier 1 ISP or purchase transit to provide its end users access to a streaming video service that it is not directly connected to.
Alternatively, a streaming video service may purchase transit or CDN services from network operators that already have relationships with Tier 2 or Tier 3 ISPs so that video subscribers on those ISPs will receive the best possible streaming experience.
Content Distribution Networks
Over the last decade, Content Distribution Networks (CDNs) have grown steadily in importance to become one of the most popular solutions for streaming video delivery.
CDNs specialize in delivering large volumes of traffic (this could be video, but might also be audio, or software downloads) over multiple ISPs, varying geographies, and piping it through infrastructure — including fiber-optic lines and Internet traffic exchange centers — controlled by still more companies.
In short, CDNs take the guess-work and heavy lifting out of the complex task of delivering video, in a reliable manner, to millions of viewers served by a variety of different ISPs.
In 2014, traffic from CDNs will account for nearly two-thirds of U.S. consumer backbone Internet traffic — traffic outside of the "last mile" ISPs — according to DeepField (pdf). (See chart, above.)
Akamai is the largest CDN in the U.S. based on its share of this traffic, according to our estimates, based on DeepField's (pdf) data.
Akamai accounted for nearly one-third of U.S. CDN traffic in 2013.
Limelight was the second-largest CDN, accounting for roughly one-fifth of CDN traffic.
Level 3 rounded out the top three, with 17% of U.S. CDN traffic in 2013.
Similar to transit, CDN services are priced based on three factors: monthly traffic volume, bandwidth, and the customer’s annual total spend (high-volume and high-spending customers will pay less on a per-unit basis). Greater speed/bandwidth is more expensive, as is more download volume.
Here’s a look at CDN pricing in terms of sustained bandwidth (a standard industry measurement, which is the bandwidth required at 95% of the monthly peak level) and traffic volume, according to data from StreamingMedia.com. Please note that our traffic price charts are meant to be illustrative. Pricing structure varies greatly, and is governed by complex contracts that cover a great deal of factors, including quality and performance metrics.
Also, the charts below do not mean that traffic is priced by bandwidth (i.e., download rate) in combination with volume pricing. Usually, one or the other method is used. (See charts, below.)
Here's a look at the price of CDN traffic by volume:
Content Provider-Run CDNs
Some content providers have built out their own CDN networks. These include Netflix OpenConnect and Google Global Cache.
In the case of Netflix, in a bid to increase performance and reduce the congestion on last-mile ISPs, the streaming video company is building out its OpenConnect CDN.
The program places Netflix servers within the final segment of a participating ISP's network. Traffic is then offloaded from the ISPs to the co-located Netflix hardware, meaning Netflix ends up handling its own traffic.
Netflix OpenConnect already accounts for 13% of CDN traffic as of 2013.
In this arrangement, Netflix controls both the hardware and software of its CDN, including storage architecture, caching frequency, and what video formats are stored on the CDN.
ISPs, however, are still responsible for the physical management of Netflix’s hardware, including powering, cooling, and supplying a connection to the rest of the ISP's network.
While custom-built CDNs, like Netflix OpenConnect, give streaming video services greater control over the performance and cost of content delivery, they typically come at a high upfront cost. Just like third-party CDNs, content provider-run CDNs must also establish peering relationships with other network operators.
Netflix, for example, would not have had to establish an individual peering agreement with Comcast, if it had continued to use a third-party CDN service instead of OpenConnect.
Transit
Network operators may also connect to each other through a paid transit provider, such as Level 3, Cogent, or XO Communications.
Transit contracts are negotiated on an ad-hoc basis, but might include several key factors:
Sustained throughput (i.e., sustained bandwidth): this is the key metric, which drives most of the cost of transit agreements. In the video streaming world, it's all about the cost of speed/bandwidth (measured in Megabits-per-second).
Traffic volume.
Time of use (peak time vs. off-peak time).
Transit contracts also typically include a service-level agreement (SLA), which guarantees the quality of service.
Many other factors might be involved, but as we imply above, the main driver of price in transit contracts is sustained throughput (or bandwidth). The transit industry uses a 95/5 pricing model — meaning that transit clients pay based on the measurement of the 95th percentile of their monthly bandwidth usage. In other words, the monthly price of transit is determined according to how much bandwidth the client requires at 95% of the peak level in a given month.
Here’s a look at monthly transit price ranges based on bandwidth, according to data from StreamingMedia.com:
The relationship between Netflix and Cogent is a notorious example of a transit relationship gone sour.
Prior to reaching a peering agreement with Comcast, Netflix used the transit provider Cogent to deliver a major share of its traffic to the last-mile segment of Comcast’s network.
But Comcast and Cogent began battling over the fee that Cogent should pay to access Comcast's network. Cogent complained that Comcast should not charge it for peering (they wanted what is known as "settlement-free peering," which we'll explain below), which obviously did not seem fair to Comcast, which was no doubt receiving a fire hose of Netflix traffic through Cogent.
The upshot was a direct peering deal between Comcast and Netflix.
Settlement-Free Peering
Settlement-free peering connects network operators on a contract-free, non-paid basis. Under this arrangement, network operators exchange traffic between one another at co-locations or Internet exchange points. These agreements reduce the added cost of paying for transit and negotiating the amount, and are mutually beneficial when participating operators exchange roughly equal traffic volumes.
Settlement-free peering agreements generally lack quality of service (QoS) guarantees — meaning that neither party is obligated to sustain a constant level of throughput or ensure consistent packet delivery.
For example, a Tier 3 network operator may establish a settlement-free peering relationship with a Tier 2 or Tier 1 player, with the expectation that both parties will exchange roughly equal traffic volumes. Such a deal would give the Tier 3 network operator broader access to the Internet without directly raising its costs.
Last-Mile ISPs
The last-mile of the Internet has been one of the most contentious parts of the net neutrality debate. Last-mile Internet service providers (ISPs) connect their subscribers to the rest of the Internet.
Cable or telecommunication companies are the entities that typically operate last-mile ISPs.
Comcast’s recent acquisition of rival Time Warner Cable would see the combined company account for well over one-third of broadband Internet subscriptions in the United States, or more than 32 million combined subscribers as of the end of 2013, according to Leichtman Research Group.
Comcast is the largest broadband Internet provider in the U.S., with more than 20.5 million subscribers.
AT&T is the second-largest, with nearly 16.5 million subscribers.
Time Warner Cable rounds out the top three, with more than 11.5 million subscribers.
A combined TWC-Comcast will have 32 million subscribers.
Streaming video performance varies greatly between last-mile ISPs.
Google Fiber, Cablevision, and Cox have consistently ranked as the best-preforming ISPs on Netflix’s monthly U.S. ISP Speed Index. Comcast and Verizon, on the other hand, both took highly publicized hits in streaming performance starting in the fall of 2013.
This dip in streaming performance has led some to speculate that last-mile ISPs are throttling traffic from bandwidth-intensive sites like Netflix.
But it's more likely that the decline in performance is the result of how Netflix's traffic is routed into last-mile ISPs.
Netflix dynamically routes traffic between different transit providers and CDNs depending on which network operator can provide the best performance at the time.
There is also some evidence that the streaming video company routes traffic depending on what device is used for playback, in order to ensure the best possible streaming performance on that device.
If Netflix routes traffic through a transit provider that has inadequate access to a last-mile ISP, Netflix subscribers on that ISP will experience poor streaming performance.
To improve streaming performance, Netflix may reroute traffic through a higher-capacity transit provider, use a third-party CDN, or establish its own peering relationship with the last-mile ISP.
Peering Agreements And Net Neutrality
Each side in the video traffic debate feels that the other should be responsible for the increased cost resulting from the surge of streaming video traffic.
Netflix believes that it should not be required to negotiate individual peering agreements with last-mile ISPs (even after doing so with Comcast).
In a post on the streaming video company’s blog, CEO Reed Hastings argued that ISPs should be responsible for the cost of fulfilling the traffic and bandwidth demands of their customers.
Hastings says government regulations should protect content providers from having to establish individual peering agreements with last-mile ISPs. "Without strong net neutrality, big ISPs can demand potentially escalating fees for the interconnection required to deliver high quality service," Hasting wrote.
Level 3, which provides both transit and CDN services, believes that last-mile ISPs must increase their network capacity in order to meet their customers' demands. "These ISPs break the Internet by refusing to increase the size of their networks unless their tolls are paid," Level 3 general counsel of regulatory policy Michael Mooney wrote in a post on the company’s blog.
Last-mile ISPs, however, believe that existing peering conventions provide a fair system for distributing the cost of delivering network traffic. "If there’s a cost of delivering Mr. Hastings’s movies at the quality level he desires – and there is – then it should be borne by Netflix and recovered in the price of its service," wrote AT&T senior executive vice president James Cicconi, in a response to Reed Hasting’s blog post.
Many content providers and network operators have refrained from entering the debate. It’s likely that smaller streaming video services are able to achieve adequate streaming performance through existing transit or CDN relationships. Meanwhile, larger operators, including Google and CDN provider Akamai, have already negotiated favorable peering agreements under the existing regulatory regime.
For now, the debate seems centered on Netflix, the large ISPs, and certain network operators like Cogent and Level 3, which feel that the ISPs are standing in the way of a good end-user experience at a fair price.
THE BOTTOM LINE
Getting streaming video from its source to the end user isn't a simple proposition. Video is too data-intensive to flow automatically in a glitch-free manner from a video company's servers to an end user watching a movie on an Internet-connected TV.
Transit providers like Cogent, Level 3, and XO Communications are the basic pipelines for moving video around.
Content Distribution Networks, or CDNs, such as Akamai and LimeLight are the specialists in making sure that the content providers are getting the best video performance at the best price.
Transit and CDN prices are negotiated on an ad-hoc basis, based on a number of factors, including traffic volume, time of use, and other determinants. But throughput (bandwidth/speed) is the key factor driving traffic prices.
In an effort to gain more control over the quality and cost of moving video around, many content providers have built out their own CDN networks.
The relationship between content providers, CDNs, transit providers, and the ISPs are governed by "peering agreements," which is just another way of referring to the contracts that stipulate the volume of traffic that each player is entitled to at different prices, including free and paid tiers.
Both content providers and ISPs have misrepresented the economics and mechanics of video streaming in order to advance their agendas in the context of the debates over fuzzy concepts like "net neutrality."
The disputes over video streaming fees, and the recent peering agreement between Comcast and Netflix, boil down to who should pay for the exploding popularity of streaming video services like Netflix, and at what quality of service.
Join the conversation about this story »