We all remember the mythical good ol' days, when no access points had passwords on them, contracts weren't required, and the entire city was covered with them so you never had to pay for your own internet access? Right? Right? How our memories are so convenient. If you're just arrived to Japan from overseas, you will notice that:
There is plenty of Wi-Fi in Japan, but not much of it is unencrypted (a good thing, actually), and even less is free access or registration-less.
Most of the Wi-Fi access points in Japan are not for laptops in the coffee shop. They're for smartphones to augment their access speeds and data capacity. They're also for giving data access to shops, bars, and restaurants whose patrons can't get a cellular signal easily (many bars and restaurants are underground and/or have very radio-wave resistant walls.
I often read from people visiting Japan, hoping to avoid paying for net access while they vacation, that they are disappointed about the lack of "free Wi-Fi" in Japan. However, I travel to New York City or San Francisco or other big internet savvy cities, the "public" or "free" Wi-Fi is often poorly maintained, slow, non-responsive, and unusable. Unfortunately, it's a bit of the "you get what you pay for" dilemma: the commercial Wi-Fi spots that are locked down tend to be fast and reliable because they have to maintain a service level for the paying customers.
How did Japan get this way? How does it work? Is it any good? To start, let's begin with an ...
data consumption: D > S
Intro: 3G/LTE Demand Exceeded Supply
If you haven't gotten the memo, smartphone technology has saturated the mobile airwaves. Despite Japan's advancements in 3G and LTE, what the provider manages to provide, the smartphones instantly use up.
Softbank, who entered the smartphone business early in Japan with the iPhone 3G, was often criticized for its supposably substandard bandwidth available for it's megabyte slurping Apple toy (Japanese net users have never had to deal with AT&T and the iPhone, so their standards are higher). People complained and complained how the minute Rival X offered the iPhone, they would switch over.
I know, as I was one of the first early adopters of the Japanese market iPhone. The bandwidth crush would be so bad at times that using streaming applications such as YouTube would be impossible (buffering for up to ten seconds for every few seconds played) during "rush hour" — before 10am and from 5pm until the late in the evening, because that was when everybody else was using there smartphone.
So I used Japan's regulated MNP (mobile number portability) service to switch to KDDI with an Android phone. Sure enough, KDDI was much, much faster. At least at first, when I was one of the few on the network. Now, KDDI, which was supposed to have a much more robust network than Softbank, has the same sort of traffic jams that people complained about on Softbank now that it has a rich CDMA lineup of iPhones and Androids. I then MNPed over to Docomo. First on 3G. Then with their new LTE network. Same situation. Fast at first when smartphone adoption was low, but now docomo also suffers from the similar bandwidth scarcity slowness as the other networks during rush hour commuter times.
light waves > radio waves wrt bandwidth
Wi-Fi Connected to Landlines to the Rescue
While counterintuitive ("radio waves don't take up any space, why not just allow more radio waves?") to the layman, the problem is that the airwaves are actually scarce and precious resource, and there are literally hundreds of millions of devices in Japan alone competing for it. And they're putting ever more pressure on the airwaves as the devices get bigger (bigger batteries, meaning they're on and used more often), and more complex (applications that use more and more data).
Wi-Fi is almost (but not always -- see below) connected to a broadband line. Broadband, while also under pressure, is not limited by airspace in that networks can expand capacity and speeds by simply adding more lines.
Thus, the mobile carriers in Japan decided that it would be best to encourage their smartphone users to use Wi-Fi for their data instead of 3G or LTE as much as possible. This way, it would be a win-win; pressure would be taken off the airwaves, and users would get much less latency (3G and even LTE, which is technically not 4G, have serious startup latency compared to connected Wi-Fi) and probably much higher speeds.
But "encourage" does not mean "give away": a lot of the providers weren't so gracious as to often the service for free to non-contract devices — especially to phones using competing carriers. But the Wi-Fi plans were added as a subsidized cheap add-on to a data plan, and they often merged their existing Wi-Fi services, and partnered & formed alliances with existing commercial public Wi-Fi providers (ex. FON and UQ for Softbank, FLETS SPOT for Docomo) to make their coverage as comprehensive as possible.
not all Wi-Fi portals give your browser cookies
Baking the Solution into the Device
One of the common criticisms of Wi-Fi by novices, especially now that using them with security by default is no longer considered wise, is that they're too hard to configure. Asking an average non-power user to not only set up the SSID and WPA passwords, but also authenticate every time they wanted to use a hotspot would guarantee minimal adoption.
Softbank first attempted to tackle this problem by using iOS's built in "Profile" feature in Apple's iPhone. Apple does not allow after-market third party additions to the iOS firmware, and they even scolded Softbank for allowing its salespeople to "helpfully" (meaning often without the customer asking) configure their iPhone by installing Softbank/Yahoo! Japan applications changing the default search engine. Customers had to go to Softbank's site on their phone and manually download and install the profile. Doing this would preprogram the Wi-Fi with all of Softbank's existing (including legacy) Wi-Fi SSIDs and WEP passwords. However, you still had to type your Softbank iPhone "i.softbank.jp" email address and password in the capture portal to connect to the outside world. As the original versions of iOS did not allow for much background internet activity, this was not that debilitating when you would be near a Softbank hotspot yet not logged in; no background applications would lose connectivity due to the switchover from OTA to (capture portaled with no outside access) Wi-Fi.
The reality is that hardly any Japanese iPhone users installed that special profile. Softbank and other carriers learned from this initial early adopter experience. If they were going to get the average smartphone user to use these "public" access points, the setup would have to be painless, and no extra installation of applications necessary.
Fortunately, open source Android allows carriers to do this.
a giant Galápagos turtle
Galapagos Japanese Android smartphones & Wi-Fi
Android software is commonly modified by the carriers and the hardware manufacturers with add-ons. In addition to Japanese fonts and input methods that are often better than what the stock builds offer, Japanese carriers have been including Wi-Fi connection daemons in their firmware that can aggressively and automatically switch to a supported Wi-Fi spot whenever one is in range.
These applications are built into the firmware (they're closed source), and they can be updated from the Play application market from the carriers exclusive channels (A docomo phone cannot see or download the au and SoftBank Wi-Fi applications, for example)
The setup options are designed to be as painless as possible. In KDDI/au's case, it will extract the "login id" (the "au id") from the phone automatically (which is the phone's phone number), so all the user needs to configure is the password (which can be as simple as a 4 digit numeric PIN).
How well do they perform? I've had a chance to play with three carriers solutions on a variety of phones over the years. While I need to caveat my information with the disclaimer that these apps are constantly being updated so my information may be a little out of date, I have used all three solutions within the last three months.
I'll rate them from worst to best.
Bronze: KDDI/au
KDDI/au's software has the fewest amount of access points (most of them are inside the KDDI/au shop stores), and the least reliable access software; even when inside a au shop, the software would often fail to "hold on" to a connection or would fail to negotiate with a KDDI/au access point. Of course, during this negotiation phase, your phone will be unusable for about ten to fifteen seconds, wasting your time.
The other problem with KDDI's software is that of the software solutions here (none are perfect), it is the least likely to "play nice" with the built-in Android Wi-Fi auto-connect logic. It is only through trial and error that I managed to get it to work well with the long list of password protected and non-protected list of SSIDs within my device.
One unique feature of KDDI right now is the availability of its access points on some city buses. As to whether this will provide an advantage over using your own phone radio remains to be seen.
KDDI's software is the easiest to configure of the three reviewed here. It's almost "install and go", if it's not already baked into the firmware.
I have not had a chance to test out how well the KDDI iPhone connects to their network — only Android Gingerbread based devices.
Silver: Softbank
Softbank has made a lot of deals with affiliates such as FON and other Wi-Fi providers, and its software connects to them all. Softbank was the first to innovate in the "supplement their mobile with Wi-Fi" area out of necessity: they were the first to carry the iPhone in Japan, and the Apple users soon overwhelmed their capacity, especially during commuter rush hours.
In addition to affiliates, they also have a lot of other legacy connection points due to their previous incarnations and deals with Vodaphone and J-Phone.
Softbank's weakness is that some of their affiliates (which their software knows about and will attempt to connect to any of them indiscriminately) are of questionable quality and speed — especially the user crowdsourced access points. You are often better off using your own mobile radio than the "Softbank" Wi-Fi access.
Fortunately, Softbank allows you to configure as to whether you want to connect to FON access points (the least reliable) separately from the Softbank access points.
Gold: Docomo
Docomo doesn't have as many access points or affiliates as Softbank. But when it chooses to partner with affiliates, it seems to choose partners that maintain a high service quality level. For example, docomo tends to work very will in the Tokyo Subway (not the Tokyo metro). It works well on the Narita Express as well.
Docomo's software also allows you to connect to overseas Wi-Fi as well, called "World Wing Wi-Fi". This is a separate, non-free service with separate credentials that I have not had a chance to evaluate. I do not know how good the coverage is or how well it works.
Docomo's drawback is that the software can be a bit hard to configure if you try to avoid carrier-lock in and don't use any carrier services. Initially, I found configuring the special ID and passwords can be trickier than the other services. Docomo goes overboard on security/configurability where it wants a separate user id and password for a lot of its services. Sometimes it difficult to remember which docomo username/password/pin corresponds to which service.
Docomo does has a nice mode in its software called "home mode" where you can tell it the SSID and credentials of your "home" Wi-Fi and it will treat this as special; it will allow you to toggle auto-connect (even when you do not auto connect to Docomo networks) to your home network and gives you a visual indicator with its on-home screen widget.
Weaknesses of all solutions and proposals for improvements
The logic in the auto-connect for KDDI/au and Softbank could be much improved. The problem with the aggressive auto-connect is that when you're walking down the sidewalk outside, you will inevitably walk by a few cafés and stores that have carrier-sponsored or carrier affiliated setups. The problem is the switchover: often times the switchover causes a disconnect/abort of whatever you happen to be downloading or streaming.
If you're browsing web pages, this is not a problem as the cycle for web pages is download burst and disconnect, a long period of net inactivity while you read, then a click on a link for another burst.
However, if you're watching a video or doing something that requires a connection, often the switchover from mobile to Wi-Fi causes whatever your doing to stop. Depending on the intelligence of the application, this may or may not require you to restart whatever you were doing manually; even worse, you may have to restart from the beginning.
And if you do restart with the new Wi-Fi connection, often you will only have that connection for about 10 seconds before you walk out of range and have to restart the connection resume process again.
The solution is to either sense that you're not simply "passing by" the Wi-Fi spot. Either a heuristic or manually configurable setting should allow you to tell the software to only connect if you're staying near the Wi-Fi spot for some time (for example, eating/drinking at the bar or restaurant with the Wi-Fi spot).
Softbank's network of Wi-Fi spots uses a lot of FON and user contributed devices ("crowdsourcing"). While this gives you access to a LOT of spots and availability, the quality of these "user maintained" access points is often questionable. The problem is some of these routers may be on but are often left unplugged into a landline broadband or mobile cellular device. Thus, the device connects yet it's a "road to nowhere" from net connectivity. These "bogus" spots are not deliberate. Some people physically disconnect their routers from the internet and forget to turn the router off. Others are simply neglected.
The solution here is much harder; the solution would be to do what Android devices do: Android devices ping Google occasionally to see if the connection it has made (when the Wi-Fi meter turns from black to gray) can get to the outside world, and then turn the meter from gray to blue or green when a connection to the outside world can be confirmed.
Unfortunately, this isn't something that user-level software can do without disrupting the existing mobile connection; standard Android builds don't yet allow for the use of the mobile broadband radio and the Wi-Fi radio at the same time.
A lot of bars and restaurants in Japan are located underground or in buildings that have difficulty getting a mobile cellular signal. These restaurants will provide, as a service to their patrons, a Wi-Fi router so their customers phones can use the internet via their Wi-Fi. What they'll do is put the Wi-Fi router near the stairwell or entranceway of the restaurant and connect this router to a mobile router, essentially "bridging" from where the mobile cell signal is reachable to the customers tables inside with Wi-Fi.
The mobile carriers encourage this solution, giving the restaurants stickers to place on their entranceway to place alongside their "Visa and Mastercard and JCB accepted" stickers; patrons seeing this know they can relax without having to sacrifice (data) connectivity indoors.
The problem with this solution is often the modern phones (LTE) the customers have are faster than the speeds the restaurant's mobile router provides (often just 3G). And even if the restaurant does have LTE or faster, multiple customers using the same access point mean the access point is often quickly saturated.
Furthermore, the "bridge" is often located in a sub-optimal spot (not near a place with a good signal) and doesn't have "full bars", meaning the connection may not be full speed — or worse, drop out occasionally.
The solution to this is to have the software judge the quality of the Wi-Fi provided and the speeds it provides. If your phone can get a better connection or faster speed with its own cellular radio, it should use that instead of the Wi-Fi spot. Unfortunately, this solution is similar to the previous solution, in that it's not something that user software alone can solve — a modification would be needed to the Android net stack to allow the Wi-Fi switching software to probe and test the connection while the cellular radio was still doing its thing for the user.
All of the solutions, depending on their software version, have difficulty "cooperating" with the built-in Android Wi-Fi auto-connect software. Sometimes the daemon fights with the onboard daemon as to which one should take priority regarding which site to connect or disconnect to. The is especially true with Jellybean Android, which, similar to iPhone, understands the concepts of "login capture portals" and provides special support for them in the status bar and browser.
As KDDI, Softbank, and docomo all make their own custom versions of Android, they should work on their version of Android to make sure that their solution is better integrated and their Wi-Fi auto-connect software works more in harmony with the built in Wi-Fi auto-connect software,