D2Moddin was a service I wrote to bring Dota 2 custom game modes to the public from May to June 2014. The game modes were based on the game’s seasonal game modes, and were created initially without the support of the developer, Valve. After the massive demand demonstrated by the service, Valve ultimately released the Workshop Tools Alpha far earlier than expected, including an unannounced Source 2 distribution, much to the surprise of the Source and Dota communities. Their in-game custom game system eventually went on to replace D2Moddin.
There was some work towards a system to host the new custom game lobbies on the D2Moddin server network, but I stopped working on it due to the massive resource requirement of the optimized Source 2 engine.
History and People
I began the project after seeing the CustomDOTA video about Fight or Flight. I was looking for a project to work on at the time, and decided to create a platform to bring these game modes to the public. I never expected it to grow to the scale it eventually reached.
The initial release was developed using Meteor which offered rapid prototyping and reactive development at the price of insane performance problems at scale. Read on to find out the technical background of the project.
The initial release was made under the Masters of Development banner, a group run by Jicyphex. I later left the group along with the developers of the original game modes for various reasons. I was the sole developer at that time. The overwhelming response to the release resulted in a complete crash of the optimized Meteor app.
I spent around a week attempting to scale the app, rewriting several parts to no real avail. The result was incredibly expensive over-the-top hosting solutions for what was ultimately a somewhat simple app at that point. It quickly became clear that a full rewrite was necessary. Using different technologies that allowed me to develop every single component of the system from the ground up, I re-wrote D2Moddin in a three day period, while streaming my development, often in nine to ten hour periods.
The re-release went far smoother than the initial release of the platform. Over time I added more features to the system, and several contributors joined the project.
Finally Valve’s system was announced and implemented in game, in their Dota 2 Workshop Tools announcement.
Contributors
I was the lead developer for the project, but members of the community contributed with GitHub issue reports, and in particular, I would like to thank the two contributors who joined the project later in its development:
Ilian000 began with contributions to the mod manager and later became a core developer of the project, ultimately re-writing the Windows UI for the mod manager.
SuricateCan dabbled in just about every area of the system. His most prominent contribution was the core code of the matchmaking and ELO system.
jimmydorry created the excellent D2Moddin Stats part of his GetDotaStats website.
Medowar was the admin of many of D2Moddin’s servers, maintaining most of the European network.
Alex Pronin did a lot of art for the site.
AD7 concepted the re-design that was in place about 2 weeks before the site ended.
Statistics
About halfway through its lifetime, D2Moddin added a stats API, which jimmydorry used to create the D2Moddin Stats page of the GetDotaStats website. It is a good representation of the popularity of each mod and our overall uptime and player count over time.
Some general stats:
Unique Players: Approx. 174,000
Max Concurrent: Approx 1,700 players
Avg. Concurrent: Approx 1,000 players
Average Concurrent Minimum: Approx 600 players
Game Modes: 12 when the project ended.
We had around 13 different servers that were on the network over time. The system load balanced players across the network to support the playerbase.
Reddit Posts
Here are the posts I made on Reddit regarding the project:
July 2014: D2Moddin Out of Beta
June 2014: Initial Re-release of Beta
Rollout and Q/A
May 2014: Initial Announcement
The initial announcement was published by Jicyphex and backed by DotaCinema. The combined Reddit hype and DotaCinema followers resulted in a massive amount of traffic. See the details below on the flawed design decisions and my resulting rewrite.
Stream Clips
Sadly there are only two stream clips from the three days of streams.
Developing D2Moddin 1 hour clip
Playing Reflex on the new system just after it was completed.
Technical Details
In every implementation I used MongoDB as the database. This was my first time working with NoSQL and Mongo, but I enjoyed it and will be using it for future projects.
I tried just about every hosting PaaS out there, from Heroku to OpenShift to Modulus. Nothing worked better than bare-metal hosting. I used DigitalOcean droplets for hosting the web and master server.
Meteor.JS Implementation
The original implementation was written in Meteor.JS. It hosted both the website and the master server. It never worked properly for more than 100 players, and discouraged me from using Meteor for any large scale projects until it matures some more in the future. After being forced to learn other web frameworks, however, I enjoy the level of control granted by other frameworks more than Meteor’s start-to-finish easy solution. Too much is abstracted here for finely tweaking the system in production.
.NET Master Server Implementation
I wrote the re-write of the server implementation using C#. Originally I wrote the system using WebSockets.NET but later discovered issues with it. After rotating through a few other options I landed on XSockets.NET which worked very nicely for low level communication. The team has recently released XSockets 4 which I am now using for other projects. It supports RPC, WebSockets, WebRTC, and long polling fallback and is a great solution for communication between web browsers and .NET apps, as well as between .NET and .NET apps.
The system tracked users, lobbies, browsers, and servers. It kept state on everything and was responsible for authentication, distributing updates, interfacing with Amazon S3 for file storage and download authentication, and more. It also tracked load across the system and balanced players onto the available server hardware.
Angular.JS / Node Web Implementation
I needed a quick replacement for Meteor’s reactive DOM and easy templating. Angular was a natural choice. I was already working in Node.JS with Meteor, and stepped down to a basic Express.JS and Passport.JS setup for the server. Passport handled the authentication, using MongoDB for the backend.
The web server served the Angular app and managed authentication. It did not interact with the server network like the old Meteor system did.
The Angular app used the simple ng-route router, and Bootstrap for basic styling. I also used PNotify for notifications.
D2MPServer: The Server Manager
While the networking behind the server manager changed between master server implementations, the core code changed very little from the initial release.
The server manager was responsible for:
Building the folder structure (with just the EXE as the base)
Download SteamCMD
Use SteamCMD to bootstrap Steam and the game directories
Download and update Dota2 with SteamCMD
Download the latest reverse engineered Source dedicated server
Patch the Dota 2 client to work as a dedicated server
Track addon versions and download the latest game mode updates (later done without restarting).
Launch SourceDS server instances on demand of the Master server.
This part of the system worked so well that we could just extract the base compiled program into a folder and run it. It would build the entire setup automatically as well as download the latest addon versions. This allowed us to quickly add new servers and scale up without restarting the system.
The Mod Manager
Note: the source can be found on GitHub.
D2Moddin was a step forward from previous services like D2Ware. While D2Ware allowed the Dota 2 client to simply connect to modded Dota 2 servers, it never allowed for client-side file changes or additions. D2Moddin allowed for custom models, maps, textures, animations, characters, allowing for rich experiences rather than simple game modifications.
This required a client-side program that would move files around for the players. As I designed the system with the intent of ease-of-use and simplicity, the manager would have to do everything for the user automatically, and require little to no manual actions to work properly.
Furthermore I wanted to dispel fears of a virus inside the program, so I built it to be as temporary as possible. The launcher EXE was very lightweight and deleted itself after being run. It simply downloaded/updated/ran the actual manager which sat in the user’s temporary directory. This allowed it to run with no administrator privileges and could be removed with a single delete operation.
The program sat in the system tray and would exit if the user was disconnected from the site for more than a few minutes.
It would perform the following operations:
Track addon versions (in the addons directory in Dota2).
Patch the gameinfo.txt file to add the D2Moddin addon directory to the game’s search paths. This was not used during any official matchmaking / lobby games, only during custom games.
Download compressed addons and extract them to the addons directory
Inform the master server about the status of the player, including the Dota 2 client state as well as the mod installation status.
Send the connect command to the Dota 2 client.
When a user joined a lobby the manager would:
Move the files into the active D2Moddin game mode directory.
Launch Dota 2 in case it was not already running.
When the game started the manager would:
Automatically send the connect command to Dota 2 through steam:// URIs.
Re-send this command whenever the user clicked the reconnect button.
Originally everything was done through the Web UI but later Ilian added a in-app UI for managing mods.
Conclusion
This was my first foray into large-scale service development. I learned a lot about server infrastructures, public relations, scaling networks, developing with speed in mind, and web frameworks. All in all, I enjoyed the experience and will be looking to do fun projects like this again in the future! Thanks again to the community for supporting my work.
Quantum / Paralin