We have received the following grant application "Revitalize blogs.perl.org". Please leave feedback in the comments field by July 26th, 2015. If your comment does not appear in 24 hours, contact me at tpf-grants-secretary at perl-foundation.org.
Revitalize blogs.perl.org
Name:
Jeffrey Goff, Amalia Pomian
Amount Requested:
USD 3000
Synopsis
blogs.perl.org is in need of replacement. Evozon would like to offer a customized instance of PearlBee in its place, with source freely available on GitHub, and hosted on Evozon infrastructure.
Benefits to the Perl Community
In the last few weeks, blogs.perl.org has suffered from a number of authentication issues, affecting many people including the authors of this document. The current web host, MovableType has not addressed these issues, and placed the fate of blogs.perl.org in question.
While many solutions have been suggested, Evozon would like to offer its services. We would like to provide a blogging platform based on PearlBee customized to the existing blogs.perl.org's requirements. Evozon will donate hardware and developer time in exchange for hosting blogs.perl.org on our servers. The new blogs.perl.org code will be a publicly available fork of PearlBee on GitHub, customized to match the existing MovableType routes, and using the existing MovableType URLs so that extant links and search engine content remains in contact.
Since the goal is for this to be a community project, we'd like to integrate with Travis CI so that interested parties can keep track of progress and add their own comments and suggestions in the form of pull requests.
Most of the development activity is expected to be around matching schemas, and ensuring that existing users, whether they're logging in with their own username and password or with their OpenID account, can log in smoothly and be able to post and respond to postings.
Evozon will provide UI and JavaScript design services in order to give blogs.perl.org an appropriate look-and-feel.
Deliverables
Host blogs.perl.org source on a public GitHub repository
Integrate Travis CI with the blogs.perl.org repository for continuous (and visible) integration and testing
Bring blogs.perl.org up to feature parity with existing MovableType.org server
Migrate MovableType.org database to blogs.perl.org schema
Test alpha.blogs.perl.org in-house with special attention paid to existing users' login
Public alpha trial of alpha.blogs.perl.org again emphasizing existing users
Roll comments and bug fixes into beta.blogs.perl.org, again emphasizing login
Release final blogs.perl.org on Evozon infrastructure
Monitor blogs.perl.org after release, respond to issues as they come up.
Project Details
The new blogs.perl.org will be a PearlBee fork written on top of Dancer 2 and DBIx::Class, with a mySQL database imported from the existing blogs.perl.org, and session management provided by Memcached. Development and test deployment will initially be done on VirtualBox instances, possibly migrating to Docker containers later.
Internally we're using http://trello.com|Trello for our projects. we already have a board for PearlBee, but we'll create a separate blogs.perl.org board and give Dave plus the TPF grant manager(s) access so they can follow along and ask questions of the developers.
Having the application publicly visible on GitHub will also let others help spot potential security holes and other attack vectors, which is of course important because this is a publicly visible application that accepts several different kinds of authentication, displays user-composed HTML and reads/writes a database, all of these being possible attack vectors.
We should have two instances of the application running behind a software loadbalancer, so that we can stop one instance at a time in order to perform maintenance without interrupting service. Since the application has such a low database write rate, deadlocks and race conditions shouldn't be a concern. Evozon has data centers in the US, UK and Singapore should the latency accessing the site from overseas become a concern.
Unit testing will be performed through Travis CI, and automated UI tests through Selenium as they become necessary, although the good ol' Mark I eyeball will suffice for most purposes.
We'll start by performing a code review of the existing fork to make sure it conforms to current Perl best practices, and is suitable for deployment. When that's complete the next step will be to migrate the existing database into the new blogs.perl.org schema. Dave has assured us that he can share the existing MovableType password hashing scheme, so that we can continue to support existing users without asking them to change login IDs or passwords.
Once real users can log in, we'll turn our attention to making sure that existing blog posts are displayed with a new look and feel courtesy of Evozon's design team, with careful attention to UTF-8 encoding, code samples, syntax highlighting and comments.
After we've made sure that users can view their blog postings as they were meant to be seen, we have to make sure that users can view their old blog posts in the new editor and be able to save the edited versions in the original format, which will save hassle when we perform the final migration. I'd like to preserve beta users' posts, and making sure the editor is backwards-compatible is one important step in that regard.
Now, regular users should be able to log in, view posts and comments, edit existing posts and comments, and create new posts and comments. Since that's pretty much what a blog site is expected to do, at this point we can open it up to other testers on a limited alpha trial, with the usual understanding that their posts and comments may not make it into the final version.
We'll take feedback and comments from alpha testers, keeping in mind that lots of blogs.perl.org users live in the US, yet the servers are (for the moment) here in Romania. Hopefully there won't be visible latency, but it's something to keep an eye on. The comments we'll collect will lead to a beta release which will be open to all users, again with the understanding that their postings may not make it into the final release.
Then it's time for assessment, and getting ready for final release. If all works out, we should be able to obtain the final live blogs.perl.org schema, load that onto our local mySQL servers, deploy from Travis CI to live, and perform a final smoketest before throwing the switch.
Inch-stones
Of course, no battle plan ever survives contact with the enemy, but this should give an idea of how we want to proceed.
Set up infrastructure
blogs.perl.org fork
Travis CI
Trello board
outside access to testing systems
Migrate the database
Add foreign keys to MovableType schema
Import into DBIx::Class
Load MovableType users and blog entries into blogs.perl.org database
Mesh blog contents
View MovableType content in blogs.perl.org template
Read MovableType blog content into blogs.perl.org editor
Fix HTML editor to support blogs.perl.org content
Write blogs.perl.org content back into database
OpenID, Google authentication
Add OpenID/Google authentication to blogs.perl.org fork
Test with existing and new accounts, especially creating new accounts
External testing
Now that users can view blog posts, log in, create and edit posts, and log out....
Release alpha version
And fix bugs as they arise, making sure to add tests to Travis where possible.
Beta testing
Open beta.blogs.perl.org for testing
Watch site performance, especially load due to static content and lag to the USA where most users will be.
Fix bugs as they arise, and of course add regression tests to Travis where possible.
Cut off the beta after, say, 2 weeks.
Release time
Prepare rollback plan in case it's needed, probably as simple as pointing DNS back to the original blogs.perl.org but we'll discuss with Dave.
Take a final database dump from (now old) blogs.perl.org
Import onto the live machine
Bring up the memcached server pool
Bring up the first instance, making sure an instance can survive on its own without its twin.
Smoketest the instance, make sure the front page can be viewed and at least the admin and two (non-Evozon, not Dave) users can log in and edit posts)
Bring up the second live instance, kill the first live instance and make sure users can still edit posts
Bring the first live instance back up
Open the floodgates
Fix whatever problems arise, if any. Hopefully none.
Make certain monitoring is in place and live, let the application run and answer whatever questions people may have.
Project Schedule
I'd love to release beta.blogs.perl.org in time for YAPC::EU, but that's too aggressive of a schedule, especially given that we're moving offices in August. The three main issues I can see here are the database migration, user authentication and finally making sure the old blogs.perl.org content is compatible with the new system.
We'll have to start off making sure that the existing PearlBee fork can support the existing blogs.perl.org content, namely user administration, privileges, and the various URL routes. This will probably take about a week of poking around in the various corners of the application because any major changes or additions would require database changes, and those should be done before I start writing a migration process.
Once that's done, the database migration will probably take 2 weeks, mostly to be able to pick apart the old schema.User authentication is always a concern, and as such we'll schedule 2 weeks for whatever issues we find. Finally, 2 weeks to impedance-match the new HTML editor with the existing MovableType content seems a bit of overkill but should allow us some slop should other bits go over schedule. The actual content looks like it's mostly groomed HTML, so it should go smoothly, but we have to make sure that it displays in the new editor, and the new editor can save text.
Setting up the infrastructure here probably will take about a week at the most, and I can do that in parallel with checking over the application for missing features. Mostly it'll involve setting up VirtualBox instances, getting Amalia to create a new Trello board and give outside people access to it. I've (Jeffrey) been meaning to get a Travis CI account anyway, and will link that to blogs.perl.org.
Ignoring time out for YAPC and time for the office move, a tentative schedule looks like this:
Administrative tasks, checking PearlBee for needed features: Aug 2-8
Most of the admin tasks will be handled by others, I just have to keep a checklist.
Database migration - Aug 9-22
It really shouldn't take this long, but I'm factoring in searching for and adding foreign keys to a database that wasn't equipped for them, always fun.
User authentication - Aug 23-Sep 12
Lots of slop there, Dancer2 plugins should be able to handle most of the tasks. I'm factoring in time for getting keys from various parties and integrating the logos on the new front page, we'll have to have the designers come up with an appropriate UI.
Matching blog content to the new blogs.perl.org editor - Sep 13-26
Probably some slop here as well, because MovableType stores things in HTML and that should be easy to integrate.
Public beta - Sep 27-Oct 3
I'm not envisioning any major fallout from this, but then again Perl hackers can be a persnickety bunch. Hopefully they'll do their best to break our HTML rendering engine, and we can keep an eye out for that.
Release - Oct 4-Oct 17
The fun part of any schedule. I've expanded it to 2 weeks because we'll have to coordinate with Dave Cross in order to transfer the old domain and solve any authentication and security issues that arise from moving shared secrets between IP addresses and to new domain owners.
Completeness Criteria
blogs.perl.org on GitHub and integrated with Travis
blogs.perl.org feature complete
Live schema imports via DBIx::Class into blogs.perl.org with no errors or constraint violations
Blog text, code snippets, quoting and comments display in clean UTF-8
Admin and test users can log in to their account through direct access
Beta users can log in through OpenID and whatever other IDs we support at rollout
Live users can view content, log in with their existing ID, create and comment on content
Bio
Jeffrey Goff is a long-time member of the Perl community, CPAN author, speaker at Perl conventions worldwide, active on IRC for an embarrassingly long time, one of the original Perl6 release managers and an active Perl6 core contributor. Before moving to Evozon.com and taking over buitinperl.com, he worked on release teams for the website builder MoonFruit and architected a Hadoop pipeline for TagMan.
Amalia is has been involved in the Perl community since starting the local Perl Mongers group, Cluj.pm, as an initiative of the Perl Department at Evozon. She has been organizing the meetups of the Cluj community since March 2012, and also attending worldwide Perl events, like YAPC::Europe, London Perl Workshop, or FOSDEM, either as a regular attendee or volunteer.
Her involvement in the Perl Community has been officially recognised, in December 2014 being the proud recipient of a White Camel Award.
In addition, she is also involved in the development and growth of builtinperl.com.