Archive for October
New page
== October 2012 ==
=== Technical Operations ===
* Redis/Memcached
** Deploy limited php-redis on the apaches, and all new sessions are being written to a redis instance (mc1). [Tim/Asher] [Done]
*Swift
** start migrating swift backend servers to new NAS1 server [Mark] [done]
** Evaluate Ceph [Faidon/Asher/Aaron] [on-going]
* Commons infrastructure
**migrate ms7 (commons) to Nas1 [Done]
** replicate Nas1 (Tampa) to Nas2 (Ashburn) [done]
*Parser Cache
** build 2 more parser cache instances@ Tampa [Mark] [Done]
* Deploy Varnish on upload@Eqiad to replace squid [Mark] {done]
* Deploy 4 new servers for bits@ESAM [Mark] [done]
* Deploy Wikidata
=== [[Wikimedia Features engineering|Features]] ===
* [[VisualEditor]]/[[Parsoid]] (Trevor, Roan, Rob, Gabriel, Subbu, MarkTraceur, + Wikia)
** API finalization/debugging ''[IN PROCESS]''
** Iteration on Visual Editor for Dec release ''[IN PROCESS]''
** Parsoid testing framework for semantic/superficial regressions ''[IN MAINTENANCE]''
** [http://parsoid.wmflabs.org:8001/stats Node.js parsoid to completion] ''[ONGOING, 75% exact, 93% semantic]''
** experimentation into C/C++ version ''[ON HOLD UNTIL JAN]''
* '''New Pages Feed (fka [[Page Triage]])''' (Kaldari, Benny)
** Additional socialization and public announcement/launch '''[DONE]'''
** Descriptive statistics on NPF (revised dashboard on toolserver) [Dario] ''[IN PROCESS]''
** Bug fixes (no more feature development) ''[ONGOING]''
* '''[[Echo (Notifications)]]''' and '''[[Flow]]''' (Kaldari, Benny, Werdna, Luke Welling, Preilly, Aaron S)
** re-deployment of Echo prototype for Mediawiki.org [Werdna/Vibha/Kaldari] ''[NOV 1]''
** issue with relative timestamping (WSTimestamp::GetHumanTimestamp and Language::prettyTimestamp patch) ''[IN PROCESS]''
** Continued development of Echo prototype for Mediawiki.org [Werdna/Vibha/Kaldari/Benny] ''[ONGOING]''
** Selection of initial use cases [Vibha/Fabrice] ''[DONE]''
** UI and bugfixing of Echo [Krenair] ''[ONGOING]''
** Global user changes [JamesF] ''[IN PROCESS]''
** (see Performance engineering for infrastructure prep work:)
*** queueing abstraction '''[DONE]'''
*** queueing for Echo ''[IN PROCESS]''
*** [[Database sharding]] & Flow
* '''[[Article feedback]]''' (Matthias)
** (most delayed because of WikiVoyage resourcing)
** Oversight checklist (to help reduce the volume of oversight requests) '''[DONE]'''
** Continued AbuseFilter work '''[DONE]'''
** Conversion analysis: '''[DONE]'''
** performance/db related changes to AFTv5 queries before scaling, + sharding (see '''[[Database sharding]]''') [Matthias] ''[IN PROCESS]''
* '''[[E3]]''' (Ori, S, Luke)
** Clicktracking/Analytics
*** Event tracking live ''[DONE]''
*** EventTracking schema filtering ''[IN PROCESS]''
*** Clicktracking sunset ''[Scheduled for 10/31]''
*** Funnel tracking of Community Portal ''[Started]'' (will be ongoing)
** [[Post-edit_feedback|Post-edit Feedback 2]]
*** Deployed to production '''[DONE]'''
*** Data analysis once experiment is complete ''[DONE]''
** [[ACUX|Account Creation User Experience]]
*** Deployment of first test to production '''[DONE]'''
*** Deployment of second test to production (client-side validation) '''[10/24]'''
*** Minor UI tweaks ''[IN PROCESS]''
*** Data analysis once experiment is complete ''[IN PROCESS]''
** [[Micro_Design_Improvements|MicroDesign]]
*** edit window templates ''[IN PROCESS]''
*** (see [[Wikimedia Foundation Design|Agora]])
* List of other projects on radar
** [[ResourceLoader|ResourceLoader2]]
** [[Article creation workflow]]
** [[LiquidThreads 3.0]]
==== Fundraising ====
* Data discrepency error/regressions ''[IN PROCESS]''
==== Multimedia ====
* '''[[TimedMediaHandler]]'''
** Bugfixing of player on test2 ''[https://bugzilla.wikimedia.org/show_bug.cgi?id=27699 DONE]'' (Michael Dale)
=== [[:m:Mobile Projects|Mobile]] ===
* Beta features
** left nav '''[DONE]'''
** new typography '''[DONE]'''
** login '''[DONE]'''
* Prod
** left nav '''[DONE]'''
** new typography '''[DONE]'''
* promote wikipedia beta app to stable '''[DONE]'''
* mobile friendly resource loader '''[DONE]'''
* dynamic sections '''[DONE]'''
* pickup any remaining redirects [office, chapters, ...] '''[DONE]'''
* Launch Saudi '''[DONE]'''
==== QA ====
* Created basic Regression Tests [http://www.mediawiki.org/wiki/Mobile_Browser_Regression_Tests Mobile Browser Regression Tests ]
* Start Implementing Watir-based browser Mobile Testing [http://watirwebdriver.com/mobile-devices/ Watir-webdriver Mobile Testing]
* Continue working with Chris and Zeljko to use CI framework that the Platform QA team is setting up
* Continue to work with the Product team to Test prior to each deployment
=== [[Wikimedia Platform Engineering|Platform]] / Dev-Ops ===
==== MediaWiki Core ====
* Datacenter migration (see TechOps section above)
* [[Git/Conversion]] (Chad/Antoine)
** Mirror mediawiki-core to GitHub [Chad] '''[DONE]'''
** Have last of extensions out of SVN and into Git ''[Done]''
** Mark extensions directory as read-only [Chad]
** Finish replicating all repos to GitHub '''[DONE]'''
*** Begin researching how to get pull requests back in
* Swift
** See ''Swift'' section in Ops
* '''[[MediaWiki 1.20/Roadmap]]'''
** Bi-weekly deployment cycle continues
*** [[MediaWiki 1.21/wmf1]] [DONE]
*** [[MediaWiki 1.21/wmf2]] [DONE]
*** [[MediaWiki 1.21/wmf3]] [DONE]
* '''[[Continuous integration]]'''
** Integrate zuul into Jenkins pipeline [Antoine] ({{Bugzilla|40681}}) [IN PROGRESS, finish in November]
** Move Jenkins job setup out of web interface into Git using Jenkins Job Builder [Antoine] [AFTER ZUUL]
** Upgrade Gallium to Precise [(someone in Ops) + Antoine] [DONE]
* '''[[Admin tools development]]'''
** Deploy global AbuseFilter improvements [CSteipp] (https://gerrit.wikimedia.org/r/#/c/23509/ / https://gerrit.wikimedia.org/r/#/c/23514/ )
** Build and deploy CentralAuth mass account locking [CSteipp]
* '''[[Wikidata deployment]]'''
** Code review of extensions [Sam, Tim, Chris] [DONE]
*** Need to review DataValues
** Deploy MediaWiki version with ContentHandler branch merged in [Sam] [Done]
** Create and deploy wikidata.org [Sam,Chad] [DONE]
* '''[[Site performance]]'''
** Make libxml use php memory allocator [Tim]
* '''[[Lua scripting]]''' [Tim]
** Benchmarking Lua templates relative to Wikitext counterparts [Tim]
* '''[[Wikivoyage migration]]''' [Chris, Sam]
** Deployment to en.wikivoyage to Beta [DONE]
** Deployment to other langages [ongoing]
==== QA ====
* [[Beta cluster|Beta labs]]
** Continue refining [[Beta cluster|beta labs]] performance [Antoine/Chris/Željko] [ONGOING]
* Begin work on Watir-based browser testing framework
** Create at least one useful end-to-end test for important Wikipedia feature, with automatic execution (Jenkins-driven) and results reporting [Chris/Željko]
** Create a list of targets for browser testing according to priority
* Continue integrating Mobile QA with overall QA/testing activities
==== Analytics ====
See also: [[Analytics/Roadmap]]
===== Kraken =====
* Set up Maven / Somatype Artifact Repository (continuous integration) [dsc]
** WMF Maven parent pom [dsc]
* Puppetize Kraken [otto] ''(Ongoing)''
* Set up JMX monitoring -- needs to be our LAN [otto + dsc]
** Hadoop Ganglia Monitoring [otto] (Oct) [DONE]
* Get Storm set up [dsc + otto]
** Start work on ETL topology [dsc]
* Hardware reinstallation -- Depends on Ops [otto]
* Get to consensus with Ops regarding logging of the firehose [dsc + otto] [DONE]
**
Research needed: test running cli JVM producers does not cause extra load [otto]
[not doing]
===== Limn =====
* Refactor charting to use d3 [dan + dsc]
** Feature Parity with Dygraphs (plus bugfixes, etc) (Oct)
* Bugfixes (like Save-As, UI Error Notifications, ...) [dan] (Oct)
* Improve Limn wiki, docs, & guides (esp those shameful screenshots) [dan] (Oct)
* UI support for remote datasets via proxy [dsc + dan] (Oct)
* Deploy reportcard / gerrit-stats on stat1001 (aka, "the Debian packaging discussion") [otto + dsc]
* Support the Global Dev dashboard [evan] (Ongoing)
* Support the Gerrit Stats dashboard [diederik] (Ongoing)
===== Legacy Log Collection =====
* Add support for new domain names in webstatscollector (blog, etc) [diederik]
* udp2log filters
** Update filters for Wikipedia Zero [otto] (Ongoing)
** Filter by X-Carrier headers. [otto + asher + diederik]
** udp-filter to filter by http status. [otto]
===== WikiStats =====
* Reduce backlog regarding Wikistats traffic (squid etc) scripts [stefan]
* Repair data errors in wikistats, and add process for checking data integrity [ezachte] (Sept)
* Make wikistats more robust (MoM validations) [ezachte]
* Add Blackbox testing to WikiStats [diederik + ezachte]
===== Infrastructure =====
* Access/support requests for stat1, stat1001 [otto] (Ongoing)
* Migrate Reportcard off Labs onto stat1001 -- reportcard.wikimedia.org [otto + dsc] (Oct)
* Maintenance of oxygen/emery/locke [otto] (Ongoing)
===== Data Releases =====
* Publish [http://reportcard.wmflabs.org Monthly Report Card] -- deal with monthly data processing irregularities, perform correction/validation [ezachte + diederik + dsc] (Ongoing)
* Create Data Release Practices Task Force [diederik]
* Start pushing datasets to AWS [diederik]
* Finalize scripts to massively compact dammit.lt data [erik]
** Blogpost about what awesome stuff you can do with this [diederik + ?]
==== Engineering Community Team ====
[ok to remove, updated/moved to Nov]
=== [[Wikimedia Language engineering| i18n/L10n]]===
* '''[[Internationalization/Localization]]'''
** '''[[Universal Language Selector]]'''
*** Version 1 development ''[IN PROGRESS]''
*** User Experience / UI testing ''[ONGOING]''
*** Technical documentation ''[IN PROGRESS]''
** '''[[Project Lassi: Translate UI/UX Improvements]]'''
*** User Experience / UI testing Round 1 '''[DONE]'''
*** UI design finalization for development ''[IN PROGRESS]''
*** Prototype development ''[IN PROGRESS]''
** '''[[Translate API]]'''
*** API definition ''[On Hold]''
** '''[[Central Notice-Translate integration]]'''
*** Code changes '''[[DONE]]'''
*** Code review/testing: The i18n team has put this integration project on hold until availability from fundraising engineers to complete code review for deployment. ''[[ON HOLD]]''
*** Deployment - scheduled for next sprint ''[[SCHEDULED]]''
** '''[[Extension:WebFonts|Web Fonts]]'''
*** Adding languages ''[ONGOING]''
** '''[[Help:Extension:Narayam|Narayam]]'''
*** Adding input methods for other languages ''[ONGOING]''
** '''[[Milkshake|Project Milkshake]]'''
*** Design specification, API specification, Documentation ''[ONGOING]''
*** jQuery.i18n Development ''[IN PROGRESS]''
*** jQuery.uls Development ''[IN PROGRESS]''
*** jQuery.ime Development ''[IN PROGRESS]''
*** jQuery.webfonts Development ''[IN PROGRESS]''
*** Technical documentation '''[DONE]'''
** '''[[Language search]]'''
*** Initial analysis '''[DONE]'''
*** Prototype initial concept ''[IN PROGRESS]''
*** Integration into ULS ''[IN PROGRESS]''
** '''[[Standards participation]]'''
*** Comments on ARB Specification '''[DONE]'''
** '''[[Language Summit]]'''
*** Preparation for November 6-7 Language Summit ''[IN PROGRESS]''
** '''[[Language Team Community Meetup]]'''
*** Language Team Pune Meetup Nov 4 ''[IN PROGRESS]''
*** Language Team Bangalore Meetup Nov 9 ''[IN PROGRESS]''
** '''[[Bangalore DevCamp November2012|India DevCamp]]'''
*** Preparation of Topics for DevCamp ''[IN PROGRESS]''
*** Registration now open ''[IN PROGRESS]''