2014-09-29

Major Highlights:

shipped 10 products in less than one day

Completed work (resolution is 'FIXED'):

Balrog: Backend

Add interpolation for schema v2 url params (eg openURL)

Fix up balrog-server.py for missing specialForceHosts

create balrog rules to properly support OS blocking and watershed updates on beta (and equivalent test) channel

tracking bug for work required to make Balrog usable for Betas

force=1 isn't passed on for special domains

Buildduty

Intermittent Android 4.0 debug jsreftest/reftest failures with no error summary (timeout/crash/hang?)

Upload new versions of mozlog, mozprocess, mozcrash. mozdevice to internal pypi

Something wrong with windows build slaves ("LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt")

Upload mozversion to internal pypi

Failures on Android 2.3

Due to a failed reconfig, master buildbot-master68 needs to be restarted

High packet loss from usw2 causing frequent timeouts

builds-4hr.js.gz et al aren't updating

Determine why batches of pandas stopped reporting on 2014-04-25 and 2014-07-13

Need manual shutdown of runaway talos-mtnlion-r5-029

Intermittent "Unable to remove C:\slave\test\build!"

AWS Linux builders fail to allocate LVM free space

Builds fail with configure: error: Windows SDK at /c/Program Files (x86)/Windows Kits/8.0/ appears broken. Try updating to MozillaBuild 1.9 final or higher. *** F

High pending count for Windows test slaves

Frequent timeouts downloading from pvtbuilds and usw2 proxxy

Install boto on internal PyPI

Frequent proxxy timeouts on usw2

ESR31 fennec symlinks missing from buildbot masters

"No such file or directory" error causing upload failures

please loan mshal a windows build machine without vs2013

Builds fail with file not recognized: File truncated

General Automation

Add nightlies for linux64 Mulet

Expand xpcshell on mozharness for Thunderbird to gecko 33

Allow to find which builders have changed before/after a patch

Disable Thunderbird 10.6 Mac Mozmill test builders

disable mar/gpg signing formats on mac signing servers

Enable b2g tests on jamun project branch

[Flame] Turn off FX OS Version 34.0a1 nightly updates until instability is fixed

Figure out the correct path setup and mozconfigs for automation-driven MSVC2013 builds

android single locale central/aurora nightlies not reporting to balrog

Can we get a copy of bors.cfg from buildbot-master-servo-01?

Schedule web-platform-tests on linux64 opt on trunk branches

upload a new talos.zip to pick up directory tile fixes

get rid of comm-release configs/hacks

Enable e10s Linux opt browser chrome tests

Make enable_perproduct_builds opt out rather than opt in

Add support for mochitest-jetpack tests & enable on cedar

Do debug B2G desktop builds

Enable ASAN builds/tests on Holly

create webgl mochitest run for android

Linux Mulet nightlies failing during the uploadsymbols step of the build process

Split Gij and Gip tests into several chunks and run on Cedar

Run mock against repos other than puppetagain

bump max token age on signing servers + clients

Please schedule b2g emulator cppunit tests on trunk/aurora trees and have them ride the 34 train

Missing hsb and dsb single-locale Aurora Fennec builds

Schedule all non-perf unittests against Linux64 Code Coverage builds on Cedar

Make runner responsible for buildbot startup on Ubuntu test

Blobber errors in all runs: "ValueError: No JSON object could be decoded" and "simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 (char 0)"

B2G Updates are broken when transitioning off Mozilla-Aurora to versioned Gecko

Make some mozharness jobs easier to run outside of automation + allow for http authentication when running outside of the releng network

pgo talos builders don't get "env" set

all android single locale repacks broken

Move runner to the 'mozilla' github org

Please enable nightly builds on Maple

Kill mozilla-b2g28_v1_3 after Sep 2 merge day

Intermittent command timed out: 3600 seconds without output, attempting to kill fetching b2g bits from gitmo

[Flame] Create Kitkat based pvtbuilds for 2.0 and 2.1

ccache sometimes hits fatal permission denied errors when trying to create temp files

Upload gecko.log for non-emulator Mn/Mnw tests

Add mapping for crashreporter-symbols.zip for nightly builds on ftp.mozilla.org

Requesting twig repo gum be reset

Tooltool upload request for statically linked sixgill

b2g debug win32 desktop builds not triggered on try

(fx-l10n-sl) Add 'sl' to the Firefox Nightly build

Please install Visual Studio 2013 on a Builder

blocklist update failing to wget from amo

update_shared_repos exits non zero if find returns no repos

mozharness configuration is writable in query_abs_dir()

AWS AMIs should use LVM for their root device

Consider dropping update_spot_stats() in spot_sanity_check.py

Tracking bug for 02-sep-2014 migration work

Create builds for Dolphin on mozilla-central, mozilla-inbound, b2g-inbound, fx-team, and mozilla-aurora

Get e10s tests running on inbound, central, and try

Update sync-hg-blocklist to the new blocklist domain

Intermittent build failures with "ccache: failed to create /builds/ccache/tmp (No such file or directory)" ("make[3]: *** [configure] Error 1", "make[2]: *** [configure] Error 1" or "make: *** [configure] Error 1")

Adjust Thunderbird MozMill test installation to be able to install mozbase

b2g branching cleanup

Start doing mulet builds on Fig

Schedule full emulator tests on b2g18 v1.1hd

manage_instance_storage.py should log to syslog

Add mozharness script for b2g builds in taskcluster

Bump build_space for Mulet

Turn on mochitest-e10s for ASAN builds on trunk

Lots of old builds in latest-comm-aurora-l10n/ and latest-comm-central-l10n/ on Thunderbird's ftp

Blobber doesn't appear to work on C++ Unit test runs

Stop running builds and tests on pushes which only include changes to the b2g or mobile or browser directories

cope with signing server transition in buildbot configs

Loan Requests

Slave loan request for gbrown

Requesting a loaner machine b2g_ubuntu64_vm to diagnose bug 1053703

please loan OS X 10.x hardware slave to dminor

Slave loan request for a b-2008-sm machine

Loaner slave matching WinXP mochitest1 slaves for :walter

create AWS dev instance for zeller to showcase ship-ip ui changes

Slave loan request for tst-linux64-spot

Requesting a loaner machine b2g_ubuntu64_vm to diagnose Bug 1043892

Slave loan request for a w64-ix machine

Requesting a loaner machine b2g_ubuntu64_vm to diagnose Bug 1057198

Mozharness

Increase free space requirement for b2g hazard builds

vcs2vcs sync in mozharness does not work if repo_name is different to basename of repo url

Mozharness test configs should live in-tree

Improve handling of packages locally (file:// support)

Unzip on developer's Mac do not like test.zip files produced in releng infra

Make Proxxy a standalone class not a Mixin

python errors in mobile_l10n.py

Move some android and b2g manifest configs out of mozharness and into the tree

Thunderbird xpcshell tests need extra arguments in mozharness

talos_from_code.py should allow custom .zips from ash project, throw a sensible error when talos.zip comes from bad location

Mozharness desktop unittests don't use the in tree mozharness config

Process extensions in packaged tests if they exist

[mozharness] Move structured logging support code into mozharness proper

Make sure to mark jobs as failures when structured logger parsing fails

Other

Improve cleanup of tinderbox-builds

[tracker] Machine move SCL1 -> SCL3

Hg clone: "abort: error: nodename nor servname provided, or not known"

sign Thunderbird hot-fix addon - final version

Firefox Beta 11.0 (XUL) is not displayed on the Android Market for Tablets

hgpoller + new pushlog picks up wrong changeset

Add sheriffs to the vpn_releng_self_serve LDAP group so they can deploy trychooser

delete old snippet backups from aus3-staging

moving panda infrastructure from scl1->scl3

Request for ElasticBeanStalk account to test deployment of build-funsize

delete old snippet backups from aus3-staging (take 2)

Platform Support

windows build times exceeding timeout on release branches and nonunified trunk branches

Intermittent Android x86 We have not been able to establish a telnet connection with the emulator

Update Android SDK and build tools to 20

Upload JDK1.7 on builders

Update version of pip installed on automation machines from 0.8.2 to 1.5.5

b-2008-sm machines being incorrectly flagged as unreachable

Release Automation

Add new table(s) to shipit database

send e-mail to release-drivers when a build is requested through ship it

make ReleaseUpdatesFactory fail if balrog submission fails

Consider fully disabling mochitest-bc and mochitest-dt on the B2G release branches

flip the switch to move beta users over to balrog

Add REST API entry point to shipit that allows shipit-agent to enter release data into shipit database

ESR support for Android 2.2 on gecko 31

Fix the ship-it test suite

enable spot instances for some release builders

Dashboard check automatically managed by ship-it

Incorrect revision in build start email

stop sending out "updates available on betatest" mail for releases that are pushed to mirrors automatically

stage-tasks.sh is using bashim but declare /bin/sh as shebang

WIndows beta build failing to clone mozilla-beta repo

enable android 2.3 release/test builders on esr31 branch

Fennec armv6 esr31 differences between release and nightly in-tree configs that are not whitelisted

release runner breaks if there isn't an existing buildbot-configs check out

partner repack scripts need to resign builds with OS X signature

xulrunner_source failed uploading file for signing, file too large

Please disable the servo bors cronjob

adjust e-mail for update_shipping builder to be more intelligible

deploy ship it notifier

Adjust get_apk.sh with non existing armv6 build

use beta-cdntest instead of releasetest for Beta channel in Balrog

update staging release configs

Use hardlinks when pushing to mirrors to speed it up

Adjust braindump/releases-related/push_fennec_release.sh to copy in armv6 build

Releases

tracking bug for build and release of Thunderbird 31.1.0

Sign and dev deploy the Firefox update hotfix (v20140527.01.4)

tracking bug for build and release of Thunderbird 24.8.0

Please temporarily specify promptWaitTime as 24 hours for Beta snippets

tracking bug for build and release of thunderbird 31.0

Add SeaMonkey 2.29 Beta 2 to bouncer

Disable Flame OTA until bug 1068539 lands

tracking bug for build and release of Firefox and Fennec 32.0.1

Trim rsync modules (May 2014 OMG we still have to do this edition)

Add SeaMonkey 2.29 to bouncer

unstrand users who have stripped an architecture away from our official releases

Add SeaMonkey 2.29.1 to bouncer

tracking bug for build and release of Firefox 31.1.0 ESR

disable updates to Firefox 32.0.1

Disable Flame OTA and rebuild with bug 937485 backed out

bouncer checks for aurora stub installer broken

Remove sw during beta-release migration for Firefox 32, keep dsb

Add SeaMonkey 2.29 Beta 1 to bouncer

tracking bug for build and release of Firefox and Fennec 32.0

tracking bug for build and release of Firefox 24.8.0 ESR

outdated link: releases.mozilla.org/pub/mozilla.org/thunderbird/releases/latest

Releases: Custom Builds

Remove Blackbird builds for Fx32

Remove Onet builds for Fx33

ATT FF Custom Build Home Page Set

Modify AOL Repack Configuration for Firefox 33

Partner repack changes for Yandex - new tab images (regular/retina)

product signed builds for yahoo-en-US_att

Firefox 31 mail.com partner build is not advertising updates

Modify AOL Repack Configuration for Firefox 32

Version bumps for Yahoo FF 32

Partner repack changes for Yandex for Fx 32 release

add new image sizes to Bing/MSN search plugins

Partner repack changes for web.de, gmx, mail.com and 1&1 (Firefox 32)

Respin yandex-* and aol-* partner repacks.

Remove Rambler config to inactive and remove builds from FTP

Partner repack changes for web.de, gmx, mail.com and 1&1 (Firefox 33)

remove out of date Bing plugins from yandex-tr configs

Repos and Hooks

Give Treeherder URLs alongside TBPL ones in the hg.mozilla.org push response

make 'fwunit' a grown-up project

Add QA to the footer list of project repositories on hg.mozilla.org

https://hg.mozilla.org/repo-group?repo=/mozilla-central no longer works

Migrate hg_templates into version-control-tools repo

Request for a new repository in /gaia-l10n: pt-PT

Pulling with Mercurial 3.1 takes several minutes

Please create hg.m.o repo for b2g 2.0m

changelog Atom feed is hard to read with long lines wrapped in

Perform hg.m.o failover testing during TCW of Sep 20

Change user to emailuser in hg annotate web interface

Mixed-content SSL warning on hg.mozilla.org due to livemarks image from www.mozilla.org

Convert some hooks tests to Mercurial testing format

[b2g] Add gaia-l10n repos for Finnish and Armenian locales

XSS vulnerability with pushloghtml's 'startdate' and 'enddate' params

install firmware & reboot hgssh1 during next TCW

Request for a new repository for NFC emulation library

hg annotate output on hgweb is painfully slow

Port Mercurial commit message hook tests to .t test

create hook to mirror pushlog db

gitweb has the wrong autodiscovery feed link for /repo/tags

Stray > (unclosed tag) in hg.mozilla.org repository layout section

Request to mirror mozilla-b2g/android-hardware_legacy

pushlog should support query by date

changelog Atom feed is hard to read with long lines wrapped in

gaia-l10n mercurial repository for br

hg.m.o graphs no longer working?

Add changeset UUID to the graph view

Implement a string checker hook on mozilla-aurora,mozilla-beta and mozilla-release repos to refuse string changes without specific approval

hg.mozilla.org should serve well-formed xhtmls

Viewing a changeset should link to a bug

Request for a new repository - hg.m.o/build/relabs/puppet

Remove unnecessary s and per element styles on hgweb pages

Better faster annotate left out the id on line numbers, so links with anchors don't scroll

Update Mozilla templates to work with new Mercurial templater

html pushlog should fold merge changesets into one line

hgweb's 'base' link on the file revisions page should link to revision history of the old file, not annotated view

Enable graph view on hg.mozilla.org

Estimate web cluster size needed to support try repository load

Use new-style template syntax in gitweb_mozilla

Tools

Add unit tests to validate json files in tools repo to avoid bustage

Run manage_foopies.py after reconfig is completed in end_to_end_reconfig.sh

tooltool should output informational and progress messages with default options

Funsize tool handling is broken and incorrect

Store the timing of actions in slaveapi

Run slaverebooter more often than once every 4 hours

Slave health reboot/shutdown fails with "Expecting property name: line 4382 column 1 (char 128981)"

Wiki publish no longer works in end_to_end_reconfig.sh since bug 1032351 landed

Runner dies before starting buildbot due to hg failures and leaves slave doing nothing

Give Treeherder URLs in Try responses / emails.

Signing servers (signing4, signing5, signing6) running out of disk space

Add b2g hazard build to trychooser

Add mochitest-e10s to trychooser

Deploy trychooser from tools tip to pick up bug 1059689 and bug 1059689

make "max log length exceeded" error message show the log size it found

Update platform parsing in buildapi wait times emails

Please deploy trychooser from tools tip to pick up bug 1057050 and bug 1057301

Add gaia-build-unit-tests to Trychooser

Emphasise treeherder links in Try server emails & fix links for try-comm-central

SlaveAPI should be more precise on request TS

Add web-platform-tests to trychooser

In progress work (unresolved and not assigned to nobody):

Balrog: Backend

add schema for h264 updates

support comparison operators for matching version & buildID in rules

Buildduty

Handle shutdown & restart of instances for EC2 maintenance

Investigate why backfilled pandas haven't taken any jobs

[tracker] windows build infrastructure issues

Windows Builds fail with configure: error: To build the installer you must have the latest MozillaBuild or Unicode NSIS version .46 or greater in your path.

General Automation

Enable chunking for B2G ICS Emulator Debug XPCShell tests

Add Windows/OSX nightlies for Mulet

try emails should include commit messages from pushed commits

Need controlled build and machine platform data in pulse

[Tracking] Stand up Code Coverage build and test runs

Make it easier to select mochitest-bc, mochitest-dt, and e10s tests

Do open264 builds

Skip every nth job for debug tests on mozilla-inbound

ALL b2g builds exceeding buildbot output limits

disable uploading to update.boot2gecko.org for mozilla-central/mozilla-aurora/1.4

web-platform-tests-reftests missing configuration file on linux-64-asan

AWS region-local caches for https stuff

Modify file structure of Firefox.app to allow for OSX v2 signing

Build a proof of concept for log aggregation

hgtool should retry or exit if it hits a DNS or server error during pull, not clobber and unbundle

Run mozbase unit tests from test package

Enable web-platform-tests on Linux

Enable web-platform-tests-reftests

Partial update generation service

mac-v2-signing servers sometimes don't return a response to the client

Start to drop "mozilla" objdir from Thunderbird builds

web-platform-tests shouldn't run on Aurora

Do nightly builds with profiling disabled

Releng work for producing two ARMv7 APKs to target different API ranges

Remove dependance on fennec_ids.txt from automation

Add "Android instrumentation" tests runs to buildbot config

Schedule Talos (including e10s variants) on Holly

aus4 is not advertising Mac partial updates

Support multiple virtualization and root device type AMIs in aws_watch_pending.py

For Win32 get a set of plain mochitests running on holly with the content sandbox enabled

Write an S3 upload Python library that encapsulates most of blobber's functionality

FlatFish: Integrate boot.img and recovery.img into the build system

Allow mochitest-debug to mean mochitest-debug-*

-u web-platform-tests in try should select all web-platform-tests suites

Split webgl tests into a separate mochitest-gl suite on desktop

Integrate transplant tool into relengapi

Tracking bug for 13-oct-2014 migration work

stop jobs from hg'n mozharness/tools repos

manage check_ami.py by puppet

Loan Requests

Please loan jgriffin an ubuntu64 AWS VM with g2.2xlarge instance type

Slave loan request for a bld-linux64-ec2 vm for pmoore

Requesting a loaner B2G Desktop Linux x64 machine

Slave loan request for a tst-linux64-ec2 vm

Loan :sfink a linux64 b-linux64-hp-0024

loan kmoir an aws test slave for testing in bug 1057025

loan linux32 hardware talos slave to jmaher

talos-linux-ix-118 loan request for jmaher

loan windows 7 hardware talos slave to jmaher and rvitillo@mozilla.com

loan kmoir a dev-linux64-ec2 slave

Slave loan request for b-2008-sm-0023 for pmoore

loan kmoir a tst 32 aws slave for bug 1051886

Requesting a loaner machine b2g_ubuntu64_vm to diagnose bug 1065300

R5 request for testing puppet on OSX 10.10 yosemite

Slave loan request for a bld-lion-r5 machine

Slave loan request for a w64-ix machine

loan glandium and mshal a Linux ec2 instance

Slave loan request for a b-2008-ix machine

Slave loan request for a bld-linux64-ec2 vm

Loan t-w732-ix-003, t-w732-ix-004 to Q

Slave loan request for a tst-linux64-ec2 vm for :jgilbert

Slave loan request for a VS2013 build machine

Mozharness

Mozharness' structuredlog.py should get run status directly from mozlog's status handler

Make mozlog usable in mozharness

Allow scripts invoking test harnesses using structure logging to use a structured output parser

Improve spidermonkey shell builds

Enable blobber to do the upload even when the harness is killed because of a test timeout

make mozharness test scripts easier to run standalone

Add command line options to web-platform-tests for crash reporting.

Move Firefox Desktop repacks to use mozharness

verify script called by mozharness for android devices doesn't reboot via mozpool

Increase space requirement for b2g device builds

Read first in-tree configs (self.tree_config), then mozharness-configs (self.config)

Remove CCACHE variables from taskcluster mozharness config

Use tooltool.py to download cached artefacts for android builds

mozharness DesktopUnittestOutputParser.evaluate_parser() should record a failure if the return code was non-zero

Make mozharness use structured logging for marionette tests

Local mozharness B2G builds fail if B2G.git doesn't already exist

Code coverage test jobs should set GCOV_PREFIX, package .gcda files and upload them to blobber

Don't use properties of the output parser directly to determine failures in b2g_emulator_unittest.py

Other

figure out what to do with the n900s

Evaluate other work-tracking tools

build/unix/build-clang.py does not work with LLVM trunk on OSX 10.9

grant b2g release engineers access to balrog b2g rules/releases

Upgrade ASan Clang to trunk r214699

Platform Support

[tracker] Stop testing on tegras

evaluate mac cloud options

windows non unified builds failing under mozharness + mach

Install vs2013 Update 3, NSIS 3.0a2, and Mercurial 2.9.2 on 5 machines for testing

remove old code that supports tegras in buildbot-configs/buildbotcustom/tools/mozharness/puppet

slave pre-flight tasks (runner)

Run unittests/talos on OS X 10.9 Mavericks

Upgrade Valgrind on build machines to 3.10.0

Upgrade Mercurial clients to 2.9.2

Release Automation

release automation should update ship it at certain points

pushing release builds to beta doesn't work in balrog

comm-esr31 does wasteful things while cloning mozilla-esr31

Mobile release automation to move candidates to releases/ (push to mirrors)

replace columns in shipit database to track kickoff status

Update channels for single locale Beta and Release builds of Firefox for Android 30 (and beyond)

balrog submitter doesn't set previous build number properly

Add a standalone process that listens to pulse for release related buildbot messages

Update transparently the product details when pushing to the mirror

Empty mozilla-releases rsync module

allow post release tasks to be triggered from ship it

new Builder to run pushsnip and hook it and the existing postrelease builder up to their own Schedulers that can be triggered through SendChange for shipit

Update release runner to be able to trigger pushsnip and postrelease builders through SendChange

cache MAR + installer downloads in update verify

Ensure that Fennec builds from mozilla-esr31 have a buildID to allow for armv6/Android 2.2 users to update to mozilla-esr31 apks

Releases

tracking bug for build and release of Thunderbird 31.2.0

tracking bug for build and release of Firefox 31.2.0 ESR

tracking bug for build and release of Firefox and Fennec 33.0

Show whatnsew page to ESR24 users when they're offered ESR 31.2.0

do a staging release of Firefox and Fennec 34.0b1

Add SeaMonkey 2.30 Beta 1 to bouncer

Logic for when to display /whatsnew in Firefox 33

Sign and dev deploy the Firefox update hotfix (v20140527.01.6)

Releases: Custom Builds

discontinue MSN builds

Repos and Hooks

Emphasise treeherder links in Try server hghook response & fix links for try-comm-central

(HEADlessTry) Reimplement try in HEADless, scalable, cloudy way

Provide advice on SCM system for Public Suffix List

Migrate pushlog into version-control-tools repo

find RFO for git.m.o OOM condition in bug 985864

Disallow "b=N" (instead of "Bug N") in commit message hook

Please sync mozilla-b2g32_v2_0m to git mirror

Remove links to TBPL from Try server hghook response

Interrupting a push with "Ctrl+c" can keep your (successful) push from making it into the pushlog

Add a hook to detect changesets with wrong file metadata

Tools

incorrect try submission summary email

Make Funsize generated partials identical across platforms and environments

Remove links to TBPL from Try server emails

Get catlee's cool fabric actions in production for e.g. managing pulse queues on buildbot masters

cut over l10n repos to the new vcs-sync system

Add an option to trychooser to select --bisect-chunk options

Move mqext into the new https://hg.mozilla.org/hgcustom/version-control-tools repo

Transplant tool (Hg to Hg) for sheriffs

Need script to compute name of mirrored repository

Repos are being merged twice during a reconfig AND changes to tools repo are not tracked / updated in wiki, nor used to decide whether to update foopies and masters or not

Docs for Funsize

vcs-sync needs to be able to account for empty source repos

Fix tools/update-packaging/common.sh for consistent behaviour across platforms

Show more