2014-09-08

Major Highlights:

new page for hg status and issues: https://wiki.mozilla.org/DeveloperServices/hg

improve capacity by running debug tests based off every 2 builds instead of on each one for on mozilla-inbound: bug 1056787

Completed work (resolution is 'FIXED'):

Buildduty

update bm-remote with new mobile_tp4.zip file

Builds fail with file not recognized: File truncated

builds-4hr.js.gz et al aren't updating

Trees closed due to Windows test backlog

upload a new mobile_tp4.zip pageset to the 3 headed remote talos server (round 2)

upload a new mobile_tp4.zip pageset to the 3 headed remote talos server

Last non-pending Linux 32 opt build on fx-team was 17 hours ago

Frequent timeouts downloading from pvtbuilds and usw2 proxxy

"No such file or directory" error causing upload failures

Linux64 non-unified builds aren't starting on mozilla-central

upload a new talos.zip file to get nonlocal source fixes

Legacy vcs sync not able to update legacy mapper quick enough for b2g_build.py, causing b2g builds calls to mapper to timeout

Failures on Android 2.3

Determine why batches of pandas stopped reporting on 2014-04-25 and 2014-07-13

Intermittent "Unable to remove C:\slave\test\build!"

Windows builds pending on inbound for > 2 hours

Install boto on internal PyPI

Decommission foopy117 and associated tegras

Machine being used for tests added to prod pool before being cleaned up

Install treeherder-client on internal PyPI

Install fxa-python-client on internal PyPI

b2g_bumper is stalled

ESR31 fennec symlinks missing from buildbot masters

please loan mshal a windows build machine without vs2013

General Automation

Make blobber uploads discoverable

Increase free space requirement for release-mozilla-beta-android_build jobs

Run linux64-br-haz builder on b2g-inbound/mozilla-inbound/fx-team/mozilla-central and try-by-default

Self-serve should be able to request arbitrary builds on a push (not just retriggers or complete sets of dep/PGO/Nightly builds)

[Flame] Turn off FX OS Version 34.0a1 nightly updates until instability is fixed

Clean up / handle case where spot request is active but instance has gone away

Requesting twig repo gum be reset

android single locale central/aurora nightlies not reporting to balrog

Disable non-unified builds on b2g32

get rid of comm-release configs/hacks

Enable e10s Linux opt browser chrome tests

Disable Jetpack tests on B2G release branches

Run reftests on B2G Desktop builds

You can't choose mochitest-N where N>8 for Android specific mochitests

Do debug B2G desktop builds

Enable ASAN builds/tests on Holly

There hasn't been a single successful Windows opt/nightly build off a b-2008-sm slave since the uplift

create webgl mochitest run for android

Make runner responsible for buildbot startup on CentOS

Schedule all non-perf unittests against Linux64 Code Coverage builds on Cedar

Allow test machines in AWS to get to github via internet gateway

Run mock against repos other than puppetagain

bump max token age on signing servers + clients

create signing servers for v2 mac signing

Monitor aws_stop_idle.py hungs

B2G Updates are broken when transitioning off Mozilla-Aurora to versioned Gecko

Try is attempting to trigger periodic non-unified builds

Increase Android 2.3 mochitest chunks

Turn on mochitest-e10s for ASAN builds on trunk

Kill mozilla-b2g28_v1_3 after Sep 2 merge day

Cleanup bogus Linux PGO jobs in scheduler db

Create Linux64 code coverage builds on Cedar

Intermittent command timed out: 3600 seconds without output, attempting to kill fetching b2g bits from gitmo

Create builds for Dolphin on mozilla-central, mozilla-inbound, b2g-inbound, fx-team, and mozilla-aurora

Upload new version of xulrunner to tooltool

Add task-specific configuration to Runner

Turn off Firefox desktop nightlies on the B2G release branches

Enable b2g tests on jamun project branch

(fx-l10n-sl) Add 'sl' to the Firefox Nightly build

Please install Visual Studio 2013 on a Builder

When requesting spot instances, give up on an az the first time there is no slave name

Explicitly list gaia-try's priority rather than relying on the fallback

Automatically delete obsolete latest-* builds for calendar

Consider dropping update_spot_stats() in spot_sanity_check.py

Figure out the correct path setup and mozconfigs for automation-driven MSVC2013 builds

Expand xpcshell on mozharness for Thunderbird to gecko 33

Get e10s tests running on inbound, central, and try

Schedule mochitest-plain on linux64-mulet on all branches

Add mozharness script for b2g builds in taskcluster

Don't schedule non-unified B2G hazard analysis builds

Adjust Thunderbird MozMill test installation to be able to install mozbase

b2g branching cleanup

enable os x l10n builds on oak

Tracking bug for 02-sep-2014 migration work

Please schedule b2g emulator cppunit tests on Cedar

Start doing mulet builds on Fig

Add --disable-mock option to b2g_build.py

buildbot changes to run selected b2g tests on c3.xlarge

Adjust cloudtrail related crojob times

Infrastructure for 3rd-party builds, tests to react to 'build events'

bump dmg max filesize

upload a new talos.zip to pick up directory tile fixes

hg hook update for mozilla-b2g18

make sure signing server works on 10.9 and with v2 signatures

repurpose a snow leopard machine for 10.9 mac signing

Blobber doesn't appear to work on C++ Unit test runs

Stop running builds and tests on pushes which only include changes to the b2g or mobile or browser directories

cope with signing server transition in buildbot configs

Move runner to the 'mozilla' github org

Timeouts during pip install ("Error running install of package, /builds/slave/test/build/venv/bin/pip install --download-cache /builds/...", "Error running install of package, /builds/slave/talos-slave/test/build/venv/bin/pip install --download-cache...")

Loan Requests

Slave loan request for gbrown

Slave loan request for some bld-lion-r5 machines in the try VLAN

Slave loan request for gbrown

Slave loan request for a b-2008-sm machine

Slave loan request for a talos-mtnlion-r5 machine

Need a WinXP test slave

create AWS dev instance for zeller to showcase ship-ip ui changes

Requesting a loaner machine b2g_ubuntu64_vm to diagnose bug 1017490

loan request to investigate bug 1026805

Requesting a loaner machine to diagnose bug 1039739

Requesting a loaner machine b2g_ubuntu64_vm to diagnose Bug 1043892

Slave loan request for a bld-linux64-ec2 vm

Requesting Android 4.2 x86 emulator machine for jchen

Slave loan request for a bld-lion-r5 machine

Mozharness

move ash-mozharness to hg.m.o/build

vcs2vcs sync in mozharness does not work if repo_name is different to basename of repo url

for mozharness desktop builds, replace usage of client.mk with upcoming mach/python script

fx desktop builds in mozharness

Failures during install_module() are shown as ERRORs in the log even though the run succeeded

Turn on blobber upload for Gaia Unit Tests

AndroidEmulatorTest should use MozbaseMixin

Process extensions in packaged tests if they exist

Thunderbird xpcshell tests need extra arguments in mozharness

Investigate switching Thunderbird comm-central xpcshell-tests to mozharness

Other

Improve cleanup of tinderbox-builds

[tracker] Machine move SCL1 -> SCL3

Please create a Mobile1.1 tinderbox page

Figure out retention of flame tinderbox-builds

New AWS account, 'mockbuild-repos-upload'

Rename all things Tryserver/MozillaTry to just Try in order to have uniform naming from repo-name on down

Need read permissions to subnets

Windows builds on cedar broken

Create bookmarklet to simplify the TBPL -> 'viewing job on the buildbot master' workflow

abort: pretxnchangegroup.c_try_mandatory hook failed

linux desktop qt builds for firefox

moving panda infrastructure from scl1->scl3

Request for ElasticBeanStalk account to test deployment of build-funsize

Platform Support

c3.xlarge instance types are expensive, let's test running those tests on a range of instance types that are cheaper

Upload new version of xulrunner to tooltool

Updated android builders to include google play services (and android support v7 if it isn't already included)

Configure AWS Windows Server 2012 instances for audio support

prepare the new seamicro machines for production

update zip and unzip on mac os x build/test machines to 6.0

Intermittent Android x86 We have not been able to establish a telnet connection with the emulator

Update Android SDK and build tools to 20

Upload JDK1.7 on builders

Update version of pip installed on automation machines from 0.8.2 to 1.5.5

Determine number of iX machines to request for 2014

Disable armv6 builds and tests everywhere apart from esr31

Change Android 2.2 builds to be called Android 2.3 builds

[mozpool-client] TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

Release Automation

xulrunner_source failed uploading file for signing, file too large

enable spot instances for some release builders

Automatic filling of the branch field in ship-it

Dashboard check automatically managed by ship-it

Tree closure hooks for esr31

Fix the ship-it test suite

Have release automation support signing OSX builds

When no comment is provided, the email still show Comment:

make aus3 and aus4 responses consistent

Thunderbird ship-it notifications have wrong mozilla-beta revision

Add new table(s) to shipit database

send e-mail to release-drivers when a build is requested through ship it

Fennec armv6 esr31 differences between release and nightly in-tree configs that are not whitelisted

pymake hacks in release repacks drop all but one signing server from list

Consider fully disabling mochitest-bc and mochitest-dt on the B2G release branches

With French locale, the sort of the date is broken

Add REST API entry point to shipit that allows shipit-agent to enter release data into shipit database

update partner repack scripts to cope with new format after mac signing changes

Tree closure hooks for esr17

upload a new talos.zip to pick up all the fixes

Add a footer to ship-it with a link to source + bugzilla

Releases

tracking bug for build and release of Thunderbird 31.1.0

Sign and dev deploy the Firefox update hotfix (v20140527.01.4)

tracking bug for build and release of Thunderbird 24.8.0

tracking bug for build and release of thunderbird 31.0

Add SeaMonkey 2.29 Beta 2 to bouncer

Remove sw during beta-release migration for Firefox 31, keep hsb

Trim rsync modules (May 2014 OMG we still have to do this edition)

Add SeaMonkey 2.29 to bouncer

tracking bug for build and release of Firefox 31.1.0 ESR

Disable Flame OTA and rebuild with bug 937485 backed out

bouncer checks for aurora stub installer broken

Remove sw during beta-release migration for Firefox 32, keep dsb

Add SeaMonkey 2.29 Beta 1 to bouncer

tracking bug for build and release of Firefox 24.8.0 ESR

outdated link: releases.mozilla.org/pub/mozilla.org/thunderbird/releases/latest

Releases: Custom Builds

Remove Blackbird builds for Fx32

Partner repack changes for Yandex - new tab images (regular/retina)

Modify AOL Repack Configuration for Firefox 32

Version bumps for Yahoo FF 32

Respin yandex-* and aol-* partner repacks.

add new image sizes to Bing/MSN search plugins

Partner repack changes for web.de, gmx, mail.com and 1&1 (Firefox 32)

Remove Rambler config to inactive and remove builds from FTP

remove out of date Bing plugins from yandex-tr configs

Repos and Hooks

The order of reviewers matters for the webidl commit hook

fx-team branch of gecko-dev hasn't synced in 8 hours

Make prevent_webidl_changes.py correctly deal with merges

Roll local changes to treeclosure.py into repo (update treeclosure hook for mozilla-2.1, mobile-2.0, mozilla-2.0 trees)

Give Treeherder URLs alongside TBPL ones in the hg.mozilla.org push response

Mercurial extension for monitoring servers

Change user to emailuser in hg annotate web interface

Convert some hooks tests to Mercurial testing format

push hooks should print out hg url for the changeset

releases/mozilla-1.9.1, releases/mozilla-1.9.2, and releases/mozilla-2.0 repositories are missing

Move hghooks repository into version-control-tools

Tree closure hook should be more robust

Request for a new repository for NFC emulation library

Port Mercurial commit message hook tests to .t test

Skip testTipDirRenameShouldFail on non-case sensitive filesystems

Enforce a set of allowed commit messages as hg precommit hooks on hg.m.o

Please create hg.m.o repo for b2g 2.0m

Allow pushing Instantbird-only changes on a closed c-c- tree

test-push_printurls.t failing on Linux due to shell syntax

Reject changesets containing "[PATCH"

https://hg.mozilla.org/repo-group?repo=/mozilla-central no longer works

treeclosure hook fetches 2MB of HTML to get at a few hundred bytes that are in TreeName/status.html

Implement a string checker hook on mozilla-aurora,mozilla-beta and mozilla-release repos to refuse string changes without specific approval

Approval hook should not reject backout commits that do not contain an a=foo

Make mercurial closure hooks fail closed once treestatus.mozilla.org proven reliable

Sync two repos for dolphin builds on master

esr31 branch on https://github.com/mozilla/gecko-dev

remove jhopkins and mihai from releng whitelist

We should probably reset try to stop breaking hg.m.o

"sr=[DOM peer]" should be allowed as well as "r=[DOM peer]" for webidl changes

Tools

Verify that trychooser's tryload is looking at the right names for pending jobs and adding them up into the correct buckets

Update platform parsing in buildapi wait times emails

Deploy newer slaverebooter

Add mochitest-e10s to trychooser

tree closure hooks for integration/b2g-inbound

Add gaia-build-unit-tests to Trychooser

Allow filtering tests by b2g and Android

Prune the allowed origins list for buildapi

Add slavename to title for individual slave's slave health page

Allow editing of relevant slavealloc fields directly from slave health

Please deploy trychooser from tools tip to pick up bug 1057050 and bug 1057301

slaverebooter should attempt reboot if graceful shutdown fails

Funsize should return correct MIME type

Give Treeherder URLs in Try responses / emails.

Run slaverebooter more often than once every 4 hours

Runner dies before starting buildbot due to hg failures and leaves slave doing nothing

Funsize tool handling is broken and incorrect

Deploy newer slaverebooter

slavealloc API URL used when making changes from slave_health contains an additional forward slash

Run manage_foopies.py after reconfig is completed in end_to_end_reconfig.sh

Slaverebooter should honor pep8 and where-applicable pylint

make "max log length exceeded" error message show the log size it found

Deploy new vcs_sync into production with changes from bug 962853

Add timeouts to buildapi's urllib.urlopen() requests

Add linux64-mulet to trychooser

Aggregate similar exceptions in emails from masters

Funsize returns errors out and returns an HTTP 500 when requested for a non-existent partial

Update trychooser for Android again

Add web-platform-tests to trychooser

In progress work (unresolved and not assigned to nobody):

Balrog: Backend

create balrog rules to properly support OS blocking and watershed updates on beta (and equivalent test) channel

add schema for h264 updates

support comparison operators for matching version & buildID in rules

Balrog: Frontend

balrog's rules page is excessively wide

Buildduty

Slaverebooter hung on Aug 25

loan t-snow-r4-0099 to jlund

[tracker] windows build infrastructure issues

Something wrong with windows build slaves ("LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt")

Windows Builds fail with configure: error: To build the installer you must have the latest MozillaBuild or Unicode NSIS version .46 or greater in your path.

General Automation

Add nightlies for Mulet

Enable chunking for B2G ICS Emulator Debug XPCShell tests

try emails should include commit messages from pushed commits

Allow to find which builders have changed before/after a patch

[Tracking] Stand up Code Coverage build and test runs

Thunderbird Linux and Mac MozMill builders are using the system python (fix bustage on Thunderbird Mac MozMill builders)

Do open264 builds

ALL b2g builds exceeding buildbot output limits

Increase web-platform-tests to 4 chunks

Make enable_perproduct_builds opt out rather than opt in

disable uploading to update.boot2gecko.org for mozilla-central/mozilla-aurora/1.4

web-platform-tests-reftests missing configuration file on linux-64-asan

AWS region-local caches for https stuff

Modify file structure of Firefox.app to allow for OSX v2 signing

Add signing for firefox and org.mozilla.updater binaries on OSX

Remove dependance on fennec_ids.txt from automation

Splitting Gij and Gip tests in several chunks

hgtool should retry or exit if it hits a DNS or server error during pull, not clobber and unbundle

[tracking] rollout desktop mozharness mach builders across all branches

Please schedule b2g emulator cppunit tests on trunk/aurora trees and have them ride the 34 train

generate "flatfish" builds for B2G

mac-v2-signing servers died under load

Make runner responsible for buildbot startup on Ubuntu test

Start to drop "mozilla" objdir from Thunderbird builds

Make some mozharness jobs easier to run outside of automation + allow for http authentication when running outside of the releng network

Support multiple virtualization and root device type AMIs in aws_watch_pending.py

web-platform-tests shouldn't run on Aurora

port buildapi to relengapi

Don't clobber the source checkout

[Flame] Create Kitkat based pvtbuilds for 2.0 and 2.1

Add "Android instrumentation" tests runs to buildbot config

Schedule web-platform-tests on linux64 opt on trunk branches

b2g debug win32 desktop builds not triggered on try

Port Runner cross-platform

Stop rebooting after every job

Nightly Aurora Fennec l10n repacks not available

pgo talos builders don't get "env" set

Write an S3 upload Python library that encapsulates most of blobber's functionality

FlatFish: Integrate boot.img and recovery.img into the build system

Add dependency management to Runner

Split webgl tests into a separate mochitest-gl suite on desktop

Run unittests on Win64 builds

Skip every nth job for debug tests on mozilla-inbound

stop jobs from hg'n mozharness/tools repos

Loan Requests

Slave loan request for a w64-ix machine

loan glandium and mshal a Linux ec2 instance

loan kmoir a dev-linux64-ec2 slave

Slave loan request for a tst-linux64-ec2 vm

Slave loan request for a bld-linux64-ec2 vm

Loan an ami-6a395a5a instance to Aaron Klotz

Slave loan request for a bld-linux64-ec2 vm for pmoore

Loaner slave matching WinXP mochitest1 slaves for :walter

Loan t-w732-ix-003, t-w732-ix-004 to Q

loan kmoir an aws test slave for testing in bug 1057025

loan linux32 hardware talos slave to jmaher

talos-linux-ix-118 loan request for jmaher

Slave loan request for tst-linux64-spot

loan kmoir a tst 32 aws slave for bug 1051886

Loan :sfink a linux64 b-linux64-hp-0024

Slave loan request for a tst-linux64-ec2 vm for :jgilbert

loan windows 7 hardware talos slave to jmaher and rvitillo@mozilla.com

Slave loan request for a VS2013 build machine

Slave loan request for a w64-ix machine

R5 request for testing OSX 10.10 yosemite base image in deploystudio

Slave loan request for a bld-lion-r5 machine

Mozharness

Make Proxxy a standalone class not a Mixin

Increase free space requirement for b2g hazard builds

Add crash log formatting to StructuredFormatter

Enable blobber to do the upload even when the harness is killed because of a test timeout

mozharness DesktopUnittestOutputParser.evaluate_parser() should record a failure if the return code was non-zero

Add command line options to web-platform-tests for crash reporting.

Move Firefox Desktop repacks to use mozharness

verify script called by mozharness for android devices doesn't reboot via mozpool

Use tooltool.py to download cached artefacts for android builds

Make mozharness use structured logging for marionette tests

Code coverage test jobs should set GCOV_PREFIX, package .gcda files and upload them to blobber

Make sure to mark jobs as failures when structured logger parsing fails

Other

figure out what to do with the n900s

Hg clone: "abort: error: nodename nor servname provided, or not known"

Upgrade ASan Clang to trunk r214699

build/unix/build-clang.py does not work with LLVM trunk on OSX 10.9

Make AWS node type available to graphite & build metadata

Platform Support

[tracker] Stop testing on tegras

evaluate mac cloud options

create HVM ami for Ubuntu64

Install vs2013 Update 3 on 5 machines for testing

slave pre-flight tasks (runner)

Run unittests/talos on OS X 10.9 Mavericks

Upgrade Mercurial clients to 3.0.2+

Release Automation

release automation should update ship it at certain points

pushing release builds to beta doesn't work in balrog

comm-esr31 does wasteful things while cloning mozilla-esr31

Mobile release automation to move candidates to releases/ (push to mirrors)

Figure out how to offer release build to beta users

deploy ship it notifier

Update channels for single locale Beta and Release builds of Firefox for Android 30 (and beyond)

replace columns in shipit database to track kickoff status

adjust e-mail for update_shipping builder to be more intelligible

Add a standalone process that listens to pulse for release related buildbot messages

Update transparently the product details when pushing to the mirror

Empty mozilla-releases rsync module

flip the switch to move beta users over to balrog

Use hardlinks when pushing to mirrors to speed it up

allow post release tasks to be triggered from ship it

new Builder to run pushsnip and hook it and the existing postrelease builder up to their own Schedulers that can be triggered through SendChange for shipit

Update release runner to be able to trigger pushsnip and postrelease builders through SendChange

cache MAR + installer downloads in update verify

Ensure that Fennec builds from mozilla-esr31 have a buildID to allow for armv6/Android 2.2 users to update to mozilla-esr31 apks

Releases

tracking bug for build and release of Thunderbird 31.2.0

tracking bug for build and release of Firefox 31.2.0 ESR

tracking bug for build and release of Firefox and Fennec 32.0

tracking bug for build and release of Firefox and Fennec 33.0

do a staging release of Firefox and Fennec 34.0b1

Releases: Custom Builds

Partner repack changes for Yandex for Fx 32 release

Remove Onet builds for Fx33

Repos and Hooks

(HEADlessTry) Reimplement try in HEADless, scalable, cloudy way

Provide advice on SCM system for Public Suffix List

find RFO for git.m.o OOM condition in bug 985864

Add a hook to detect changesets with wrong file metadata

Disallow "b=N" (instead of "Bug N") in commit message hook

Please sync mozilla-b2g32_v2_0m to git mirror

Migrate hg_templates into version-control-tools repo

Pulling with Mercurial 3.1 takes several minutes

Estimate web cluster size needed to support try repository load

Tools

incorrect try submission summary email

Funsize requires file level diff caching for speedups in partial generation

Add an option to trychooser to select --bisect-chunk options

Make Funsize generated partials identical across platforms and environments

Add new parameter - releaseBuildOnBetaChannel - to ship it

tooltool should output informational and progress messages with default options

Docs for Funsize

Reconfigs should be automatic, and scheduled via a cron job

Move mqext into the new https://hg.mozilla.org/hgcustom/version-control-tools repo

Add b2g hazard build to trychooser

Need script to compute name of mirrored repository

permanent location for vcs-sync mapfiles, status json, logs

Repos are being merged twice during a reconfig AND changes to tools repo are not tracked / updated in wiki, nor used to decide whether to update foopies and masters or not

Dev services (e.g. hg.mozilla.org and git.mozilla.org) status page(s)

vcs-sync needs to be able to account for empty source repos

Fix tools/update-packaging/common.sh for consistent behaviour across platforms

Show more