Major Highlights
big time saving in releases thanks to:
Bug 807289 - Use hardlinks when pushing to mirrors to speed it up
Completed work (resolution is 'FIXED'):
Balrog: Backend
force=1 isn't passed on for special domains
Buildduty
update bm-remote with new mobile_tp4.zip file
Intermittent Android 4.0 debug jsreftest/reftest failures with no error summary (timeout/crash/hang?)
High pending count for Windows test slaves
Machine being used for tests added to prod pool before being cleaned up
Trees closed due to Windows test backlog
upload a new mobile_tp4.zip pageset to the 3 headed remote talos server (round 2)
upload a new mobile_tp4.zip pageset to the 3 headed remote talos server
Last non-pending Linux 32 opt build on fx-team was 17 hours ago
Frequent timeouts downloading from pvtbuilds and usw2 proxxy
"No such file or directory" error causing upload failures
Linux64 non-unified builds aren't starting on mozilla-central
Builds fail with file not recognized: File truncated
upload a new talos.zip file to get nonlocal source fixes
Failures on Android 2.3
Determine why batches of pandas stopped reporting on 2014-04-25 and 2014-07-13
Intermittent "Unable to remove C:\slave\test\build!"
Builds fail with configure: error: Windows SDK at /c/Program Files (x86)/Windows Kits/8.0/ appears broken. Try updating to MozillaBuild 1.9 final or higher. *** F
High packet loss from usw2 causing frequent timeouts
Windows builds pending on inbound for > 2 hours
Install boto on internal PyPI
builds-4hr.js.gz et al aren't updating
Install treeherder-client on internal PyPI
Frequent proxxy timeouts on usw2
ESR31 fennec symlinks missing from buildbot masters
please loan mshal a windows build machine without vs2013
General Automation
Expand xpcshell on mozharness for Thunderbird to gecko 33
Run linux64-br-haz builder on b2g-inbound/mozilla-inbound/fx-team/mozilla-central and try-by-default
Self-serve should be able to request arbitrary builds on a push (not just retriggers or complete sets of dep/PGO/Nightly builds)
[Flame] Turn off FX OS Version 34.0a1 nightly updates until instability is fixed
mozharness configuration is writable in query_abs_dir()
Requesting twig repo gum be reset
android single locale central/aurora nightlies not reporting to balrog
upload a new talos.zip to pick up directory tile fixes
get rid of comm-release configs/hacks
Enable e10s Linux opt browser chrome tests
Please schedule b2g emulator cppunit tests on trunk/aurora trees and have them ride the 34 train
Do debug B2G desktop builds
Enable ASAN builds/tests on Holly
create webgl mochitest run for android
You can't choose mochitest-N where N>8 for Android specific mochitests
Schedule all non-perf unittests against Linux64 Code Coverage builds on Cedar
Clean up / handle case where spot request is active but instance has gone away
Run mock against repos other than puppetagain
bump max token age on signing servers + clients
create signing servers for v2 mac signing
Missing hsb and dsb single-locale Aurora Fennec builds
Monitor aws_stop_idle.py hungs
Make runner responsible for buildbot startup on Ubuntu test
B2G Updates are broken when transitioning off Mozilla-Aurora to versioned Gecko
Make some mozharness jobs easier to run outside of automation + allow for http authentication when running outside of the releng network
pgo talos builders don't get "env" set
all android single locale repacks broken
Increase Android 2.3 mochitest chunks
Turn on mochitest-e10s for ASAN builds on trunk
Kill mozilla-b2g28_v1_3 after Sep 2 merge day
There hasn't been a single successful Windows opt/nightly build off a b-2008-sm slave since the uplift
Create Linux64 code coverage builds on Cedar
Intermittent command timed out: 3600 seconds without output, attempting to kill fetching b2g bits from gitmo
ccache sometimes hits fatal permission denied errors when trying to create temp files
Lots of old builds in latest-comm-aurora-l10n/ and latest-comm-central-l10n/ on Thunderbird's ftp
Schedule web-platform-tests on linux64 opt on trunk branches
Tooltool upload request for statically linked sixgill
Enable b2g tests on jamun project branch
(fx-l10n-sl) Add 'sl' to the Firefox Nightly build
Please install Visual Studio 2013 on a Builder
When requesting spot instances, give up on an az the first time there is no slave name
Consider dropping update_spot_stats() in spot_sanity_check.py
Figure out the correct path setup and mozconfigs for automation-driven MSVC2013 builds
Create builds for Dolphin on mozilla-central, mozilla-inbound, b2g-inbound, fx-team, and mozilla-aurora
Get e10s tests running on inbound, central, and try
Move runner to the 'mozilla' github org
Upload gecko.log for non-emulator Mn/Mnw tests
Add mozharness script for b2g builds in taskcluster
Don't schedule non-unified B2G hazard analysis builds
Adjust Thunderbird MozMill test installation to be able to install mozbase
b2g branching cleanup
enable os x l10n builds on oak
Tracking bug for 02-sep-2014 migration work
Please schedule b2g emulator cppunit tests on Cedar
Start doing mulet builds on Fig
Add --disable-mock option to b2g_build.py
Turn off Firefox desktop nightlies on the B2G release branches
Adjust cloudtrail related crojob times
Infrastructure for 3rd-party builds, tests to react to 'build events'
bump dmg max filesize
hg hook update for mozilla-b2g18
make sure signing server works on 10.9 and with v2 signatures
repurpose a snow leopard machine for 10.9 mac signing
Blobber doesn't appear to work on C++ Unit test runs
Stop running builds and tests on pushes which only include changes to the b2g or mobile or browser directories
cope with signing server transition in buildbot configs
Timeouts during pip install ("Error running install of package, /builds/slave/test/build/venv/bin/pip install --download-cache /builds/...", "Error running install of package, /builds/slave/talos-slave/test/build/venv/bin/pip install --download-cache...")
Loan Requests
Slave loan request for a w64-ix machine
Slave loan request for some bld-lion-r5 machines in the try VLAN
Slave loan request for gbrown
Slave loan request for gbrown
please loan OS X 10.x hardware slave to dminor
Slave loan request for a b-2008-sm machine
Slave loan request for a talos-mtnlion-r5 machine
Loaner slave matching WinXP mochitest1 slaves for :walter
create AWS dev instance for zeller to showcase ship-ip ui changes
Slave loan request for tst-linux64-spot
loan request to investigate bug 1026805
Requesting a loaner machine to diagnose bug 1039739
Requesting a loaner machine b2g_ubuntu64_vm to diagnose Bug 1043892
Requesting a loaner machine b2g_ubuntu64_vm to diagnose bug 1017490
Slave loan request for a bld-linux64-ec2 vm
Requesting Android 4.2 x86 emulator machine for jchen
Mozharness
vcs2vcs sync in mozharness does not work if repo_name is different to basename of repo url
for mozharness desktop builds, replace usage of client.mk with upcoming mach/python script
fx desktop builds in mozharness
Failures during install_module() are shown as ERRORs in the log even though the run succeeded
Turn on blobber upload for Gaia Unit Tests
AndroidEmulatorTest should use MozbaseMixin
Process extensions in packaged tests if they exist
Thunderbird xpcshell tests need extra arguments in mozharness
Investigate switching Thunderbird comm-central xpcshell-tests to mozharness
Other
Improve cleanup of tinderbox-builds
[tracker] Machine move SCL1 -> SCL3
Hg clone: "abort: error: nodename nor servname provided, or not known"
Please create a Mobile1.1 tinderbox page
Figure out retention of flame tinderbox-builds
hgpoller + new pushlog picks up wrong changeset
Rename all things Tryserver/MozillaTry to just Try in order to have uniform naming from repo-name on down
Need read permissions to subnets
Add sheriffs to the vpn_releng_self_serve LDAP group so they can deploy trychooser
Create bookmarklet to simplify the TBPL -> 'viewing job on the buildbot master' workflow
Windows builds on cedar broken
abort: pretxnchangegroup.c_try_mandatory hook failed
moving panda infrastructure from scl1->scl3
Request for ElasticBeanStalk account to test deployment of build-funsize
Platform Support
c3.xlarge instance types are expensive, let's test running those tests on a range of instance types that are cheaper
Upload new version of xulrunner to tooltool
Updated android builders to include google play services (and android support v7 if it isn't already included)
Configure AWS Windows Server 2012 instances for audio support
prepare the new seamicro machines for production
update zip and unzip on mac os x build/test machines to 6.0
Update Android SDK and build tools to 20
Upload JDK1.7 on builders
Update version of pip installed on automation machines from 0.8.2 to 1.5.5
Determine number of iX machines to request for 2014
Change Android 2.2 builds to be called Android 2.3 builds
[mozpool-client] TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
Release Automation
xulrunner_source failed uploading file for signing, file too large
enable spot instances for some release builders
Automatic filling of the branch field in ship-it
Dashboard check automatically managed by ship-it
use beta-cdntest instead of releasetest for Beta channel in Balrog
Tree closure hooks for esr31
Fix the ship-it test suite
Have release automation support signing OSX builds
make aus3 and aus4 responses consistent
Thunderbird ship-it notifications have wrong mozilla-beta revision
stop sending out "updates available on betatest" mail for releases that are pushed to mirrors automatically
Add new table(s) to shipit database
When no comment is provided, the email still show Comment:
Fennec armv6 esr31 differences between release and nightly in-tree configs that are not whitelisted
pymake hacks in release repacks drop all but one signing server from list
Consider fully disabling mochitest-bc and mochitest-dt on the B2G release branches
Add REST API entry point to shipit that allows shipit-agent to enter release data into shipit database
Use hardlinks when pushing to mirrors to speed it up
update partner repack scripts to cope with new format after mac signing changes
Tree closure hooks for esr17
upload a new talos.zip to pick up all the fixes
Releases
tracking bug for build and release of Thunderbird 31.1.0
Sign and dev deploy the Firefox update hotfix (v20140527.01.4)
tracking bug for build and release of Thunderbird 24.8.0
Please temporarily specify promptWaitTime as 24 hours for Beta snippets
tracking bug for build and release of thunderbird 31.0
Add SeaMonkey 2.29 Beta 2 to bouncer
Remove sw during beta-release migration for Firefox 31, keep hsb
Trim rsync modules (May 2014 OMG we still have to do this edition)
Add SeaMonkey 2.29 to bouncer
tracking bug for build and release of Firefox and Fennec 32.0.1
tracking bug for build and release of Firefox 31.1.0 ESR
Disable Flame OTA and rebuild with bug 937485 backed out
bouncer checks for aurora stub installer broken
Remove sw during beta-release migration for Firefox 32, keep dsb
Add SeaMonkey 2.29 Beta 1 to bouncer
tracking bug for build and release of Firefox and Fennec 32.0
tracking bug for build and release of Firefox 24.8.0 ESR
outdated link: releases.mozilla.org/pub/mozilla.org/thunderbird/releases/latest
Releases: Custom Builds
Remove Blackbird builds for Fx32
Partner repack changes for Yandex - new tab images (regular/retina)
Firefox 31 mail.com partner build is not advertising updates
Modify AOL Repack Configuration for Firefox 32
Version bumps for Yahoo FF 32
Respin yandex-* and aol-* partner repacks.
add new image sizes to Bing/MSN search plugins
Partner repack changes for web.de, gmx, mail.com and 1&1 (Firefox 32)
Remove Rambler config to inactive and remove builds from FTP
remove out of date Bing plugins from yandex-tr configs
Repos and Hooks
The order of reviewers matters for the webidl commit hook
fx-team branch of gecko-dev hasn't synced in 8 hours
make 'fwunit' a grown-up project
Add QA to the footer list of project repositories on hg.mozilla.org
Make prevent_webidl_changes.py correctly deal with merges
Migrate hg_templates into version-control-tools repo
Roll local changes to treeclosure.py into repo (update treeclosure hook for mozilla-2.1, mobile-2.0, mozilla-2.0 trees)
Pulling with Mercurial 3.1 takes several minutes
Give Treeherder URLs alongside TBPL ones in the hg.mozilla.org push response
changelog Atom feed is hard to read with long lines wrapped in
Mercurial extension for monitoring servers
Change user to emailuser in hg annotate web interface
Mixed-content SSL warning on hg.mozilla.org due to livemarks image from www.mozilla.org
Convert some hooks tests to Mercurial testing format
push hooks should print out hg url for the changeset
releases/mozilla-1.9.1, releases/mozilla-1.9.2, and releases/mozilla-2.0 repositories are missing
XSS vulnerability with pushloghtml's 'startdate' and 'enddate' params
Move hghooks repository into version-control-tools
Tree closure hook should be more robust
Request for a new repository for NFC emulation library
hg annotate output on hgweb is painfully slow
Port Mercurial commit message hook tests to .t test
create hook to mirror pushlog db
gitweb has the wrong autodiscovery feed link for /repo/tags
Skip testTipDirRenameShouldFail on non-case sensitive filesystems
Stray > (unclosed tag) in hg.mozilla.org repository layout section
pushlog should support query by date
Please create hg.m.o repo for b2g 2.0m
Allow pushing Instantbird-only changes on a closed c-c- tree
test-push_printurls.t failing on Linux due to shell syntax
changelog Atom feed is hard to read with long lines wrapped in
We should probably reset try to stop breaking hg.m.o
Reject changesets containing "[PATCH"
https://hg.mozilla.org/repo-group?repo=/mozilla-central no longer works
hg.m.o graphs no longer working?
treeclosure hook fetches 2MB of HTML to get at a few hundred bytes that are in TreeName/status.html
Add changeset UUID to the graph view
Implement a string checker hook on mozilla-aurora,mozilla-beta and mozilla-release repos to refuse string changes without specific approval
Approval hook should not reject backout commits that do not contain an a=foo
hg.mozilla.org should serve well-formed xhtmls
Viewing a changeset should link to a bug
Make mercurial closure hooks fail closed once treestatus.mozilla.org proven reliable
Sync two repos for dolphin builds on master
"sr=[DOM peer]" should be allowed as well as "r=[DOM peer]" for webidl changes
Remove unnecessary s and per element styles on hgweb pages
remove jhopkins and mihai from releng whitelist
Better faster annotate left out the id on line numbers, so links with anchors don't scroll
Update Mozilla templates to work with new Mercurial templater
html pushlog should fold merge changesets into one line
hgweb's 'base' link on the file revisions page should link to revision history of the old file, not annotated view
Enable graph view on hg.mozilla.org
Use new-style template syntax in gitweb_mozilla
Enforce a set of allowed commit messages as hg precommit hooks on hg.m.o
Tools
Verify that trychooser's tryload is looking at the right names for pending jobs and adding them up into the correct buckets
Update platform parsing in buildapi wait times emails
Deploy newer slaverebooter
Add mochitest-e10s to trychooser
tree closure hooks for integration/b2g-inbound
Add gaia-build-unit-tests to Trychooser
Allow filtering tests by b2g and Android
Prune the allowed origins list for buildapi
Add slavename to title for individual slave's slave health page
Deploy newer slaverebooter
Please deploy trychooser from tools tip to pick up bug 1057050 and bug 1057301
slaverebooter should attempt reboot if graceful shutdown fails
Funsize should return correct MIME type
Run slaverebooter more often than once every 4 hours
Runner dies before starting buildbot due to hg failures and leaves slave doing nothing
Funsize tool handling is broken and incorrect
Run manage_foopies.py after reconfig is completed in end_to_end_reconfig.sh
Slaverebooter should honor pep8 and where-applicable pylint
Funsize returns errors out and returns an HTTP 500 when requested for a non-existent partial
Add b2g hazard build to trychooser
Deploy trychooser from tools tip to pick up bug 1059689 and bug 1059689
Add timeouts to buildapi's urllib.urlopen() requests
Add linux64-mulet to trychooser
Aggregate similar exceptions in emails from masters
make "max log length exceeded" error message show the log size it found
Update trychooser for Android again
Add web-platform-tests to trychooser
In progress work (unresolved and not assigned to nobody):
Balrog: Backend
Add interpolation for schema v2 url params (eg openURL)
Fix up balrog-server.py for missing specialForceHosts
add schema for h264 updates
create balrog rules to properly support OS blocking and watershed updates on beta (and equivalent test) channel
Balrog: Frontend
balrog's rules page is excessively wide
Buildduty
Slaverebooter hung on Aug 25
[tracker] windows build infrastructure issues
Something wrong with windows build slaves ("LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt")
Windows Builds fail with configure: error: To build the installer you must have the latest MozillaBuild or Unicode NSIS version .46 or greater in your path.
General Automation
Add nightlies for Mulet
Enable chunking for B2G ICS Emulator Debug XPCShell tests
try emails should include commit messages from pushed commits
Allow to find which builders have changed before/after a patch
[Tracking] Stand up Code Coverage build and test runs
Make it easier to select mochitest-bc, mochitest-dt, and e10s tests
Thunderbird Linux and Mac MozMill builders are using the system python (fix bustage on Thunderbird Mac MozMill builders)
Do open264 builds
ALL b2g builds exceeding buildbot output limits
Increase web-platform-tests to 4 chunks
Make enable_perproduct_builds opt out rather than opt in
disable uploading to update.boot2gecko.org for mozilla-central/mozilla-aurora/1.4
web-platform-tests-reftests missing configuration file on linux-64-asan
AWS region-local caches for https stuff
Modify file structure of Firefox.app to allow for OSX v2 signing
Add signing for firefox and org.mozilla.updater binaries on OSX
Remove dependance on fennec_ids.txt from automation
Splitting Gij and Gip tests in several chunks
hgtool should retry or exit if it hits a DNS or server error during pull, not clobber and unbundle
[tracking] rollout desktop mozharness mach builders across all branches
Run mozbase unit tests from test package
Enable web-platform-tests on Linux
Enable web-platform-tests-reftests
generate "flatfish" builds for B2G
mac-v2-signing servers died under load
Start to drop "mozilla" objdir from Thunderbird builds
Support multiple virtualization and root device type AMIs in aws_watch_pending.py
web-platform-tests shouldn't run on Aurora
port buildapi to relengapi
Don't clobber the source checkout
Build a proof of concept for log aggregation
[Flame] Create Kitkat based pvtbuilds for 2.0 and 2.1
Add "Android instrumentation" tests runs to buildbot config
b2g debug win32 desktop builds not triggered on try
Port Runner cross-platform
Stop rebooting after every job
Nightly Aurora Fennec l10n repacks not available
Write an S3 upload Python library that encapsulates most of blobber's functionality
FlatFish: Integrate boot.img and recovery.img into the build system
Allow mochitest-debug to mean mochitest-debug-*
Add dependency management to Runner
-u web-platform-tests in try should select all web-platform-tests suites
Split webgl tests into a separate mochitest-gl suite on desktop
Run unittests on Win64 builds
Skip every nth job for debug tests on mozilla-inbound
stop jobs from hg'n mozharness/tools repos
Loan Requests
Slave loan request for a w64-ix machine
loan glandium and mshal a Linux ec2 instance
loan kmoir a dev-linux64-ec2 slave
Slave loan request for a tst-linux64-ec2 vm
Slave loan request for b-2008-sm-0023 for pmoore
Loan an ami-6a395a5a instance to Aaron Klotz
Slave loan request for a bld-linux64-ec2 vm for pmoore
Slave loan request for a bld-linux64-ec2 vm
Loan t-w732-ix-003, t-w732-ix-004 to Q
loan kmoir an aws test slave for testing in bug 1057025
Requesting a loaner machine b2g_ubuntu64_vm to diagnose bug 1053703
loan linux32 hardware talos slave to jmaher
talos-linux-ix-118 loan request for jmaher
loan kmoir a tst 32 aws slave for bug 1051886
Loan :sfink a linux64 b-linux64-hp-0024
Slave loan request for a tst-linux64-ec2 vm for :jgilbert
Requesting a loaner machine b2g_ubuntu64_vm to diagnose bug 1065300
loan windows 7 hardware talos slave to jmaher and rvitillo@mozilla.com
Slave loan request for a VS2013 build machine
R5 request for testing puppet on OSX 10.10 yosemite
Slave loan request for a bld-lion-r5 machine
Mozharness
Make Proxxy a standalone class not a Mixin
Increase free space requirement for b2g hazard builds
Make mozlog usable in mozharness
Add crash log formatting to StructuredFormatter
Enable blobber to do the upload even when the harness is killed because of a test timeout
mozharness DesktopUnittestOutputParser.evaluate_parser() should record a failure if the return code was non-zero
Add command line options to web-platform-tests for crash reporting.
Move Firefox Desktop repacks to use mozharness
verify script called by mozharness for android devices doesn't reboot via mozpool
Use tooltool.py to download cached artefacts for android builds
Make mozharness use structured logging for marionette tests
Code coverage test jobs should set GCOV_PREFIX, package .gcda files and upload them to blobber
Make sure to mark jobs as failures when structured logger parsing fails
Other
Evaluate other work-tracking tools
grant b2g release engineers access to balrog b2g rules/releases
build/unix/build-clang.py does not work with LLVM trunk on OSX 10.9
Upgrade ASan Clang to trunk r214699
figure out what to do with the n900s
Make AWS node type available to graphite & build metadata
Platform Support
[tracker] Stop testing on tegras
evaluate mac cloud options
create HVM ami for Ubuntu64
Install vs2013 Update 3 on 5 machines for testing
remove old code that supports tegras in buildbot-configs/buildbotcustom/tools/mozharness
slave pre-flight tasks (runner)
Run unittests/talos on OS X 10.9 Mavericks
Upgrade Valgrind on build machines to 3.10.0
Upgrade Mercurial clients to 3.0.2+
Release Automation
release automation should update ship it at certain points
pushing release builds to beta doesn't work in balrog
comm-esr31 does wasteful things while cloning mozilla-esr31
Mobile release automation to move candidates to releases/ (push to mirrors)
Figure out how to offer release build to beta users
deploy ship it notifier
Update channels for single locale Beta and Release builds of Firefox for Android 30 (and beyond)
Add a standalone process that listens to pulse for release related buildbot messages
Update transparently the product details when pushing to the mirror
Empty mozilla-releases rsync module
flip the switch to move beta users over to balrog
replace columns in shipit database to track kickoff status
allow post release tasks to be triggered from ship it
new Builder to run pushsnip and hook it and the existing postrelease builder up to their own Schedulers that can be triggered through SendChange for shipit
Update release runner to be able to trigger pushsnip and postrelease builders through SendChange
cache MAR + installer downloads in update verify
Ensure that Fennec builds from mozilla-esr31 have a buildID to allow for armv6/Android 2.2 users to update to mozilla-esr31 apks
Releases
tracking bug for build and release of Thunderbird 31.2.0
tracking bug for build and release of Firefox 31.2.0 ESR
tracking bug for build and release of Firefox and Fennec 33.0
do a staging release of Firefox and Fennec 34.0b1
Add SeaMonkey 2.30 Beta 1 to bouncer
Logic for when to display /whatsnew in Firefox 33
Releases: Custom Builds
Partner repack changes for Yandex for Fx 32 release
Remove Onet builds for Fx33
ATT FF Custom Build Home Page Set
Repos and Hooks
(HEADlessTry) Reimplement try in HEADless, scalable, cloudy way
Provide advice on SCM system for Public Suffix List
Migrate pushlog into version-control-tools repo
find RFO for git.m.o OOM condition in bug 985864
Estimate web cluster size needed to support try repository load
Disallow "b=N" (instead of "Bug N") in commit message hook
Please sync mozilla-b2g32_v2_0m to git mirror
Interrupting a push with "Ctrl+c" can keep your (successful) push from making it into the pushlog
Add a hook to detect changesets with wrong file metadata
Tools
incorrect try submission summary email
Funsize requires file level diff caching for speedups in partial generation
Make Funsize generated partials identical across platforms and environments
Add new parameter - releaseBuildOnBetaChannel - to ship it
tooltool should output informational and progress messages with default options
cut over l10n repos to the new vcs-sync system
Add an option to trychooser to select --bisect-chunk options
Move mqext into the new https://hg.mozilla.org/hgcustom/version-control-tools repo
Need script to compute name of mirrored repository
permanent location for vcs-sync mapfiles, status json, logs
Repos are being merged twice during a reconfig AND changes to tools repo are not tracked / updated in wiki, nor used to decide whether to update foopies and masters or not
Docs for Funsize
Dev services (e.g. hg.mozilla.org and git.mozilla.org) status page(s)
vcs-sync needs to be able to account for empty source repos
Fix tools/update-packaging/common.sh for consistent behaviour across platforms