Here is my monthly update covering what I have been doing in the free software world (previous month):
Created github-sync, a tool to mirror arbitrary repositories onto GitHub.
Submitted two pull requests to the word-wrap Chrome browser extension that adds the ability to wrap text via the right-click context menu:
Support dynamically-added <textarea> elements in "rich" Javascript applications such as mail clients, etc. (#2)
Avoid an error message if no "editable" has been selected yet. (#1)
Submitted a pull request to wordwarvi (a "retro-styled old school side-scrolling shooter") to ensure the build is reproducible. (#5)
Filed a pull request with the yard Ruby documentation tool to ensure the generated output is reproducible. (#1048)
Made some improvements to travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds on every code change:
Merged a pull request from Evgeni Golov to allow for skipped tests. (#39)
Add logging when running autopkgtests. (commit)
Merged a pull request from jwilk for python-fadvise my Python interface to the posix_fadvise(2) interface to predeclare an pattern for accessing data. (#6)
Filed an issue against the redis key-value database regarding build failures on non-x86 architectures. (#3768)
Reproducible builds
Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users.
The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced — either maliciously or accidentally — during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.
(I have previously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.)
This month I:
Presented Reproducible Builds: Two years in the trenches at linux.conf.au, Hobart, Australia. (YouTube)
Recorded an episode of the Changelog podcast on Reproducible Builds. It is scheduled to go live in early February.
Submitted the following patches to fix reproducibility-related toolchain issues within Debian:
mono: Please make the output of dh_makeclilibs reproducible
yard: Please make the output reproducible
jenkins.debian.org: Please install Build-Depends prior to locale switch
I also submitted 4 patches to fix specific reproducibility issues in flask-limiter, fontypython, pciutils & sed.
Chaired an IRC meeting on 3rd January.
Worked on publishing our weekly reports. (#88, #89, #90, 92 & 91)
I also made the following changes to our tooling:
diffoscope
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
Comparators:
Display magic file type when we know the file format but can't find file-specific details. (Closes: #850850).
Ensure our "APK metadata" file appears first, fixing non-deterministic tests. (998b288)
Fix APK extration with absolute filenames. (Closes: #850485).
Don't error if directory containing ELF debug symbols already exists. (Closes: #850807).
Support comparing .ico files (Closes: #850730).
If we don't have a tool (eg. apktool), don't blow up when attempting to unpack it.
Output formats:
Add Markdown output format. (Closes: #848141).
Add RestructuredText output format.
Use an optimised indentation routine throughout all presenters.
Move text presenter to use the Visitor pattern.
Correctly escape value of href="" elements (re. #849411).
Tests:
Prevent FTBFS by loading fixtures as UTF-8 in case surrounding terminal is not Unicode-aware. (Closes: #852926).
Skip tests if binutils can't handle the object file format. (Closes: #851588).
Actually compare the output of text/ReST/Markdown formats to fixtures.
Add tests for: Comparing two empty directories, HTML output, image.ICOImageFile, --html-dir, --text-color & no arguments (beyond the filenames) emits the text output.
Profiling:
Count the number of calls, not just the total time.
Skip as much profiling overhead when not enabled for a ~2% speedup.
Misc:
Alias an expensive Config() lookup for a 10% optimisation.
Avoid expensive regex creation until we actually need it, speeding up diff parsing by 2X.
Use Pythonic logging functions based on __name__, etc.
Drop milliseconds from logging output.
buildinfo.debian.net
buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.
Store files directly onto S3.
Drop big unique_together index to save disk space.
Show SHA256 checksums where space permits.
Debian LTS
This month I have been paid to work 12.75 hours on Debian Long Term Support (LTS). In that time I did the following:
"Frontdesk" duties, triaging CVEs, etc.
Issued DLA 773-1 for python-crypto fixing a vulnerability where calling AES.new with an invalid parameter could crash the Python interpreter.
Issued DLA 777-1 for libvncserver addressing two heap-based buffer overflow attacks based on invalid FramebufferUpdate data.
Issued DLA 778-1 for pcsc-lite correcting a use-after-free vulnerability.
Issued DLA 795-1 for hesiod which fixed a weak SUID check as well as removed the hard-coding of a fallback domain if the configuration file could not be found.
Issued DLA 810-1 for libarchive fixing a heap buffer overflow.
Uploads
python-django:
1:1.10.5-1 — New upstream stable release.
1:1.11~alpha1-1 — New upstream experimental release.
gunicorn (19.6.0-10) — Moved debian/README.Debian to debian/NEWS so that the recent important changes will be displayed to users when upgrading to stretch.
redis:
3:3.2.6-2 & 4:4.0-rc2-2 — Tidy patches and rename RunTimeDirectory to RuntimeDirectory in .service files. (Closes: #850534)
3:3.2.6-3 — Remove a duplicate redis-server binary by symlinking /usr/bin/redis-check-rdb. This was found by the dedup service.
3:3.2.6-4 — Expand the documentation in redis-server.service and redis-sentinel.service regarding the default hardening options and how, in most installations, they can be increased.
3:3.2.6-5, 3:3.2.6-6, 4:4.0-rc2-3 & 4:4.0-rc2-4 — Add taskset calls to try and avoid build failures due to parallelism in upstream test suite.
I also made the following non-maintainer uploads:
cpio:
2.12+dfsg-1 — New upstream release (to experimental), refreshing all patches, etc.
2.12+dfsg-2 — Add missing autoconf to Build-Depends.
xjump (2.7.5-6.2) — Make the build reproducible by passing -n to gzip calls in debian/rules. (Closes: #777354)
magicfilter (1.2-64.1) — Make the build reproducible by passing -n to gzip calls in debian/rules. (Closes: #777478)
Debian bugs filed
diffoscope: Comparing a device with a regular file fails
python-django: Please avoid accessing the internet for intersphinx mapping
xcalib: "Error - unsupported ramp size 0" when trying to invert screen
strip-nondeterminism: Not running full testsuite on ci.debian.net
polygen: fix seriuos → serious typo
RC bugs
python-mne: bootstrap.min.js corrupted when built with dash as /bin/sh
xdotool: "type" subcommand fails with "no windows on the stack"
I also filed 16 FTBFS bugs against bzr-git, coq-highschoolgeometry, eclipse-anyedit, eclipse-gef, libmojolicious-plugin-assetpack-perl, lua-curl, node-liftoff, node-liftoff, octave-msh, pcb2gcode, qtile, rt-authen-externalauth, ruby-hamster, ruby-sshkit, tika & txfixtures.
FTP Team
As a Debian FTP assistant I ACCEPTed 35 packages: chromium-browser, debichem, flask-limiter, golang-github-golang-leveldb, golang-github-nebulouslabs-demotemutex, golang-github-nwidger-jsoncolor, libatteanx-endpoint-perl, libproc-guard-perl, libsub-quote-perl, libtest-mojibake-perl, libytnef, linux, lua-sql, node-graceful-readlink, node-invariant, node-rollup, node-socket.io-parser, node-timed-out, olefile, packaging-tutorial, pgrouting, pyparallel, python-coards, python-django-tagging, python-graphviz, python-irc, python-mechanicalsoup, python-persistent, python-scandir, python-stopit, r-cran-zelig, ruby-ast, ruby-whitequark-parser, sagetex & u-boot-menu.