Here is my monthly update covering what I have been doing in the free software world (previously):
Worked on nsntrace, a userspace tool to perform network traces on processes using kernel namespaces:
Overhauled error handling to ensure the return code of the wrapped process is returned to the surrounding environment. (#10).
Permit the -u argument to also accept uids as well as usernames. (#16).
Always kill the (hard-looping) udp_send utility, even on test failures. (#13).
Updated configure.ac to look for iptables in /sbin & /usr/sbin (#11) and to raise an error if pcap.h is missing (#15).
Drop bashisms in #!/bin/sh script (#14) and ignore the generated manpage in the Git repository (#12).
Independently discovered an regression in the Django web development framework where field__isnull=False filters were not working with some foreign keys, resulting in extending the testsuite and release documentation. (#7104).
Proposed a change to django-enumfield (a custom field for type-safe constants) to ensure passing a string type to Enum.get returned None on error to match the documentation. (#36).
Fixed an issue in the Mopidy music player's podcast extension where the testsuite was failing tests in extreme timezones. (#40).
Proposed changes to make various upstream's reproducible:
botan, a crypto/TLS library for C++11. (#587).
cookiecutter, a project template generator, removing nondeterministic keyword arguments from appearing in the documentation. (#800).
pyicu, a Python wraper for the IBM Unicode library. (#27).
Integrated a number of issues raised by @piotr1212 to python-fadvise, my Python interface to posix_fadvise(2), where the API was not being applied to open file descriptors (#1) and moving the .so to a module directory (#2).
Various improvements to try.diffoscope.org, a hosted version of the diffoscope in-depth and content-aware diff utility, including introducing an HTTP API (#21), updating the SSL certificate and correcting a logic issue where errors in diffoscope itself were not being detected correctly (b0ff49). Continued thanks to Bytemark for sponsoring the hardware.
Fixed a bug in django-slack, my library to easily post messages to the Slack group-messaging utility, correcting an EncodeError exception under Python 3 (#53) and updated the minimum required version of Django to 1.7 (#54).
Various updates to tickle-me-email, my Getting Things Done-inspired email toolbox, to also match / in IMAP's LIST separators (#6) and to encode the folder list as UTF-7 (#7). Thanks to @resiak.
Clarified the documentation for travis.debian.net — my hosted script to easily test and build Debian packages on the Travis CI continuous integration platform — regarding how to integrate with Github (#20).
Reproducible builds
Whilst anyone can inspect the source code of free software for malicious flaws, most Linux distributions provide binary (or "compiled") packages to end users.
The motivation behind the Reproducible Builds effort is to allow verification that no flaws have been introduced — either maliciously and accidentally — during this compilation process by promising identical binary packages are always generated from a given source.
Toolchain issues
I submitted the following patches to fix reproducibility-related toolchain issues:
dh-python: Please make builds reproducible with a "mismatched" kernel and userland
dh-lua: Please make the substvars reproducible
filepp: Please make the output (and build) reproducible
javatools: Please make the Recommends substvars reproducible
Perl:
Please make the output of ExtUtils::MM_Unix reproducible
Please make the output of ExtUtils::Command::MM reproducible
Please make the output of libmodule-build-withxspp-perl reproducible
My work in the Reproducible Builds project was also covered in our weekly reports. (#67, #68, #69, #70).
Diffoscope
diffoscope is our "diff on steroids" that will not only recursively unpack archives but will transform binary formats into human-readable forms in order to compare them:
Added a command-line interface to the try.diffoscope.org web service.
Added a JSON comparator.
In the HTML output, highlight lines when hovering to make it easier to visually track.
Ensure that we pass str types to our Difference class, otherwise we can't be sure we can render them later.
Testsuite improvements:
Generate test coverage reports.
Add tests for Haskell and GitIndex comparators.
Completely refactored all of the comparator tests, extracting out commonly-used routines.
Confirm rendering of text and HTML presenters when checking non-existing files.
Dropped a squashfs test as it was simply too unreliable and/or has too many requirements to satisfy.
A large number of miscellaneous cleanups, including:
Reworking the comparator setup/preference internals by dynamically importing classes via a single list.
Split exceptions out into dedicated diffoscope.exc module.
Tidying the PROVIDERS dict in diffoscope/__init__.py.
Use html.escape over xml.sax.saxutils.escape, cgi.escape, etc.
Removing hard-coding of manual page targets names in debian/rules.
Specify all string format arguments as logging function parameters, not using interpolation.
Tidying imports, correcting indentation levels and drop unnecessary whitespace.
disorderfs
disorderfs is our FUSE filesystem that deliberately introduces nondeterminism in system calls such as readdir(3).
Added a testsuite to prevent regressions. (f124965)
Added a --sort-dirents=yes|no option for forcing deterministic ordering. (2aae325)
Other
Improved strip-nondeterminism, our tool to remove specific nondeterministic information after a build:
Match more styles of Java .properties files.
Remove hyphen from "non-determinism" and "non-deterministic" throughout package for consistency.
Improvements to our testing infrastucture:
Improve the top-level navigation so that we can always get back to "home" of a package.
Give expandable elements cursor: pointer CSS styling to highlight they are clickable.
Drop various trailing underlined whitespaces after links.
Explicitly log that build was successful or not.
Various code-quality improvements, including prefering str.format over concatentation.
Miscellaneous updates to our filter-packages internal tool:
Add --random=N and --url options.
Add support for --show=comments.
Correct ordering so that --show-version runs after --filter-ftbfs.
Rename --show-ftbfs to --filter-ftbfs and --show-version to --show=version.
Created a proof-of-concept reproducible-utils package to contain commonly-used snippets aimed at developers wishing to make their packages reproducible.
I also submitted 92 patches to fix specific reproducibility issues in advi, amora-server, apt-cacher-ng, ara, argyll, audiotools, bam, bedtools, binutils-m68hc1x, botan1.10, broccoli, congress, cookiecutter, dacs, dapl, dateutils, ddd, dicom3tools, dispcalgui, dnssec-trigger, echoping, eekboek, emacspeak, eyed3, fdroidserver, flashrom, fntsample, forkstat, gkrellm, gkrellm, gnunet-gtk, handbrake, hardinfo, ircd-irc2, ircd-ircu, jack-audio-connection-kit, jpy, kxmlgui, libbson, libdc0, libdevel-cover-perl, libfm, libpam-ldap, libquvi, librep, lilyterm, mozvoikko, mp4h, mp4v2, myghty, n2n, nagios-nrpe, nikwi, nmh, nsnake, openhackware, pd-pdstring, phpab, phpdox, phpldapadmin, pixelmed-codec, pleiades, pybit, pygtksourceview, pyicu, python-attrs, python-gflags, quvi, radare2, rc, rest2web, roaraudio, rt-extension-customfieldsonupdate, ruby-compass, ruby-pg, sheepdog, tf5, ttf-tiresias, ttf-tiresias, tuxpaint, tuxpaint-config, twitter-bootstrap3, udpcast, uhub, valknut, varnish, vips, vit, wims, winswitch, wmweather+ & xshisen.
Debian GNU/Linux
Patches contributed
Lintian:
Warn about Python packages that ship .coverage information
Check for libjs-* binary package name outside of web section
Warn if filenames contain wildcard characters)
Correct false positives in matching typo targets with extra whitespace
devscripts: Add an "--extra-packages" option to install arbitrary extra packages
dh-python: Emit invokation and command-line arguments of "after" and "before" commands by default
debsources: Retain line context on 404s by appending the hash for specific links
python-debian: Improve message when passing incorrect type to iter_paragraphs
autopkgtest: Does not run pyflakes3 tests
python3-debian: Include examples in package
snapshot.debian.org: Correct link to API page in sidebar
dh-python: Misparses d/control and copies build profiles into substvars
auto-apt-proxy: FTBFS: In POSIX sh, 'local' is undefined
I also submitted 22 patches to fix typos in debian/rules files against ctsim, f2c, fonts-elusive-icons, ifrit, ldapscripts, libss7, libvmime, link-grammar, menulibre, mit-scheme, mugshot, nlopt, nunit, proftpd-mod-autohost, proftpd-mod-clamav, rabbyt, radvd, ruby-image-science, snmpsim, speech-tools, varscan & whatmaps.
Debian LTS
This month I have been paid to work 15 hours on Debian Long Term Support (LTS). In that time I did the following:
"Frontdesk" duties, triaging CVEs, etc.
Authored the patch & issued DLA 596-1 for extplorer, a web-based file manager, fixing an archive traversal exploit.
Issued DLA 598-1 for suckless-tools, fixing a segmentation fault in the slock screen locking tool.
Issued DLA 599-1 for cracklib2, a pro-active password checker library, fixing a stack-based buffer overflow when parsing large GECOS fields.
Improved the find-work internal tool adding optional colour highlighting and migrating it to Python 3.
Wrote an lts-missing-uploads tool to find mistakes where there was no correponding package in the archive after an announcement.
Added optional colour highlighting to the lts-cve-triage tool.
Uploads
redis 2:3.2.3-1 — New upstream release, move to the DEP-5 debian/copyright format, ensure that we are running as root in LSB initscripts and add a README.Source regarding our local copies of redis.conf and sentinel.conf.
python-django:
1:1.10-1 — New upstream release.
1:1.10-2 — Fix test failures due to mishandled upstream translation updates.
gunicorn:
19.6.0-2 — Reload logrotate in the postrotate action to avoid processes writing to the old files and move to DEP-5 debian/copyright format.
19.6.0-3 — Drop our /usr/sbin/gunicorn{,3}-debian and related Debian-specific machinery to be more like upstream.
19.6.0-4 — Drop "template" systemd .service files and point towards examples and documentation instead.
adminer:
4.2.5-1 — Take over package maintenance, completely overhauling the packaging with a new upstream version, move to virtual-mysql-server to support MariaDB, updating package names of dependencies and fix the outdated Apache configuration.
4.2.5-2 — Correct the php5 package names.
Bugs filed (without patches)
python3-debian: Can't parse input of bytes under Python 3
devscripts: Please warn if DEBUILD_DPKG_BUILDPACKAGE_OPTS contains "-i -I"
gbrowse: Ships a predictable OpenID constumer secret
Popcon, etc:
python-popcon: Must not crash on invalid data
popularity-contest: all-popcon-results.txt contains invalid data
congress: CongressParser.py not generated during build
libfuzzer-3.8-dev:
"Illegal hardware instruction"
Please add "Suggests" for (some) version of Clang
Please provide a programmatic method of finding libFuzzer.a
ceph: Maintainer fields points to a moderated mailing list
nsntrace: Please package a new upstream snapshot and please upload to jessie-backports.
safecat: Homepage field refers to missing URL
foxeye: Please make the the package autoreconfable
RC bugs
I filed 3 RC bugs with patches:
gnome-packagekit: Missing Build-Depends on 'sp'
python-popcon: Assumes all-popcon-results.txt.gz file is UTF-8
python3-popcon: Missing Depends on 'python3-xdg'
I additionally filed 8 RC bugs for packages that access the internet during build against autopkgtest, golang-github-xenolf-lego, pam-python, pexpect, python-certbot, python-glanceclient, python-pykka & python-tornado.
I also filed 74 FTBFS bugs against airlift-airline, airlift-slice, alter-sequence-alignment, apktool, atril, auto-apt-proxy, bookkeeper, bristol, btfs, caja-extensions, ccbuild, cinder, clustalo, colorhug-client, cpp-netlib, dimbl, edk2, elasticsearch, ganv, git-remote-hg, golang-codegangsta-cli, golang-goyaml, gr-radar, imagevis3d, jacktrip, jalv, kdepim, kiriki, konversation, libabw, libcereal, libdancer-plugin-database-perl, libdist-zilla-plugins-cjm-perl, libfreemarker-java, libgraph-writer-dsm-perl, libmail-gnupg-perl, libminc, libsmi, linthesia, lv2-c++-tools, lvtk, mate-power-manager, mcmcpack, mopidy-podcast, nageru, nfstrace, nova, nurpawiki, open-gram, php-crypt-gpg, picmi, projectl, pygpgme, python-apt, python-django-bootstrap-form, python-django-navtag, python-oslo.config, qmmp, qsapecng, r-cran-sem, rocs, ruby-mini-magick, seahorse-nautilus, shiro, snap, tcpcopy, tiledarray, triggerhappy, ucto, urdfdom, vmmlib, yara-python, yi & z3.
FTP Team
As a Debian FTP assistant I ACCEPTed 90 packages: android-platform-external-jsilver, android-platform-frameworks-data-binding, camlpdf, consolation, dfwinreg, diffoscope, django-restricted-resource, django-testproject, django-testscenarios, gitlab-ci-multi-runner, gnome-shell-extension-taskbar, golang-github-flynn-archive-go-shlex, golang-github-jamesclonk-vultr, golang-github-weppos-dnsimple-go, golang-golang-x-time, google-android-ndk-installer, haskell-expiring-cache-map, haskell-hclip, haskell-hdbc-session, haskell-microlens-ghc, haskell-names-th, haskell-persistable-record, haskell-should-not-typecheck, haskell-soap, haskell-soap-tls, haskell-th-reify-compat, haskell-with-location, haskell-wreq, kbtin, libclipboard-perl, libgtk3-simplelist-perl, libjs-jquery-selectize.js, liblemon, libplack-middleware-header-perl, libreoffice, libreswan, libtest-deep-json-perl, libtest-timer-perl, linux, linux-signed, live-tasks, llvm-toolchain-3.8, llvm-toolchain-snapshot, lua-luv, lua-torch-image, lua-torch-nn, magic-wormhole, mini-buildd, ncbi-vdb, node-ast-util, node-es6-module-transpiler, node-es6-promise, node-inline-source-map, node-number-is-nan, node-object-assign, nvidia-graphics-drivers, openhft-chronicle-bytes, openhft-chronicle-core, openhft-chronicle-network, openhft-chronicle-threads, openhft-chronicle-wire, pycodestyle, python-aptly, python-atomicwrites, python-click-log, python-django-casclient, python-git-os-job, python-hypothesis, python-nosehtmloutput, python-overpy, python-parsel, python-prov, python-py, python-schema, python-tackerclient, python-tornado, pyvo, r-cran-cairo, r-cran-mi, r-cran-rcppgsl, r-cran-sem, ruby-curses, ruby-fog-rackspace, ruby-mixlib-archive, ruby-tzinfo-data, salt-formula-swift, scapy3k, self-destructing-cookies, trollius-redis & websploit.