The SystemTap team announces release 2.7, RedGrizzlyCoffee!
Kernel tracepoint subsystem probes, optimized string passing, syscall tapset
improvements, highlighted educational examples, addr->file:line mapping,
SystemTap's 10th anniversary, and more!
= Where to get it
https://sourceware.org/systemtap/ - our project page
https://sourceware.org/systemtap/ftp/releases/systemtap-2.7.tar.gz
https://koji.fedoraproject.org/koji/packageinfo?packageID=615
git tag release-2.7 (commit bf16266782e1f25)
There have been over 290 commits since the last release.
There have been over 35 bugs fixed / features added since the last release.
= How to build it
See the README and NEWS files at
https://sourceware.org/git/?p=systemtap.git;a=tree
Further information at https://sourceware.org/systemtap/wiki/
= SystemTap's 10th Anniversary
February 12 was the 10th anniversary of SystemTap. We would to thank all of
our contributors over the years, listed below.
Aaron Tomlin, Abegail Jakop, Adrien Kunysz, Alexander Lochmann,
Alexander Y. Fomichev, Alfred Landrum, Ananth N Mavinakayanahalli,
Andre Detsch, Andreas Müller, Andy Fitzsimon, Anil Keshavamurthy,
Anithra Janakiraman, Anton Vorontsov, Atsushi Tsuji, Brendan Gregg,
Breno Leitao, Brent Baude, Brian Chrisman, Bryn M. Reeves, Casey Dahlin,
Charles Spirakis, Charley Wang, Chris Meek, Chung-Lin Tang, Dan Horak,
Daniel Migault, Daniel Tschan, Dave Brolley, Dave Nomura, David J. Wilder,
David Juran, David Smith, Dennis Gilmore, Diego Elio Pettenò, Don Domingo,
Elliott Baron, Erick Tryzelaar, Eugene Teo, Eugeniy Meshcheryakov,
Fabio Olive Leite, Fahad Arshad, Felix Lin, Frank Ch. Eigler,
Frederic Turgis, Fredrik Östman, Grant Edwards, Graydon Hoare, Han Pingtian,
Hien Nguyen, Hilko Bengen, Honggyu Kim, Hushan Jia, James Bottomley,
Jan Kratochvil, Jeff Moyer, Jim Keniston, Jiri Horky, Jiri Slaby,
JoeLynn Keniston, Jonathan Lebon, Josh Stone, Jóhann B. Guðmundsson,
K.Prasad, Kai Meyer, Kapileshwar Singh (KP), Keiichi KII, Kent Sebastian,
Kevin Stafford, Kim van der Riet, Kiran Prakesh, Lars R. Damerow,
Li Guanglei, Lubomir Rintel, Lukas Berk, Lukas Czerner, Lukáš Hejtmánek,
Mahesh J Salgaonkar, Malte Nuhn, Maran Pakkirisamy, Marc Milgram,
Mark McLoughlin, Mark Wielaard, Martin Cermak, Martin Hunt, Martin Lazar,
Masami Hiramatsu, Masanari Iida, Masatake YAMATO, Maynard Johnson,
Michael Meeks, Mike Mason, Miroslav Franc, Naresh Kamboju, Nathan Scott,
Negreanu Marius Adrian, Nitin A Kamble, Nobuhiro Tachino,
Oleksandr Chumachenko, Otavio Pontes, Paddie O'Brien, Peter Feiner,
Peter Kjellström, Peter Robinson, Petr Muller, Phil Muldoon, Prerna Saxena,
Przemysław Pawełczyk, Quentin Barnes, Rajan Arora, Rajasekhar Duddu,
Raphaël Beamonte, Rayson Ho, Rich Megginson, Robb Romans, Robin Hack,
Robin Lee, Roland Grunberg, Roland McGrath, Rüdiger Sonderfeld,
Santosh Shukla, Serguei Makarov, Shaohua Li, Siddhesh Poyarekar,
Srikar Dronamraju, Srinivasa DS, Stan Cox, Stefan Hajnoczi, Steve Dickson,
Sunzen Wang, Tetsuo Handa, Thang Nguyen, Theodore Ts'o, Tim Moore,
Timo Juhani Lindfors, Tom Callaway, Tom Tromey, Tom Zanussi, Tomoki Sekiyama,
Tony Jones, Torsten Polle, Varun Chandramohan, Victor Kamensky,
Vincent Bernat, Wade Farnsworth, Wenji Huang, William Cohen,
Yaakov Selkowitz, Yang Wen, Yichun Zhang (agentzh), Yoshihide Sonoda,
Zhaolei, Zifei Tong, ch huang
As a thank you for your contribution:
https://sourceware.org/ml/systemtap/2015-q1/msg00021.html
= SystemTap frontend (stap) changes
- SystemTap now allows .function probes to be specified by their full function
name, file, and declaration line number. Use the .statement probe to probe a
specific line number.
- Tracepoint probes can now also be specified by the target subsystem. For
example, the following are all supported:
probe kernel.trace("sched:sched_switch") --> probe sched_switch found in the
sched subsystem
probe kernel.trace("sched:*") --> probe all tracepoints in sched subsystem
As a result, tapset functions such as pn() will now return a different string
than before. To retain the previous behaviour, use '--compatible=2.6'.
= SystemTap runtime changes
- Passing strings to and from functions has become faster due to optimization
(passing some strings by reference instead of by value/copy). It may
be disabled by using the unoptimize flag (-u).
To make embedded-C functions eligible for the same optimization, use the pragma
/* unmodified-fnargs */ to indicate that the function body will not modify
the function arguments. Remember to use MAXSTRINGLEN for string length,
rather than sizeof(string_arg) (which might now be a pointer).
= SystemTap tapset changes
- Changed tapsets:
aux_syscalls.stp returns the pointer value instead of "UNKOWN" for an
invalid pointer. improved _msync_flag_str() and
_mlockall_flags_str(). arm64 support in __is_user_regs
collection of new _*str and other functions
nd_syscalls.stp fix compile problem. new probe nd_syscall.finit_module
correct nodebug mkdirat syscall argstr
syscalls.stp new probe syscall.finit_module. doesn't rejecting
RUSAGE_BOTH, lets the kernel do it instead.
powerpc/syscalls.stp new probe for "ppc_newuname" kernel function
x86_64/syscalls.stp sdded syscall.compat_fadvise64 probe.
syscalls.stpm added __futex_argstr, __compat_syscall_gate_negative macro
dentry.stp Adapt to changes in the dentry structure to handle
more kernels, especially 3.18
nfs.stp new probe nfs.fop.read_iter for kernels >= 3.16
and __syscall_gate_compat_simple macros
nfs_proc.stp probe nfs.proc.rename_setup added to support
asynchronous NFS client rename operations.
nfsd.stp sewrite __svc_fh() in systemtap script language
context-symbols.stp new functions symfileline, symline and symfile
- For Linux >= 3.17, added support for getrandom, memfd_create, seccomp syscalls
within [nd_]syscalls*.stp
- For Linux >= 3.19, added support for execveat syscalls within [nd_]syscalls.stp
- More work was done to improve the types of the parameters in many
syscall probe aliases ([nd_|aux_]syscalls[2].stp). Work was also done to add
compat and 32-bit support for syscall.
- The following functions are deprecated in release 2.7:
- _adjtx_mode_str(), _statfs_f_type_str(), _waitid_opt_str(),
_internal_wait_opt_str(), and _epoll_events_str().
= SystemTap sample scripts
- New collection:
lwtools lightweight tools. example tools for systems without
kernel debuginfo
- New samples:
accept2close-nd.stp traces socket duration from the accept() syscall to
close(), and provides details on the lifespan of these
passive connections, showing the distribution as a
histogram.
biolatency-nd.stp measures block I/O latency (storage I/O, ie, disk I/O),
and shows the distribution as a histogram
bitesize-nd.stp uses the kernel tracepoint block_rq_insert to read the
size of I/O. The output includes the name of the
process or thread that was on-CPU when the I/O request
was inserted on the issue queue.
execsnoop-nd.stp can identify if CPU is consumed by short-lived
processes, by tracing new process execution. will not
catch new processes that only fork(). shows every
exec(), including those if a process re-execs.
fileline-profile.stp prints a sorted list of the top twenty kernel
and/or user processes providing file:line information,
from the samples addresses gathered.
fslatency-nd-stp dynamically traces two common file system functions:
do_sync_read() and do_sync_write(), and reports a
histogram distribution of latency.
fsslower-nd.stp dynamically traces two common file system functions:
do_sync_read() and do_sync_write(), and shows details
of each call that is slower than a threshold.
glibc-malloc.stp reports on internal statistics of the glibc malloc
implementation, as used by a process restricted by -x/-c
killsnoop-nd.stp traces signals system-wide, including those sent by
the kill(1) command, and shows various details.
opensnoop-nd.stp traces the open() syscall system-wide, to show which
files are being opened, and by who.
rwtime-nd.stp traces read() and write() syscalls, producing a
histogram summary of their durations (aka latencies).
spawn_seeker.stp prints out the local time and sorted lists of which
processes and executables spawned tasks during the
previous minute.
syscallbypid-nd.stp traces syscalls system-wide, and produces a summary
report showing their counts by process ID, process
name, and syscall types.
- Changed samples:
cycle_thief.stp improve the output to include executable name
hello-world.stp use probe oneshot
iostat-scsi.stp probes the "sd_init_command" kernel function which
replaced "sd_prep_fn" in newer kernels.
pfaults.stp index call/return arrays by tid, not pid
periodic.stp print the symbol@file:line information instead of
symdata for kernel addresses
- Some SystemTap sample scripts are now identified with the "_best" keyword,
because they are generally useful or educational. They are now promoted
within the generated index files.
= Examples of tested kernel versions
2.6.18 (RHEL 5 x86 and x86_64)
2.6.32 (RHEL 6 x86 and x86_64)
3.10.0 (RHEL 7 x86_64)
3.17.7 (Fedora 20 x86_64)
3.18.3 (Fedora 21 x86_64)
3.19 (upstream)
3.20-rc (Rawhide)
= Known issues with this release
- Some kernel crashes continue to be reported when a script probes
broad kernel function wildcards. (PR2725)
- 32-on-64 bit userspace unwinding is truncated on older kernels, such
as 2.6.32 (PR15757)
- The dyninst backend is still very much a prototype, with a number
of issues, limitations, and general teething woes. For instance:
+ lack of support for multiarch/cross-instrumentation
+ tapset functions are still incomplete relative to what is supported
when the kernel backend is active
+ exception handling becomes completely broken in programs
instrumented by the current version of dyninst (PR14702)
+ not all registers are made available on 32-bit x86 (PR15136)
See dyninst/README and the systemtap/dyninst Bugzilla component
(http://tinyurl.com/stapdyn-PR-list) if you want all the gory
details about the state of the feature.
- An upstream kernel commit #2062afb4f804a put "-fno-var-tracking-assignments"
into KCFLAGS, reducing debuginfo quality which can cause debuginfo failures.
A proposed workaround to this issue exists in:
https://lkml.org/lkml/2014/11/21/505 . Fedora kernels are not affected by
this issue.
= Contributors for this release
Abegail Jakop, *Andy Fitzsimon, *Brendan Gregg, *Chung-Lin Tang, David Smith,
Frank Ch. Eigler, *Hilko Bengen, *Hushan Jia, Jonathan Lebon, Josh Stone,
Lukas Berk, Martin Cermak, Masanari Iida, *Oleksandr Chumachenko,
*Santosh Shukla, *Siddhesh Poyarekar, Stan Cox, Stefan Hajnoczi,
William Cohen
Special thanks to new contributors, marked with '*' above.
Special thanks to Abegail for compiling these notes.
= Bugs fixed for this release <https://sourceware.org/PR#####>
11528 Pass string values by reference where possible
11644 process(PID).function/statement probes
12276 lookup functions & uploaded-data for address->file:line mappings
13330 Support TRACE_SYSTEM for tracepoints
16645 Unhelpful feedback when stap fails
16920 Add aarch64 backtrace support
17049 dtrace does not support -fPIC option anymore
17126 tracepoints.exp testcase causing stalls/hang on ppc64
17140 systemtap.examples/profiling/functioncallcount.stp causing kernel panic on s390x
17190 'stap -l' output not useful as stap input
17270 uprobes_onthefly.exp causing hang on ppc64
17292 multi-line strings broken
17395 util.cxx:1120: bad switch statement ?
17413 missing timer_create syscall in 32-on-64 mode
17462 aarch64 deprecates a number of syscalls (__NR_epoll_wait not defined)
17622 On aarch64 listing_mode.exp fails to find any probe points for
process.library(""liblisting_mode.so"").function(""libfoo"")
17688 probe nfs.fop.aio_read no longer valid
17690 probe nfs.proc3.rename no longer valid
17696 fails to find kernel tracepoints when kernel is built in a separate directory from source
17706 user string copy fault on s390 in 31-on-64 mode
17710 stap-report script contains bashisms
17714 poll.c syscall testcase
17737 ""menu"" nodes in online docs have text at the end
17738 bad formatting in online docs
17743 stap --help should print to stdout
17749 stap doesn't recognize ""++"" as a use
17807 The vfs tapset is broken on 3.18
17858 helloworld doesn't with --runtime=dyninst
17860 foreach loop fails -p4 if array slice is an expression
17864 alias_tapset.exp regressed
17883 git stap commit 81acc574 doesn't compile on rhel[56]
17884 Fresh upstream bits based on git commit 81acc574 do not compile on RHEL[56].
17888 task_dentry_path.exp got broken on el7 somewhere between b9ae0bc and b19a437
17889 the translator does not understand arm64 constants in sdt markers
17894 global_opt_unknown.exp failure
17928 syscall.exp tests fail on 32-bit arm because size_t sign extended to 64-bit value
17951 cast-user.exp regressed due to the syscall.bind tapset modification
17966 A $var guarded by @defined in return probes may still raise errors
17985 babad5b31b buildok.exp regressed on el6 compared to systemtap-2.5-5.el6
17962 dtrace.exp --no-parsing fallback test fails on rhel6
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/