2014-04-25
(This article was first published on Data Driven Security, and kindly contributed to R-bloggers)
Technically this is Part 2 of Firewall-busting ASN-lookups. However, I said (in Part 1) that Part 2 would be about making a vectorized version and this is absolutely not about that. Rather than fib, I merely misdirect. Moving on…
As you can see in Part 1, we have to resort to a system() call to do the TXT record lookup with dig. Frankly, I really dislike that. It’s somewhat sloppy, wasteful of resources and we can do better. Much better (initially, just a little better, tho). R, like most modern interpreted languages, has a C interface. Hadley Wickham goes into far more detail in his epic online (and I’m assuming soon-to-be print) book Advanced R Programming than I will be doing in this post and Jonathan Callahan also has some great in-depth material you should review. You might want to take a peed at some of Dirk Eddelbuettel‘s work, too. This post will (should?) get you jumpstarted with the basics of integrating C & R and will dovetail nicely with Part 2 of the proper series, since we’ll not only be creating a vectorized version of the ip2asn() function but will also be putting it into a proper R package.
Peeking Under The Covers
Even if you’ve only dabbled with R, you’ve already used traditional C-backed functions, and if you’ve done more extensive computing with R—say, worked with the RMySQL package to connect to a database—then you’ve absolutely used the more “modern”/prevalent Rcpp-backed functions. For example, the mysqlCloseConnection looks like this:
PROTIP: you can see the source code for any R function by just typing the function name sans-parentheses & parameters at the R console prompt
Towards the bottom of the above code listing, you’ll see the .Call("RS_MySQL_closeConnection"...) line which is reaching out to the underlying C/C++ code (in RS-MySQL.c) that makes up the libary. Here’s the definition for that function:
Now, we’re not working with MySQL in this post, but that’s a fairly familiar tool and provides the framework for us to discuss how we’ll be using C/C++, Rcpp & .Call() to make DNS calls from R in a much more efficient manner.
Picking A DNS Library
For folks familiar with DNS, you may be thinking that we’re going to build an interface to the trusted old standard BIND libresolv library. While that was an option, we’re going to skip with tradition and use the ldns library from NLNet Labs, makers of the #spiffy Unbound validating recursive caching resolver (which uses ldns). Their ldns implementation has a simple but also robust API which supports IPv4, IPv6, TSIG & DNSEC plus is wicked fast, small and can make synchronous calls (which makes it easier to do a basic port). If you’re running Mac OS X, you’ll need to either use Homebrew or MacPorts or compile the library from source. I prefer Homebrew, and used:
For Linux users, you’ll need both the ldns library and the bsd library (the latter primarily for strlcopy). I gravitate towards Ubuntu for Linux and used the following there:
You’ll note the lack of a Windows section. Consider this an open offer to anyone on Windows to augment our blog with a Windows version. The Rtools package can help you get started. Hit us up for details on how to join in the fun!
We are making the broad assumption that you have the necessary development environment setup on either Linux or Mac OS X. It’s unlikely you’d be this far along in the post if not :-)
Starting Small
Rather than build an entire R interace to the whole ldns library, we’re going to focus this post on:
Getting a small Rcpp example built
Interfacing with ldns to retreive a TXT record
Building an ip2asn() R function that uses this new capability
NOTE: all of the code for this post is in this gist. You can download them all in one fell
swoop with:
git clone https://gist.github.com/hrbrmstr/11286662,
and we’ll have a proper repository for the full package impementation in later posts.
We’ll begin with having you install the Rcpp package. Fire up an R console (or use the RStudio R console pane) and do:
Next, create a directory (perhaps ip2asn for this limited example) and put the following code block into the file txt.cpp (or just use the one you cloned above):
The code is commented pretty well and I won’t be covering all of the nuances of the individual ldns calls. Please note that the function has minimal error checking since it is serving first and foremost as a compact example. The full package version will have all i’s dotted and t’s crossed and I’ll make it a point to show the differences between a “toy” example and production-worthy code when we post the package follow-up.
The code flow pattern will be the same for most of these API library mappings:
define data types that need to be passed in and returned
convert them to structures C/C++ can handle
perform your calculations/operations on that converted data
clean up after yourself
return a value R can handle
To compile that code into an object we can use in R, you need to do the following:
The export lines setup environment variables that help R/Rcpp know where to look for libraries and define the proper compiler flags for your environment. The last line does the hard work of building the proper compilation and linking directives/commands. All three of them belong in a proper Makefile (or your build system of choice). Again, we’re taking a few shortcuts to make the overall concept a bit more digestible. Complexity coming soon!
If the build was successful, you’ll have txt.o and txt.so files in your directory. Now, on to the good bits!
Interfacing With R
Having a compiled object is all well and good, but we need to be able to access the txt() function from R. It turns out that this part is pretty straightforward. Put the following into a file (perhaps ip2asn.R) or use the gist version:
To use this new function, make sure your R session is in the working directory of the library (via setwd()) and do:
That uses the function default IP address, but you can use any IP (and, it still only works with a single IP address). Kittens and polar bears will suffer greatly if you pass in anything but a single, 100% valid IP address (see, error checking saves wildlife and pets), but it gets the job done without a system() call and sets us up nicely for adding more capability.
Wrapping Up
We gave you a whirlwind tour of interfacing with R and we’ll be re-visitng this topic in later posts. If any parts were a bit confusing or your setup has some errors, drop a note in the comments here or over at the gist and we’ll do our best to help you out.
To leave a comment for the author, please follow the link and comment on his blog: Data Driven Security.
R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...