2014-08-12

(This article was first published on Data Driven Security, and kindly contributed to R-bloggers)

We’ve got some new and updated R packages that are (hopefully) helpful to security folks who are endeavouring to use R in their quest to find and prevent malicious activity. All packages now incorporate a testthat workflow and are fully roxygen-ized and present some best practices in R package development (a post on that very topic is pending).

We’ll start with the old and work our way to the new…

Changes to the resolv package

I’ve updated resolv for the newest Rcpp and for a better build on linux and OS X systems (still no Windows compatibiity). The package also includes vectorized versions of the core resolv_ functions. Here’s an example:

Doing all 1M would take a short while, but it’d be an interesting experiment to run (then, analyze the records to see which services these sites trust with their mail sending).

Updates to the netintel package

The netintel package is back from the dead! (thanks to a helpful push by David Severski).

All core functions have been re-written and the package now uses httr and data.table in some places for better realiability and speed. Functions that take AS numbers as parameters automagically strip or add the AS prefix as needed. To remind or introduce you to some of the workings:

You could then look up each peer and see how “connected” the top 10 are.

Introducing iptools

The iptools package is a set of tools for a working with IPv4 addresses. The aim is to provide functionality not presently available with any existing R package and to do so with as much speed as possible. To that end, many of the operations are written in Rcpp and require installation of the Boost libraries. A current, lofty goal is to mimic most of the functionality of the Python iptools module and make IP addresses first class R objects.

While resolv provides many helpful DNS functions, it is dependent upon the ldns library, which may not ever work well under Windows+Rcpp. The iptools package provides minimally featured functions for IPv4 PTR/A record lookups in an effort to (hopefully) make it usable under Windows.

The package also uses the v1 GeoLite MaxMind library to perform basic geolocation of a given IPv4 address. You must manually install both the maxmind library (brew install geoip on OS X, sudo apt-get install libgeoip-dev on Ubuntu) and the GeoLiteCity.dat http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz & GeoLiteASNum.dat http://geolite.maxmind.com/download/geoip/database/GeoLiteASNum.dat.gz files for the geolocation/ASN functions to work. If there’s interest in porting to the newer library/GeoLite2 format, I’ll consider updating the package.

The following functions are implemented:

Revolver-ish

gethostbyaddr - Returns all PTR records associated with an IPv4 address

gethostsbyaddr - Vectorized version of gethostbyaddr

gethostbyname - Returns all A records associated with a hostname

gethostsbyname - Vectorized version of gethostbyname

IP int/string conversion

ip2long - Character (dotted-decimal) IPv4 Address Conversion to long integer

long2ip - Intger IPv4 Address Conversion to Character

Validation

validateIP - Validate IPv4 addresses in dotted-decimal notation

validateCIDR - Validate IPv4 CIDRs in dotted-decimal slash notation

Geo/ASN Lookup

geoip - Perform (local) maxmind geolocation on IPv4 addresses (see ?geoip for details)

asnip - Perform (local) maxmind AS # & org lookup on IPv4 addresses (see ?asnip for details)

Testing

randomIPs - generate a vector of valid, random IPv4 addresses (very helpful for testing)

The following data sets are included:

ianaports - IANA Service Name and Transport Protocol Port Number Registry

ianaipv4spar - IANA IPv4 Special-Purpose Address Registry

ianaipv4assignments - IANA IPv4 Address Space Registry

ianarootzonetlds - IANA Root Zone Database

ianaprotocolnumbers - IANA Protocol Numbers

iptools Installation

NOTE: Under Ubuntu (it probably applies to other variants), this only works with the current version (1.55) of the boost library, which I installed via the launchpad boost-latest package:

homebrew (OS X) users can do: brew install boost and it should #justwork.

The first person(s) to get this working under Windows/mingw + boost/Rcpp gets a free copy of our book

We’ll give you an opportunity to play with iptools before covering some examples.

You are also encouraged to drop a note in the comments here or on github with any issues, suggestions or contributions. We’ve not quite worked out how we’ll be handling public gitlab issues/comments yet, but it’s on the TODO list.

To leave a comment for the author, please follow the link and comment on his blog: Data Driven Security.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

Show more