(This article was first published on Data Driven Security, and kindly contributed to R-bloggers)
We’ve got some new and updated R packages that are (hopefully) helpful to security folks who are endeavouring to use R in their quest to find and prevent malicious activity. All packages now incorporate a testthat workflow and are fully roxygen-ized and present some best practices in R package development (a post on that very topic is pending).
We’ll start with the old and work our way to the new…
Changes to the resolv package
I’ve updated resolv for the newest Rcpp and for a better build on linux and OS X systems (still no Windows compatibiity). The package also includes vectorized versions of the core resolv_ functions. Here’s an example:
Doing all 1M would take a short while, but it’d be an interesting experiment to run (then, analyze the records to see which services these sites trust with their mail sending).
Updates to the netintel package
The netintel package is back from the dead! (thanks to a helpful push by David Severski).
All core functions have been re-written and the package now uses httr and data.table in some places for better realiability and speed. Functions that take AS numbers as parameters automagically strip or add the AS prefix as needed. To remind or introduce you to some of the workings:
You could then look up each peer and see how “connected” the top 10 are.
Introducing iptools
The iptools package is a set of tools for a working with IPv4 addresses. The aim is to provide functionality not presently available with any existing R package and to do so with as much speed as possible. To that end, many of the operations are written in Rcpp and require installation of the Boost libraries. A current, lofty goal is to mimic most of the functionality of the Python iptools module and make IP addresses first class R objects.
While resolv provides many helpful DNS functions, it is dependent upon the ldns library, which may not ever work well under Windows+Rcpp. The iptools package provides minimally featured functions for IPv4 PTR/A record lookups in an effort to (hopefully) make it usable under Windows.
The package also uses the v1 GeoLite MaxMind library to perform basic geolocation of a given IPv4 address. You must manually install both the maxmind library (brew install geoip on OS X, sudo apt-get install libgeoip-dev on Ubuntu) and the GeoLiteCity.dat http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz & GeoLiteASNum.dat http://geolite.maxmind.com/download/geoip/database/GeoLiteASNum.dat.gz files for the geolocation/ASN functions to work. If there’s interest in porting to the newer library/GeoLite2 format, I’ll consider updating the package.
The following functions are implemented:
Revolver-ish
gethostbyaddr - Returns all PTR records associated with an IPv4 address
gethostsbyaddr - Vectorized version of gethostbyaddr
gethostbyname - Returns all A records associated with a hostname
gethostsbyname - Vectorized version of gethostbyname
IP int/string conversion
ip2long - Character (dotted-decimal) IPv4 Address Conversion to long integer
long2ip - Intger IPv4 Address Conversion to Character
Validation
validateIP - Validate IPv4 addresses in dotted-decimal notation
validateCIDR - Validate IPv4 CIDRs in dotted-decimal slash notation
Geo/ASN Lookup
geoip - Perform (local) maxmind geolocation on IPv4 addresses (see ?geoip for details)
asnip - Perform (local) maxmind AS # & org lookup on IPv4 addresses (see ?asnip for details)
Testing
randomIPs - generate a vector of valid, random IPv4 addresses (very helpful for testing)
The following data sets are included:
ianaports - IANA Service Name and Transport Protocol Port Number Registry
ianaipv4spar - IANA IPv4 Special-Purpose Address Registry
ianaipv4assignments - IANA IPv4 Address Space Registry
ianarootzonetlds - IANA Root Zone Database
ianaprotocolnumbers - IANA Protocol Numbers
iptools Installation
NOTE: Under Ubuntu (it probably applies to other variants), this only works with the current version (1.55) of the boost library, which I installed via the launchpad boost-latest package:
homebrew (OS X) users can do: brew install boost and it should #justwork.
The first person(s) to get this working under Windows/mingw + boost/Rcpp gets a free copy of our book
We’ll give you an opportunity to play with iptools before covering some examples.
You are also encouraged to drop a note in the comments here or on github with any issues, suggestions or contributions. We’ve not quite worked out how we’ll be handling public gitlab issues/comments yet, but it’s on the TODO list.
To leave a comment for the author, please follow the link and comment on his blog: Data Driven Security.
R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...