I remember working on a Rails app a few years ago and someone floated the idea of using this new service that had appeared on the scene. It was called NewRelic and they were promising to give you more insight into the performance of your Rails app, than you ever could get before. We gave it a try and it was impressive, more importantly it was something the Ruby web development ecosystem truly needed.
Fast forward to now and you’d be hard-pressed to find a Ruby web application that doesn’t have NewRelic hooked in. NewRelic as a company has continued to provide tools to monitor your Ruby apps, but they’ve also branched out into a number of other languages such as Java, Python and even .Net. But of course as the number of features you provide grows so does the complexity and the amount of documentation out there. It becomes hard to figure out where to start especially if you’re not yet an expert.
Today I thought we could go back to the roots of NewRelic and look at how we can get started with the service to monitor a Rails application.
A Basic Rails App
In order to use NewRelic we need something to monitor, so let’s set up a basic ‘Hello World’ Rails app.
The app we create will live under ~/projects/tmp/newrelic, and will be called newrelic_rails1. I assume you already have Rails installed:
There isn’t much for us to do to create our ‘Hello World’ app. We need a new controller:
Now we just need a route, we will get the root route of the application to use our controller. We also need a view, with the words ‘Hello World’. Given all this, our config/routes.rb should look like this:
Our controller (app/controller/hello_controller.rb), will be as follows:
And our view (app/views/hello/index.html.erb), will be similar to:
We can now start up our development server:
When we curl localhost:3000, we get:
Everything is working!
Hooking in NewRelic
With Ruby it’s very simple. We add a gem to our Gemfile, run a bundle install, drop a config file into the config folder and we have all we need. In fact, NewRelic is pretty good at guiding you through this. All you need to do is log in to your account and if you haven’t deployed a NewRelic agent before, it’s pretty obvious what to do:
Firstly, we install the NewRelic agent gem by adding it to our Gemfile, as per the instructions:
Our Gemfile will now look like this:
Whenever we add anything to the Gemfile we need to run:
We also need a newrelic.yml, which you can download from NewRelic:
It will come pre-configured with your license key. We need to put this file under config/newrelic.yml.
At this point if we ran our application in staging or production mode, we would already get data in our NewRelic account. So let us do so:
This time when we curl localhost:3000, we get:
There is a bunch of JavaScript that got inserted into our pages so that NewRelic can monitor browser time. This is one way we can tell that our NewRelic integration is working. But it is not the only way, NewRelic also creates a log file:
We can also check our NewRelic account to make sure a new application has appeared for monitoring:
There are however a few things that are not so nice:
Our application is named ‘My Application’
We accepted all the default configuration values, which may not suit our app
We had to launch our server in production mode (which is only possible cause it’s a brand new app that doesn’t rely on any external infrastructure)
So let us look at our newrelic.yml file in a little bit more detail to see how we can monitor our app performance exactly the way we want it.
Diving in to NewRelic Configuration
First of all, the NewRelic configuration file is extremely well commented and I encourage you to read the comments for the various configuration parameters to understand what all of them do.
Secondly, NewRelic configuration is environment aware, and configuration for all environments is defined in the one newrelic.yml file, this is very similar to, how the Rails database.yml file works. We define a bunch of common configuration values and then override the relevant ones in the specific environment blocks e.g.:
We can instantly begin to see how we can fix some of the points that we raised above. If we don’t want to have to launch our app in production mode while we’re tweaking our configuration, all we have to do is enable monitoring in development mode (we will need to remember to switch this off when we’re happy with our configuration as we don’t want development data cluttering up our NewRelic account).
We should also override our application name for every environment that we have, to make sure they’re monitored separately and the application name makes sense:
With just those configuration tweaks, when we start our server in development mode and curl localhost:3000:
We’re now monitoring our application in development mode and our app name is what we expect. If your application is saying that it’s not receiving any data, give it a minute, it takes a little while for the data to start coming through.
The next most interesting (and often the most confusing) configuration value is the Apdex T-value. Unlike most of the other configuration parameters, this value does not live in the newrelic.yml file, but is instead found in the settings for the application within NewRelic:
If you want to tweak your Apdex T-value you have to do it here, but what is this parameter and what is the right value to put in it? Well, NewRelic explains it in the following way:
Your application’s Apdex T-value is set to 0.5 seconds. That means requests responding in less than 0.5 seconds are satisfying (s), responding between 0.5 seconds and 2.0 seconds are tolerating (t), and responding in more than 2.0 seconds are frustrating (f).
Essentially, NewRelic uses the Apdex value to gauge the health of your application as far as performance is concerned, so if many of the requests that are monitored by NewRelic take longer than your Apdex value, NewRelic will consider your application to be performing poorly and if you’ve set up alerts, will notify you of the fact. Basically, you have to figure out, how fast you want each server request to be fulfilled by your application, so if you’re OK with a backend request taking two seconds, you can set your Apdex value to 2.0, but if you need a response to be returned within 100ms then you should set your Apdex value to 0.1.
If you have a new application you may set the Apdex value to the performance you desire from your application. If your app is an existing one, you may have some metrics regarding how fast it is/should be performing, and you can be guided by that. All requests which are fulfilled by the server in less than the Apdex T-value will be considered by NewRelic to be fine. All requests fulfilled within Apdex * 4 seconds will be considered tolerating (i.e. users can tolerate it). All responses that take longer than Apdex * 4 will be considered frustrating (frustrated users don’t tend to stick around). So, set your Apdex T-value in such a way that you actually get useful information out of it, the actual value depends on your domain and what you want to achieve (in terms of performance), there is no right or wrong answer.
We will set our Apdex T-value to 100ms (0.1), since all we have is a ‘Hello World’ app, and it should be able to return a response very quickly (even in development mode).
Even More NewRelic Configuration
It was a little funny that most of the configuration comes from the newrelic.yml file, but the Apdex T-value is in the application settings, so NewRelic now allows you to move all the configuration values from the YAML file into NewRelic:
The advantage of this is that you don’t have to redeploy every time you want to tweak your configuration values, so it is definitely something worth considering. We will stick with the YAML file for now.
So what are some of the other useful NewRelic parameters we should know about?
Well, there is a set of parameters dealing with the NewRelic agent log file:
These have sensible defaults, but if we want the log file to go to a specific place or if we want to see more or less info in the file, we can easily control this. Since we’re just setting up NewRelic we will set the log level to debug, to make sure we don’t miss any important information (when we deploy we may want to set it to warn, or even error).
We now get a wealth of information in the log file, which (if read carefully) can give us a lot of insights into how NewRelic works:
For example we can see that:
We can switch off monitoring even if it’s switched on in the configuration file, by setting an environment variable NEWRELIC_ENABLE=false
We can see that NewRelic inserts a bunch of Rack middleware
We’re using Webrick as our server, which is obviously in development mode, but in production it would be good to confirm that NewRelic recognises the server that we’re using
NewRelic is sending data to collector.newrelic.com:443
NewRelic is sending data every 60 seconds
Real user monitoring is done via JSONP
Very useful information when you’re trying to figure out how things hang together.
Most of the other configuration parameters are pretty self explanatory e.g.:
The only other one to possibly be aware of is:
The transaction tracer captures detailed data about requests that take too long. The transaction threshold is normally a multiple (x4) of the Apdex value, but it is often useful to divorce these values from each other. You might be happy with an Apdex score of one second, but you may want to capture detailed data about requests that take 1.5 seconds or longer (instead of the four seconds or longer which would happen by default). So you can set this parameter separately:
The NewRelic Developer Mode
One of the configuration values you may have noticed was:
This should only be switched on in development (if at all). In development mode, NewRelic agent will store performance data about the last 100 requests in memory. You can look at this data at any time by hitting the /newrelic endpoint of your running application:
I hardly ever use it, but it’s there if you need it.
Notifying NewRelic of Deployments
Whenever you’re working on the performance of your application, it’s always good to know if a particular deploy has had a positive or negative effect on performance. For this purpose, you can notify NewRelic every time you perform a deploy. This way if performance degrades or improves, you’ll be able to see which deploy was the culprit. NewRelic provides Capistrano hooks to do this, but I prefer the command line way:
The key thing is to correctly supply the application name as configured in the newrelic.yml file.
We will get nice lines on the relevant NewRelic graphs to indicate when a deployment occurred.
Conclusion
You now know a whole lot about how NewRelic works and how to start using it to monitor a Rails application. But configuring things properly is only half the battle, what kind of metrics will NewRelic actually capture for you? And how can you use them to improve the performance of your application? We will look at some of these in a subsequent article. For now, have a go at configuring NewRelic for your Rails application (you’ll get a free T-shirt) and if you have any questions don’t forget to leave a comment.