Performance Testing Your Force.com Application
One advantage of the Force.com platform is that it allows you to develop applications quickly and easily using both standard and custom features, such as custom Visualforce pages and Apex classes.
But how can you test whether your application can scale to meet the needs of your growing user base? The Force.com platform includes a framework for creating and running test classes, but when you need to account for larger volumes of data and more demanding application conditions, you might need to move a level beyond unit testing.
A previous Technical Enablement article covered a strategy called performance profiling, which involves investigating and planning for performance bottlenecks. This blog post helps you plan for growth by outlining your application testing options, explaining which ones you should use and when you should use them, and suggesting how you should work with salesforce.com Customer Support to maximize your tests’ effectiveness and value.
Types of Testing
Stress testing
Stress testing is used to gauge performance during abnormal or extreme conditions, which are often caused by diminished resources or excessive requests. The main goal of stress testing is to find the boundaries of a system.
Load testing
Load testing is used to gauge performance under expected conditions with varying loads (i.e., increasing numbers of users or transactions) and configurations.
Performance testing
Performance testing measures performance under a particular workload, and includes both stress testing and load testing.
Do you need to stress test the Force.com platform?
No. The Force.com platform serves over 1 billion transactions daily with an average response time of under 300 milliseconds, and salesforce.com tests and monitors the platform to ensure that it has excess capacity.
You should definitely take advantage of the platform’s efficiency–but not by writing inefficient code. Salesforce.com guards against highly inefficient code by using governor limits, which help ensure both that your users have a consistent application experience and that resources are available to all salesforce.com customers. (One limit, the Apex Apex script statement limit, is being phased out in the Winter ’14 release, but its original reason for existing was the same.)
Your goal should be to use load testing instead of stress testing. If you try to stress test, you’re likely to reach governor limits or have salesforce.com terminate your tests so they don’t affect shared resources on the multitenant Force.com platform. Performance test your anticipated loads instead.
Note: Salesforce.com allows performance testing only in sandbox environments, which also enforce governor limits.
When should you performance test on the Force.com platform?
Performance test if you have highly customized code or large transactional volumes.
The goals of performance testing on the Force.com platform are to:
Ensure that custom application or business logic meets your desired response times
Determine whether your estimated transaction throughput of anticipated loads is accurate
Before you consider using performance testing, unit test specific features to identify performance benchmarks for your requests. You can use the Developer Console, Workbench, and your debug logs to identify performance issues associated with your Visualforce pages and Apex classes; this article about performance profiling has a lot of information that can help you get started.
Sandbox vs. Production
Don’t benchmark performance in a sandbox organization anticipating the same performance in your production organization. Due to variables such as data distribution and user load patterns, which can be fluctuate because of database caching and the queue performance associated with asynchronous processing (@future calls), requests might perform faster or slower in production.
You can use performance testing in sandbox to uncover conflicts between requests. These conflicts might include locking issues that are caused by data skew or user requests that stall because of an integration’s DML operations. In addition, you can load test in your sandbox organization to test the performance of external Web services.
Requesting Performance Tests
Before you can performance test in your sandbox environment, you must first create a test plan and submit it to salesforce.com Customer Support.
The test plan should include:
A description of your tests
Record counts of the objects in your tests
The test scripts and tool (e.g., LoadRunner) that your tests will use
Your estimated testing loads
When you plan to conduct the tests
Your sandbox organization’s and production organization’s IDs
Contact information that salesforce.com can use to reach you during your tests
Once salesforce.com Customer Support approves your test plan, salesforce.com can help monitor your tests.
Defining the Test Plan
Identify your key business processes and build test scripts around them. You can use tools like LoadRunner to simulate concurrent users and put your application through real-life loads. If you have inbound/outbound integrations, ensure that they are included in your test scripts. Run a very low-volume test to validate these scripts. You can then compare the metrics from this small test with the baseline numbers from the unit tests.
Estimating Your Testing Load
To estimate your load, use TPS (transactions per second) as a metric. Consider the following example.
A service cloud customer has 3,000 support representatives handling five cases per hour, and each case executes 10 transactions (for Visualforce pages, Javascript Remoting, SOQL, workflow rules, and triggers). In addition, the customer application has 1,000 inbound/outbound API calls per hour, each of which executes 10 transactions.
To calculate TPS, you would make the following calculation.
((15,000 cases x 10 transactions) + (1,000 API calls x 10 transactions)) / 3,600 = 44.4 TPS
When calculating TPS, remember to factor in the potential wait time and pauses between requests. Realistic load values for the rounds of tests would be 22 TPS (~0.5 x 44.4), 33 TPS (~0.75 x 44.4), and 44 TPS. You might also want to test for growth, which would involve testing at 1.5 or 2 times your calculated TPS.
Monitoring Each Round of Testing
Monitor each round of testing as the load increases and compare your results with your projected metrics. Salesforce.com might contact you to abort the tests if their performance approaches unhealthy thresholds. Proper unit testing, accurate test scripts, and realistic estimated loads can help you avoid this scenario, and salesforce.com Customer Support can provide metrics from the salesforce.com logs to help you with your post-test analysis.
Summary
The Force.com platform eliminates the need for stress testing. If you want to ensure that your application can meet the growing needs of your growing user base, use unit testing and performance testing. Use the Developer Console and browser-specific tools, such as Firebug or Chrome Developer Tools, to identify tuning opportunities, and to provide performance benchmarks for your Visualforce pages and Apex classes.
To performance test in your sandbox organization, submit a detailed performance test plan to salesforce.com Customer Support. After salesforce.com Customer Support approves your test plan, you can analyze each round of your load tests and compare them with your expected metrics.
Related Resources
How to Write Good Unit Tests
A Guide to Application Performance Profiling in Force.com
Best Practices for Deployments with Large Data Volumes
Architect Core Resources
About the Author and CCE Technical Enablement
John Tan is an Architect Evangelist within the Technical Enablement of the salesforce.com Customer-Centric Engineering group. The team’s mission is to help customers understand how to implement technically sound Salesforce solutions. Check out all of the resources that this team maintains on the Architect Core Resources page of Developer Force.