Netsparker New Automated Form Authentication Support
In Netsparker Desktop we introduced a new form authentication mechanism that makes it easier to configure scans for websites which require user authentication. The goal was to provide the user with an easy to use and error-prone system for configuring form authentication in Netsparker.
How Netsparker Form Authentication Was Working Previously
Prior to version 4, Netsparker had a HTTP-macro based form authentication mechanism where you had to interactively record the sequence of your website’s form authentication. Netsparker was replaying this macro sequence to login to your website during the scan. Since this method replays the HTTP requests exactly in the order they are recorded with the exact parameter values, it was failing on websites which use token-like parameters due to cached reuse of token parameter values.
Netsparker had a powerful scripting capability which you could use to manipulate each request before it is issued but writing custom scripts would require knowledge of a .NET programming language (C# or VB.NET) and also knowledge of the Netsparker Authentication API. Also dealing with nitty-gritty details of HTTP requests, especially for cases where there are dozens of them, was not an ideal solution.
Form Authentication Support in Netsparker
The new form authentication mechanism introduced in Netsparker Desktop 4 approaches this problem from a completely different way and tries to solve it thinking out of the box, so to say. Netsparker 4 tries to fill and submit the login forms on your websites by means of the DOM of the login form page, i.e. without the need to record any login macros. It automatically detects the login form components, the username & password inputs, populates them and submits the login form. To do this, you only need to configure the following (as shown in the below screenshot):
Login form URL
Credentials (username & password)
How to Select a Login Form URL
The login form URL that should be configured in Netsparker is the URL of the page where your login form resides. Most websites these days have URLs like http://www.example.com/Login/ for their users to authenticate. It is also common for websites to have their login form on their homepages, usually somewhere located in the header or side bar part. If this is the case, specify your website’s home page URL in Netsparker as the login form URL. Netsparker has a better chance of detecting and filling the login form on pages dedicated to the login operation, so always specify the dedicated login form URL if you have a login form both on your homepage and on a dedicated login page.
Configuring Credentials and Multiple Persona Support
The other piece of information Netsparker requires for authenticating during a web application security scan are the credentials. As seen in the below screenshot you can enter more than a single set of credentials to simulate different kind of personas your website supports, i.e. one for regular users and one for admin users. This way, you will be able to easily switch between accounts and performs authenticated scans with different user accounts which have different privileges.
Verifying the Form Authentication Configuration by Simulating the Login and Detecting the Logout Pattern
Configuring form authentication is that simple! Once ready just click the Verify Login & Logout button and Netsparker will use the provided information to authenticate to your website. During the verification process, which is required to perform authenticated scans you will confirm that Netsparker is able to:
Authenticate with the provided configuration
Detect the difference between a logged in and logged out session
You can verify the form authentication configuration either while configuring it or before starting a scan; Netsparker will prompt you to perform the verification process if it is not already performed prior to starting the scan. Verifying the logout detection pattern is crucial because Netsparker will try to detect this pattern during the scan and when this pattern is matched, it will try to re-authenticate automatically to the website before proceeding with any other crawling or attacking requests, ensuring the password protected section is completely scanned. There are 2 types of logout detection patterns that Netsparker identifies:
Redirect-based
Keyword-based
Logout Detection
Redirect-based logout detection
A great deal of websites redirect the users back to the login form page when a restricted page is requested anonymously, without any valid session. If your website does so, Netsparker will detect a Redirect-based logout. To do this, Netsparker makes an anonymous request to a login required URL and identifies a Redirect-based logout if a HTTP 30x redirect response is detected. Netsparker simply uses the last URL that the form authentication simulation requested as the login required URL. For example, the form authentication simulation may use a URL like http://mysite.com/Dashboard/ which is a very good candidate as a login required URL.
Keyword-based logout detection
Some other websites do not issue a redirect when an anonymous request to a login required URL is performed, or the identified login required URL displays a page that is very similar to the authenticated page. In such cases Netsparker will detect and use a Keyword-based logout. This type of logout detection identifies a logged out session by searching for specified keywords in the HTTP responses. Therefore if all of the specified keywords are found in a response, Netsparker determines that the session is currently logged out, or has been invalidated.
Interactive login
Support for CAPTCHA, One Time Tokens and Two Factor Authentication Mechanisms
Many modern websites and web applications use CAPTCHA, Two factor authentication and other similar mechanisms for user authentication. Such mechanisms require you to enter a one time pin, code or other data during the authentication process, which typically is not available at the time of configuration. Netsparker supports these mechanisms through the use of the “Interactive login” feature. By checking the check box Interactive Login in the form authentication configuration, Netsparker will automatically pop up an interactive browser window during the authentication so you can perform any necessary operations to allow the authentication, be it entering a CAPTCHA or PIN sent as an SMS.
Below are some scenarios where you can use the Interactive login feature:
The website requires a CAPTCHA (to be solved during authentication or to access a particular area)
The website requires you to enter a dynamic token value like a 2FA PIN during authentication
You are not able to configure Netsparker to fill the login form and want to manually perform authentication (i.e. the login form is rendered as a Flash/Silverlight/Java Applet embed in browser)
You can also mix and match the Interactive login functionality with automatic login capability and custom scripting support. For example imagine having a website requiring you to enter a regular username and password on the first page and a 2FA PIN on second page. In such case you can configure the credentials and enable the Interactive login option, therefore Netsparker will first submit the regular login form details and then will prompt you with the interactive login browser allowing you to enter the 2FA PIN.
Combining the Use or Multiple Website Authentication Mechanisms
If your form authentication pages also require other type of authentication such as basic or client certificate, you can simply configure the settings together with the form authentication settings in the Start a New Scan dialog. When you configure them Netsparker will automatically use all the configured information when accessing the form authentication pages during a web application security scan.
Custom Scripts for Form Authentication in Netsparker
There will be cases where you need to tweak the Netsparker automatic authentication, or it will not work for your website. Do not fret; with the help of custom scripting support, you can automate the form authentication process of your websites easily. Here are some of the cases where you may need custom scripting:
The login form is not a regular login form with 2 input fields, for example you need to select a department from a select box or a drop down menu
The submit button on login form is not a regular HTML button
There are multiple forms in the login form page and Netsparker is unable to detect the correct form. For example Netsparker locates the signup form which is on the same page
The login form page is not present in the DOM by the time the page loads, but you need to click a link to make login form appear on page, usually in a virtual login dialog
The authentication process consists of several page navigations; you need to first visit a page to get a cookie and then visit another to enter the username and then yet another to enter the password
Netsparker is unable to locate the login form for various other reasons.
Custom scripts for form authentication in Netsparker can be written with the JavaScript language, which many professionals are familiar with. Netsparker executes the JavaScript code in the context of form authentication pages, where you can access and manipulate the page DOM. The code will be executed by the time the page is fully loaded. You can use any HTML API that modern browsers support to locate the login form elements and fill them.
The Custom Script editor provides you with a convenient environment to author your custom scripts. It mainly consists of 3 parts: A script editor, an embedded browser view and a developer tools pane.
You can write your scripts in the script editor on left. Any HTML/JavaScript/DOM API that is supported on a modern browser is supported here too. Additionally, you can use some of the helper functions provided by Netsparker in ns.auth namespace. These functions will help you to fill input values, click elements, etc. You can read more about how to call these functions in the Form Authentication API documentation.
The browser view on right helps you preview the login form page and generate code for elements on authentication pages. This window initially loads the login form URL when opened. You can right click elements on the page to get the context menu with code generation options. You can generate code that either works immediately or works after a delay by using “Generate element code” and “Generate element code (delay 2000ms)” menu items respectively.
When you click these menu items, a single line of code will be appended to the script editor on left. If you have generated code for an input value, a JavaScript code that sets a value will be generated. If you generate code for an element like a button or an anchor, a JavaScript code that clicks that element will be generated.
You can also customize the code that is automatically generated. For example, you can replace the variable username generated for a setValueByQuery call with a hardcoded JavaScript string like john.doe (use of dynamic variables like username and password is encouraged though, if you mean to supply credential values). There are 2 variables username and password available to your scripts as mentioned above. These variables will contain the credentials of the active persona at the time of execution and by using these variables, your script will be generic enough to support multiple persona feature.
The custom script editor also has a developer tools pane right below the browser view on lower right corner of the editor. This pane has all sorts of web development helper tabs that a seasoned web professional would be familiar with. All these tabs work in the context of the current loaded page in browser view and you can:
Inspect the current states of the HTML elements on Elements tab,
Monitor the HTTP requests on Network tab,
See logs and execute script code on Console tab,
etc.
There are several toolbar buttons on this window:
Load Login Form (F4): By pressing this button, a new browser view instance will be created and the specified login form URL will be loaded into this browser view. No custom script code is executed when you press this button and any cookie value will vanish from the previous browser view.
Test Script (F5): You can start executing the custom script code you have written by pressing this button. Your login form URL will be loaded into a new browser view and the script execution will start upon login form load completion. If you have several pages of custom script written, all of them will be executed in the order they are written, you can see the status of current page and the status of script execution right next to the address bar above the browser view.
Clear: This button helps you clear all the code on current page script editor.
Templates: Some sample script templates are listed under this drop down button. Selecting one of these will load the predefined script. You can start with these sample codes and tweak this script to suit your needs.
The Generate optimized code check box below to the browser view is checked by default. When enabled the Generate element menu items will try to generate most optimized and shortest CSS query code possible. For example if you have an HTML element with an id value, since id values in a HTML document uniquely identifies an element, a very concise CSS query selector with this value will be generated. In some cases, you may have randomly generated id values for your elements and having different values each time the page is loaded, hence in these cases case you may want to uncheck this option to generate an alternative CSS query that is not using the id value.
Executing scripts on multiple pages
Custom scripting supports scripting against multiple pages if your form authentication consists of multiple pages or has redirects. For most of these scenarios a single page of custom script will help you authenticate with the website. The following screen capture demonstrates a form authentication case where the username (email address in this case) needs to be entered on the first page and password needs to be entered on the next page navigated:
Since there is a brand new document context after each page is loaded, you are required to enter your custom script code to separate pages dedicated to that page. Netsparker will give you a chance to execute your custom script code after each page navigation during the form authentication process, therefore all you need to do is create script pages on this window and write the corresponding piece of code for that page.
Form Authentication Troubleshooting, Tips & Tricks
Q: My login form is dynamically rendered inside an inline dialog and Netsparker cannot find it, how can I fill that login form?
Write a custom script that first clicks the link or button that triggers the dialog and populate the login form after a delay:
ns.auth.clickByQuery('#header > div.row > a:nth-child(1)'); // Trigger the login dialog
ns.auth.setValueByQuery('#email', username, 2000);
ns.auth.setValueByQuery('#password', password, 2000);
ns.auth.clickByQuery('#login-button', 3000);
The code above will first trigger the login dialog (first line), fill username & password after 2 seconds and click the login button in dialog on 3rd second.
Q: My login form has some other fields along with username and password, how can I fill that login form?
A: Write custom script to fill username and password from current persona variables, hardcode the rest of the credentials to your script:
ns.auth.setValueByQuery('#Username', username);
ns.auth.setValueByQuery('#Password', password);
ns.auth.setValueByQuery('#LoginCode', '4815162342'); // Hard-coded extra credential
ns.auth.clickByQuery('#LoginButton');
Q: How can I provide custom cookies that are required during form authentication?
A: Specify the cookies in the Custom Cookies section of the General section of your current scan profile. These cookies will be issued during the form authentication requests.
Q: How can I provide custom header values or change the user agent string during form authentication?
A: You can create a scan policy with custom header values and/or modified user agent strings and select it on the current profile during form authentication.
Q: My site requires me to visit some pages before displaying the login form URL and I cannot use the login form URL directly, how should I authenticate?
A: Use the first page that is required to be visited as the Login form URL. Then, using custom scripting, write code that performs navigation for each page that needs to be visited. You can click the HTML elements via scripting or simply use code like the following to just perform the navigation:
document.location = 'https://mysite.com/login/next_page.htm';
Q: My site performs several redirects before reaching the login form, how can I write custom script code for the login form?
A: Create custom script pages for each redirect leaving the script editor empty, and write your custom script for login form on the last page. Netsparker won’t run any code for pages that perform the redirect.
Q: I need to run some script code after certain amount of time, how can I do that?
A: Use the built-in setTimeout JavaScript function:
setTimeout(function() {
// Write your JavaScript code here to execute after 2000 milliseconds
}, 2000);
TIP: Use the functions provided by Netsparker Form Authentication API (ns.auth) to set input values or click elements. These functions do not simply set values or click elements, but also simulates any necessary JavaScript events that are triggered when a user performs these tasks. Some JavaScript frameworks require these events to be fired so simply setting input values or clicking elements may not be enough.
TIP: You do not necessarily need to click the Login button on your page. If you have a JavaScript function that performs the login, you can call that function after populating the login form:
ns.auth.setValueByQuery('#Username', username);
ns.auth.setValueByQuery('#Password', password);
MyApp.LoginController.DoLogin();
Known issues
Netsparker does not have scripting support for popups opened during form authentication process. Please use the URL loaded into the popup window as your Login form URL if that is possible.
Generate element code menu items do not generate the correct CSS query for elements that are inside frames. You need to locate the elements manually by code.
https://lh6.googleusercontent.com/pUpNZvfBziUwfl2MwMXOb_R2Fq_wjThjA14I38no_cIhaBe4ow6NQwU-fR2oMlKyrXWOgQ9xBYiBd3Kd843zBsfKFN6QbVmxSDRne357MqkX7hej-sAO3JNbWDf0TP0V935CoFo
https://lh6.googleusercontent.com/pUpNZvfBziUwfl2MwMXOb_R2Fq_wjThjA14I38no_cIhaBe4ow6NQwU-fR2oMlKyrXWOgQ9xBYiBd3Kd843zBsfKFN6QbVmxSDRne357MqkX7hej-sAO3JNbWDf0TP0V935CoFo
https://lh6.googleusercontent.com/pUpNZvfBziUwfl2MwMXOb_R2Fq_wjThjA14I38no_cIhaBe4ow6NQwU-fR2oMlKyrXWOgQ9xBYiBd3Kd843zBsfKFN6QbVmxSDRne357MqkX7hej-sAO3JNbWDf0TP0V935CoFo
document.location = 'https://mysite.com/login/next_page.htm';