Configuring a test is easy in VPT. The test run configuration screen allows you to select a script and any test data you want to run and then gives you the option to set parameters for number of users, ramp up, duration, and iteration. You can also define which geographic regions your test will be run in. The following steps will walk you through a simple test setup in VPT.
From anywhere within VPT click on the “Run a Test” button on the top navigation bar, select a project, optionally select any existing templates to base your configuration on, and click the “Run Test” button. This will take you to the test run configuration screen.
Enter a name and description for your test run, then click on the “Add Files” button to upload and select a test script. Or, choose "Select Files" to select files that were previously added. You can also create/add a new script by selecting "Create a Starter Script". For Starter Script, please refer to our article on Starter Script Builder.
The test files selector allows the user to select a test script file and any supporting data files required to run the script.
When the test script and supporting files are selected, the test files table gives the user the option to split or not split each supporting data file. (Supporting data files are split by default.)
Selecting the Split Data checkbox for a supporting file will split the data file in equal sizes of lines by the number of Load Generators assigned for the test, and send the split files to the Load Generators
Deselecting the Split Data checkbox will send a copy of the test data file unchanged to each of the Load Generators
Once the test script is selected, the next section titled “User paths” will populate based on the thread groups in the selected script. From here you can select the number of users and how they will run for each user path.
The next section allows you to select where you want to run your test. From the “Regional Distribution” section, you can select which regions and load generators your test will use. This will appear differently based on your deployment type.
SaaS Cloud Hosted:
Our SaaS platform currently supports the 4 AWS United States regions (N. Virginia, Ohio, N. California, and Oregon). You can specify the number of load generators you want to use per region. The LGs will be created at test start and will be used only for the current test. These are called dynamic LGs as they are dynamically created and destroyed only when needed for test runs.
On-premises:
The on-premises deployment offers the option for always on and always ready LGs. These are known as static LGs since they are not created and destroyed using cloud infrastructure.
Once you have completed the “Regional Distribution” section, you can click the “Start Test” button to kick off the test. You can also click the “Save as Template” to create a template based on your configuration.
Optional Configurations
In addition to the required configuration settings above, there are some optional settings and integrations that can be leveraged to enhance your testing.
Network Bandwidth Emulation
The network bandwidth feature allows you to understand how users with different network speeds will experience your application. Typically, with lower bandwidths you will see higher response times with responses. You can set the network speed by selecting a predefined value or setting a custom bandwidth.
To set the network bandwidth, first change the selection from No Emulation to either Predefined or Custom.
The predefined speeds simulate common connections like modems, mobile devices, WiFi, and ethernet. This allows you to quickly set a bandwidth without calculating a specific maximum.
If you require a bandwidth that is not present in the Predefined section, then you can select Custom to set a CPS value. CPS is a unit of measure which defines the magnitude of throttling applied to the response in the test. 1 kbps is equal to a CPS value of 128. For example, to simulate a maximum bandwidth of 384 kbps (mobile data EDGE) you would multiply 384 kbps by 128 to get the corresponding CPS value (384 kbps X 128 = 49152 CPS).
Success Criteria
NOTE: This feature, while similar, is currently separate from the Success Criteria configuration found in the VPT Jenkins Plugin. For more information on our plugin’s Success Criteria, please refer to our article on How to Configure the VPT Jenkins Plugin.
Success Criteria allow you to define metrics and metric values that will determine if your test was a success or failure.
You can control the specific thresholds and comparisons for various Key Performance Indicators (KPI). If all criteria are met when the test finishes, the test run will be marked as “Success”. If any one of the criteria fails, the test will be marked as “Failure”. For more information on interpreting Success Criteria results, please refer to our article on How to Interpret Results.
You can select up to five criteria to define your test’s success. Each criterion is defined by three settings:
KPI: A metric upon which the Success Criteria is evaluated for a given test's results.
Comparison: Defines how the threshold value and actual result value will be compared.
Threshold: The value against which the actual result of the selected KPI will be compared.
You will not be allowed to start the test until each criterion has all three settings defined. If you change your mind and wish to remove a criterion, simply click the Remove button on its row.
On mobile screens and when the window width is too small to display all four columns, the rows will display an expansion arrow and a description of the criteria defined. To modify a criterion in this state, you may expand its row by clicking on the arrow to view the criteria settings and remove button.
APM Profile Selection
The application performance management (APM) section allows you to select APM profiles for a test run. APM profiles can only be selected if you have first created a connection to AppDynamics and then created APM profiles for the connection. More information about APM connections and profiles can be found in the APM Integration page.
If you have created APM profiles, then you can click on Select Profiles to bring up a list of all the existing profiles.
Select which profile(s) you would like to use for the test run and click Save to add them to the configuration.
When you run a test with APM Profiles, you will see an additional graph while the test is running. This graph will display all the AppDynamics metrics contained in the select profiles.