Web Performance Management FAQ

 

What is Neustar Web Performance Management?

Neustar Web Performance Management is a web performance service that provides both monitoring and load testing on the same platform. It is based on Selenium, open source platform, so the same script can be used for both monitoring and load testing and across different browsers.

What happened to BrowserMob?

BrowserMob is now part of Web Performance Management. Neustar has taken the best features of BrowserMob and added to it in Web Performance Management.

How can I get started?

You can sign up for a free trial of Web Performance Management.

What does a free trial of web performance give me?

A free trial gives you 60,000 web monitoring units or 30 days, whichever come first.

With 60,000 web monitoring units, you can run the following tests:

  • Up to 6 Real Browser monitors at 20 min interval from 1 location
  • Up to 3 real browser monitors at 20 min intervals from 2 locations

It also includes $10 cloud dollars to run a load test. That will allow you to run a 25 real browser units (RBU) load test for 1 hour.

What is the “three strike” policy?

Neustar’s “three strike“ policy checks three locations before sending you an alert. This helps to avoid “false positives”. If you are receiving an alert, you know that there is a real issue that needs to be addressed immediately.

Can I modify the “three strike” policy?

Yes, the default “three strike” policy can be modified.

What do I need to create my first script?

  1. At the very least, you need a modern web browser and an account on Web Performance Management. You can begin creating scripts and validating them directly in the Web Performance Management script management IDE here: https://script.wpm.neustar.biz.
  2. If you would like to use the Web Performance Management script recorder, you’ll need Firefox and the Selenium IDE plugin. Web Performance Management is currently using Firefox 12 to execute scripts for validation and monitoring, but scripts can be developed with any newer version you might have on your system.
    • Selenium IDE can be downloaded here: http://seleniumhq.org/projects/ide/
    • Firefox is available here: http://www.mozilla.org/
  3. Some very useful (but not necessary) tools for scripting include:
    • Firebug (http://getfirebug.com/)
    • Xpath Finder (https://addons.mozilla.org/en-us/firefox/addon/xpath-finder/)
    • Web Performance Management Local Validation Service
    • An IDE or text editor

In what format do scripts from Selenium IDE need to be saved?

In order to upload scripts that have been created in the Selenium IDE, they need to first be exported, by saving as a “Test Case.” Then you can upload the exported file here: https://script.wpm.neustar.biz/#/scripts/upload. “Test Suites” cannot be uploaded at this time. More in-depth scripting tutorials are on the way.

In what format do scripts from Web Performance Management Integrated Scripting IDE need to be saved?

The Web Performance Management Integrated Scripting IDE will ask you for your api key and secret in order to communicate directly with your Web Performance Management account. You can create, edit, and validate scripts directly in the browser plugin, exporting scripts is not necessary.

What format does Web Performance Management use for scripting?

Web Performance Management uses native JavaScript to drive Selenium and WebDriver scripts. This means a full JavaScript environment is available to both RBU and Basic scripts, giving you full control over execution.

Script validation failed, now what?

Every time you attempt to validate a RBU script, a screenshot is captured (Basic scripts do not generate screenshots) and the validation results are updated. To see the results, simply click the “View Validation Results” link from the script editor page. If there was a problem executing the script, or validation failed, a message will appear at the top of the results dialog along with the line number on which the error occurred. Use this information to debug your script and then retry validation.

You can add additional logging anywhere in your script using “test.log(‘message’)”. These messages will show up at the bottom of the validation results dialog. Tip: log messages are useful for tracking variables and unique values as well as key execution points.

Can Web Performance Management scripts be played back on a local machine?

Yes, the local-validator allows for script playback locally.

Is there a way to view a video of a script during validation?

Not currently. To see how the script runs on our agents, download the local-validator and watch the playback happen in real-time. The local-validation service is also useful for script development and troubleshooting.

What is the process for converting an RBU script to a Basic script?

In order to covert a script from RBU to Basic, the RBU script must first be validated. Once validated, open the script in the Web Performance Management script editor and click “Generate Basic Script.” This will create a script that makes all the same requests as the validated RBU, but replayed as simple HTTP GET/POST requests instead of using a real browser. Converted scripts can be used for both website monitoring and load testing.

How long does the validation process allow a script to execute, what if more time is needed?

The validation process allows a maximum execution time of two minutes. However, there is no time limit on script execution in load tests. If you are attempting to validate a long-running script that is intended for load tests only, there are a couple of options available. The first is to look at the amount of pause time in the script. If the script is using long pause times, the “test.isValdation()” method can be used to reduce the amount of time spent paused during validation. Here is an example:

	if(test.isValidation()){ 
		//Pause for 2 seconds in validation 
		test.pause(2000); } 
	else{ 
		//Pause for 20 seconds during load test 
		test.pause(20000); 
	} 

If, even after using "test.isValidation()" to reduce pause time, you’re stull running past the two minute limit, you will need to use the local-validation service. As with a load test, there is no time limit for script execution when using the local-validator. Once you have verified the script works as expected on your local machine, upload the script and check the “Bypass Validation” option in the script editor. Finally, it’s advisable that you run a small load test to ensure the script behaves as expected.

Are script libraries supported?

At this time they are not. If repeatable code is needed for multiple scripts it will have to be placed into each script.

My Basic script failed validation after converting from a valid RBU script. What happened?

Basic scripts are built from the recorded HTTP traffic generated during validation of the original RBU script. But as is often the case, the HTTP traffic might be different on every run of an RBU script (e.g. ads, ajax, etc.). Basic scripts replay the same requests every time, and expect the same response codes.

The most common error is the converted script expecting 3xx response codes, but seeing 200 instead (http://en.wikipedia.org/wiki/List_of_HTTP_status_codes).This is usually caused by ads, which originally redirected to register a unique impression. When the converted script attempts to replay the same request, it is caught by the ad server as a duplicate, and the response is altered to prevent additional (false) ad impressions. The fix is as simple as removing the faulty requests, or changing the expected response code to what was actually returned. For example:

c.get("http://example.com/ad?req=12345", 301);
Change to:
c.get("http://example.com/ad?req=12345", 200);

The second most common issue is content from third party domains. The blacklist requests directive from the original RBU script will not be maintained in the Basic script. Our recommendation is to remove any third party requests from the script. Here is an example of requests that would be removed:

c.get("https://connect.facebook.net/en_US/all.js", 200);
c.get(https://ssl.google-analytics.com/ga.js", 200);