Reasons for Synthetic Monitoring, Real User Monitoring & Measurements


There’s Money in the Metrics

Big Tom Callahan said, “You’re either growin’ or your dyin’ – there ain’t no third direction.”

That quote is from classic cinema, but it accurately describes today’s business arena, one that turns on a dime and gives you nine cents change. Nothing reflects this more than your website performance.To manage it, you need to balance competing demands with technical capabilities, highly empowered customers and fleeting loyalty. As Big Tom might have said, web performance ain’t for the weak of heart. It’s where the issue is decided: business growth or atrophy.

It’s easy to enter a market but hard to generate the growth cash that comes from attracting new customers and keeping the ones you have. That’s why the traditional (technical) understanding of website performance needs to be replaced with a competitive one, a model where performance equates to revenue. These days, the playing fields of business are leveling rapidly. With a practically infinite supply of tools, bandwidth and support, anyone can deliver a premium online experience, whether your business is large or small, well established or spun up out of nowhere.

But if your web experience isn’t rooted in business strategy – and you don’t constantly monitor, fine-tune and optimize your site – business growth will elude you. The metrics that matter most: those that show performance for real website users.

Real User Measurements: The New Revenue Drivers

Most companies use synthetic (sometimes called external) monitoring to track website performance. That is, they synthetically generate traffic from beyond the firewall to see how sites perform for the outside world. Although this provides an important, structured perspective of the website experience, it isn’t based on measurements of real user activity. An overreliance on this approach leads to inaccurate views. What’s needed is a perspective delivered by users themselves.

Real User Measurements (RUM) reveals exactly that. When user activity falls into gaps not covered by synthetic monitoring, RUM sees and reports it in real time. This yields a holistic understanding of performance from the user perspective and enables you to find and fix website problems faster. Because RUM data eliminates blind spots in external monitoring, you’re more confident in knowing what customers are experiencing.

In short, RUM completes the monitoring spectrum. It provides a clearer understanding of web performance, enabling you to take targeted action and remove performance inhibitors. This puts website operators in a better position to commit to customer demands, marketing strategies and business revenue objectives.

real user monitoring & measurements graphic

Figure 1: Real User Measurements completes the performance monitoring perspective.

There are three big reasons why you need RUM in your web performance arsenal: to catch, correlate and commit.

Real User Measurements CATCHES problems synthetic monitoring misses.

Synthetic monitoring shows what could happen on your site. RUM shows what is actually happening – what real users are experiencing. In testing likely scenarios, synthetic monitoring simulates the behavior of site users. Using emulated or real browsers, it monitors in intervals to measure functionality, site availability and page load times. However, while synthetic monitoring gives useful views of overall site performance, it doesn’t measure real traffic.

RUM, on the other hand, does. It sends performance data from the user’s browser to Neustar’s collection servers, allowing you to monitor from the customer’s perspective and catch problems synthetic data misses. When using synthetic monitoring, most companies focus on high-traffic paths and functions like login and checkout. They tend to go on past experience and educated guesses. Because synthetic monitoring must be scripted, it doesn’t measure performance for every twist and turn in a user’s journey. If a customer does something unscripted – if he or she acts human – the behavior and any problems go unrecorded.

real user monitoring data example

Figure 2: Worldwide real user data can be caught for a deeper website performance understanding.

For example, certain customers may be using a browser or client you’re not testing. Maybe they’re somewhere on the site you simply aren’t tracking. Sometimes customer data – what goes in a shopping cart, the length of his/her name or the length of a storage cookie – can cause the site to slow down. Regardless, when customers in Georgia who use Chrome can’t checkout, you won’t know unless you’re looking for it. With RUM, you don’t have to look. The performance data is collected and presented in analysis, live feed, and if preferred, can trigger automated alerts for remediation. Indeed, with RUM and synthetic monitoring you get total data collection. You eliminate blind spots, reduce surprises and find problems faster.

Real User Measurements helps CORRELATE data for a holistic understanding.

With synthetic monitoring, you can test a complex sequence of user behavior with data down to the object level. RUM adds to this with a range of actionable data, showing real performance by region, browser and connection speed. You can also check time to first byte, DNS time, DOM processing time, page rendering time and number of JavaScript errors. These metrics can help you zero in on “first causes” of problems, showing you where to begin looking in your infrastructure. Following is an example of synthetic and RUM data displayed side by side.

real user monitoring data correlation

Figure 3: Correlating external monitoring with real user data is essential.

The monitoring script used is a simple one, so the load times are fairly similar to the RUM results, but the error detected corresponds to a dip in the user experience. Based on this view, you could surmise the incident had a negative impact on site users. It’s a good example of how RUM gives your synthetic data more context and yields a holistic understanding of website performance.

In the days that preceded ubiquitous broadband, gigahertz-powered devices and the mobile consumer, poor performance could be hidden and competitors held at bay. Not today. Customers are watching, competitors are benchmarking and blog posts tell the world how well your site performs. Based on these perceptions, both customers and competitors make financial decisions before you can intercede to do anything about it.

The key to remaining competitive is a greater reliance on RUM. Used in conjunction with a base program of synthetic monitoring, it delivers a set of results that help power growth:

Synthetic Monitoring Real User Monitoring Both
Establish a baseline performance level, often before an application goes live. Measure the experience of users as page hits. See if the user experience consistently meets SLAs.
Quickly isolate problems to a specific component on the page. What is the end user experience like? Acceptable or frustrating? Identify problems faster and accelerate time to resolution.
Troubleshoot with accuracy, thanks to detailed diagnostics such as DNS, connection time, SSL, first byte and content load times and errors for every object on a page. Get an aggregate of end user performance information (displayed as page hits) for page load timings. Confirm if problems exist and eliminate false positives
See what’s broken or degraded on your site. See how your site is handling actual inbound and outbound traffic. Prioritize troubleshooting and make more effective investment decisions that deliver positive, measureable results.

By meshing RUM and synthetic data, you can verify if problems exist, eliminate false positives, speed time to resolution and protect the user experience.

Real User Measurements builds confidence so you can COMMIT to business objectives.

Website monitoring is part of the broader discipline of web performance. As a disciplinary practice, monitoring only works when plied consistently, driven by a commitment to meet customer expectations. Those expectations, of course, are as dynamic as the web itself. They define the experience every time someone lands on your site.

That experience has two parts: whether people can use your site and, if so, how well it worked. The first part involves problem diagnosis and dispute resolution. With RUM, when a customer has a problem you can quickly see what happened. Synthetic monitoring provides a baseline at specified intervals to help you understand the ongoing health of your website. However, problems tend to occur in between intervals and can leave you wondering what users are truly experiencing. When customers report issues, RUM can function as a real-time interrogation tool to help you know what’s going wrong. Even better, RUM data can be easily fed into alert policies which can initiate automated action. You can get a head start into fixing problems AS they occur.

While handling problems quickly is important, using real-time data to get out in front of trouble is even better. Combined, these advantages help you improve time-to-resolution and deliver exceptional user performance. In fact, faster problem management is only one way RUM builds confidence in your site performance.

The second part of the user experience, site performance and functionality, involves problem diagnosis and reporting. Because RUM lets you examine requests across a certain browser, region or URL, you can track these metrics over time to detect trends or abnormalities. You can also use RUM data to analyze performance for capacity planning, plus understand how site behavior affects the bottomline, connecting the dots between slow speeds and bounce rates or conversions. Some companies are even using RUM to complement web analytics. They gain deeper insight into A/B testing, marketing campaigns and more, anchoring any conclusions (and investments) in real results.

Real User Measurements delivers.

RUM eliminates blind spots in your website monitoring and prevents everyday problems from becoming costly issues. Even when generated with real browsers on real networks, synthetic data does not reflect the variables of real life, whether browsers, mobile devices, networks conditions or page view flow. Using both RUM and synthetic data, you can find and fix problems faster, avoid them in the first place and make confident decisions about web performance.

real user monitoring web performance data example

Figure 4.: Deep user performance data helps align improvement investments.

Knowing is always better than guessing. Today’s consumers sap every ounce of your company’s competitive strength. Their impatience may not always be rational or even reflect your site performance, but the ability to understand performance and real-user experience lets you meet their expectations as much as humanly possible. With website performance serving as both differentiator and mandate, if you don’t take advantage of RUM, your competition will.

About Neustar Web Performance

Neustar Web Performance is a cloud-based platform used to find and fix performance problems. It offers the power to monitor and load test sites and diagnose errors and bottlenecks, regardless of site size, geographic location or platform. Combining robust load testing, real user measurements and object-level monitoring, it enables website operators to identify and avoid performance problems before they frustrate customers and hurt the bottom line. Neustar’s platform makes it easier to understand site performance as it happens and take action to remove barriers.

Download White Paper