Downtime can be a major drain on your software or web site's success and the more successful you are, the more customers are affected when something goes wrong. That is why it is imperative that you detect issues as soon as possible, so they can be fixed before they snowball into something worse.
Two very useful tools in monitoring the status of a website or software application are synthetic monitoring (STM) and real user monitoring (RUM). In this post, we'll talk about what those are, how they differ, how they are the same, and how they can work together to maximize user satisfaction and minimize problems.
How Do They Work?
In order to understand the benefits and disadvantages of each of the two monitoring types, you'll need to understand how they work. After we take a look at the differences between the two options, we can take a closer look at which situations each excels at.
Real User Monitoring
It's pretty obvious from the name that real user monitoring will be tracking data from real users. In order to do that, code is placed into your website or software product that gathers information about a users session. This data includes things like the page load time for each of the pages that they visit, and their pathway through the interface. You can also write code to gather custom data if there is something specific to your needs that you'd like to track.
Synthetic monitoring replaces users with automated bots that will perform whatever tasks you program them to do at whatever intervals that you tell them to do it. Most synthetic monitoring solutions require some form of scripting in order to know which action to perform. Our product, 2 Steps, can be taught to perform an action simply by having a user perform it while the software records their actions.
What are the Advantages of Synthetic Monitoring and Real User Monitoring?
Synthetic monitoring and real user monitoring each have strengths that will make them an attractive technology to aid in your strategy of performance monitoring and problem avoidance. We'll go over what each of them excels at here.
Real User Monitoring
- Accurate real-world data - Because you are tracking real user data, RUM gives you insights and metrics based on how your users actually use your product, rather than just how you assume they will use it.
- More complete data collection - Synthetic monitoring requires you to set up tests in order to collect data. With RUM, as long as the code is on the page, you'll be gathering data.
- Collect specific data - Because the code is a part of your website or app, you can create variables to collect and send whatever data the platform you are targeting allows for.
- Reduces confounding variables - When a performance issue arises in RUM, it can be hard to track down the source. Is it limited to a certain users connection? Maybe the geographic location is the problem. With STM you can more quickly isolate those issues and get a fix out faster.
- Consistent monitoring - Real user monitoring only works when there is traffic. Synthetic monitoring, on the other hand, can be set up to run at all hours of the day so you are not limited to getting data during peak traffic times.
- Complete lifecycle solution - Another benefit of not needing traffic to work is that STM can be a part of your test procedures before a product or its update have even launched.
- Effective benchmarking - Removing confounding variables also makes STM a great tool for benchmarking. You can use it to monitor the performance impact of changes that you make to your software or for monitoring day to day interactions.
What are the Disadvantages of Each?
It stands to reason that if there are advantages to each of the two solutions, then there are also disadvantages. Here we'll discuss where the two solutions have weaknesses.
Real User Monitoring
- Requires traffic - Because RUM doesn't work when there isn't any traffic, it isn't very useful for gathering data about off-peak hours or for changes that have not been rolled out yet.
- Not accurate for benchmarking - Because RUM relies on real traffic, the data that comes in isn't consistently timed. There is also a lot of variation in the setups of the users. This noisy data makes it impossible to get accurate benchmarking data.
- Data can be noisy - Because RUM tracks everything, it can make it very hard to organize and isolate the data that you need. Having loads of data can be a great benefit, but quickly becomes a problem when you are looking for specifics.
- Less useful at gathering user insights - Because you are not monitoring actual users, you aren't really getting any data on how your product is used in the real world. This leaves potentially valuable information on the table.
- Can be difficult to set up - Although we've solved this problem with 2 Steps, other synthetic monitoring solutions can require quite a bit of coding to perform even simple tasks. That code may need to be updated when changes to the product occur that can break it.
Which Should You Choose?
The reality is that it isn't a matter of choosing between real user monitoring or synthetic monitoring. Both of the methods have advantages that you should want to have as part of your overall strategy. Once you've seen how the two technologies make up for each other's weaknesses it will quickly become apparent that they are complementary, rather than competitive in nature.
If, for some reason, you must choose only one, then consider that synthetic testing has uses outside of just a live product. The ability for your own team to use it during development will help reduce labour costs associated with software testing. This is especially true with the time-saving features of 2 Steps. So synthetic monitoring gets a slight edge if only one option is possible. Ideally, you'll want to have both once your product goes live though so that you have a complete monitoring and diagnostic solution.
If you are looking for a synthetic monitoring solution that has some game-changing features, such as no-code testing, video replay of problems, and Splunk integration, please visit our website and download our free ebook or request a free trial. We will be glad to answer any questions you have about synthetic monitoring or our 2 Steps product. All you need to do is contact us.