Synthetic monitoring (also known as active monitoring or even directed monitoring) is a method of testing your applications by simulating users. It's usually defined as the opposite of Real User Monitoring (passive monitoring). Think of the software as a bunch of bots that will run through automated tasks and scripts that mimic user interactions and user behaviour.
Synthetic Monitoring allows you to gain valuable insights into your software before it even goes live, and to ensure that everything remains functioning properly once it has been launched. Real users don't always report problems, but if one occurs, your automated testing bots will make sure that you know about it and get repair it quickly.
In this post, we'll talk more about synthetic monitoring so that you'll have a firm understanding of how adopting a synthetic monitoring solution can help improve the efficiency of your testing and development cycles.
How Does Synthetic Monitoring Work?
Applications today are more complex than ever, not just in the code they run, but in the way that they are put together. Instead of one large application written a single programming language, systems today are comprised of many different bits of software, written in different languages and interconnected through APIs or scripting. This makes older forms of automated testing much less effective. A solution is needed that can handle the many moving parts that today's demanding software requires.
Synthetic monitoring allows you to emulate the way an actual user would perform an action. This gives you a complete view of the user experience from start to finish regardless of what underlying technologies your software is built on.
With synthetic monitoring, you can perform tasks as simple as pinging your website every once in a while to make sure the server is still up, to more complex tasks like performing an entire interaction with your website from start to finish. This allows you to automate the user experience and gain valuable insights into how your software functions from the end user's perspective.
Implementation details differ across offerings. 2 Steps, for instance, allows you to set up complex tests without any coding required, and do so across any platform. That's valuable as most of the Synthetic Monitoring tools available only monitor web applications (website monitoring).
In the past, Synthetic Monitoring and other APM tools used to add only basic metrics, such as load times, response time and page load. Today, by recording the user action that you want to test for in a virtual machine, you can quickly and easily tell the software which tasks you want it to run and it will happily oblige. 2 Steps, again, will even provide a video replay, so when something goes wrong you can see exactly what your virtual customer did leading up to it.
What is Synthetic Monitoring used for?
We've already talked a little bit about the uses of synthetic monitoring in the course of describing what it does. This section will be a review of those as well as a few other use cases that make synthetic monitoring in general, and 2 Steps in particular, compelling additions to your development pipeline.
Reduce labour costs - If there are tests that must be performed regularly then synthetic testing can automate them and free your testers up for other things. With the no code required approach of 2 Steps, the savings are even more substantial, since anyone can set up the tests in a few minutes.
Iterate faster - Synthetic monitoring finds problems faster. It does this not only by allowing you to test much more often than you would be able to if an employee had to spend time doing it, but also by providing fine-grained data that employees might miss.
Generate useful reports - Speaking of fine-grained data, you'll no doubt want to pass some of that on to other members of the team at some point. Whether it is to aid developers in tracking down bugs or to update management on progress, a good synthetic monitoring system, such as 2 Steps, will provide you with the detailed reports that you need.
Stress test new products - If you have a new product, then you do not have the user base necessary to gather real user monitoring data from. Synthetic testing will allow you to emulate the customer experiences and anticipate problems that may occur so that you can address them before you launch.
What's New in Synthetic Monitoring
2 Steps brings some pretty innovative features to the synthetic monitoring arena that we are quite proud of. We've already touched on the fact that the system requires no coding from your staff. You no longer have to expect your testers to have programming skills. Anyone can very easily set up testing almost as quickly as the user can perform the action being tested.
This no-code approach means that there is nothing to install into your existing app and you don't need to write behavioral scripts in a real browser (selenium). The obvious benefit of this is that synthetic monitoring is not completely platform-independent. Any business analyst or quality assurance tester can just as easily test a mobile app running on Android as a website running on Firefox for Linux.
Also unique to 2 Steps is the video replay option mentioned previously. It isn't always enough to know that there is a problem at some specific location at some general point in time. By having an exact video replay of the issue, your developers will be able to replicate the problem just as quickly as they would if one of your human testers had documented it.
And What's Real User Monitoring?
It's pretty obvious from the name that real user monitoring will be tracking data from real users. In order to do that, code is placed into your website or software product that gathers information about a users session. This data includes things like the page load time for each of the pages that they visit, and their pathway through the interface. You can also write code to gather custom data if there is something specific to your needs that you'd like to track.
Synthetic monitoring and real user monitoring each have strengths that will make them an attractive technology to aid in your strategy of performance monitoring and problem avoidance. We'll go over what each of them excels at here.
Advantages of Real User Monitoring
- Accurate real-world data - Because you are tracking real user data, RUM gives you insights and metrics based on how your users actually use your product, rather than just how you assume they will use it.
- More complete data collection - Synthetic monitoring requires you to set up tests in order to collect data. With RUM, as long as the code is on the page, you'll be gathering data.
- Collect specific data - Because the code is a part of your website or app, you can create variables to collect and send whatever data the platform you are targeting allows for.
Advantages Synthetic Monitoring
- Reduces confounding variables - When a performance issue arises in RUM, it can be hard to track down the source. Is it limited to a certain users connection? Maybe the geographic location is the problem. With STM you can more quickly isolate those issues and get a fix out faster.
- Consistent monitoring - Real user monitoring only works when there is traffic. Synthetic monitoring, on the other hand, can be set up to run at all hours of the day so you are not limited to getting data during peak traffic times.
- Complete lifecycle solution - Another benefit of not needing traffic to work is that STM can be a part of your test procedures before a product or its update have even launched.
- Effective benchmarking - Removing confounding variables also makes STM a great tool for benchmarking. You can use it to monitor the performance impact of changes that you make to your software or for monitoring day to day interactions.
It stands to reason that if there are advantages to each of the two solutions, then there are also disadvantages. Here we'll discuss where the two solutions have weaknesses.
Disadvantages of Real User Monitoring
- Requires traffic - Because RUM doesn't work when there isn't any traffic, it isn't very useful for gathering data about off-peak hours or for changes that have not been rolled out yet.
- Not accurate for benchmarking - Because RUM relies on real traffic, the data that comes in isn't consistently timed. There is also a lot of variation in the setups of the users. This noisy data makes it impossible to get accurate benchmarking data.
- Data can be noisy - Because RUM tracks everything, it can make it very hard to organize and isolate the data that you need. Having loads of data can be a great benefit, but quickly becomes a problem when you are looking for specifics.
Disadvantages of Synthetic Monitoring
- Less useful at gathering user insights - Because you are not monitoring actual users, you aren't really getting any data on how your product is used in the real world. This leaves potentially valuable information on the table.
- Can be difficult to set up - Although we've solved this problem with 2 Steps, other synthetic monitoring solutions can require quite a bit of coding to perform even simple tasks. That code may need to be updated when changes to the product occur that can break it.
Which Should You Choose?
The reality is that it isn't a matter of choosing between real user monitoring or synthetic monitoring. Both of the methods have advantages that you should want to have as part of your overall strategy. Once you've seen how the two technologies make up for each other's weaknesses it will quickly become apparent that they are complementary, rather than competitive in nature.
Monitoring is a critical component of your planning, pre-launch, and post-launch application development. You need to see where issues are, what's causing them, and log them for further fixes. Here's the big question — should you monitor your apps with real users, or should you leverage automated processes to capture insights?
If, for some reason, you must choose only one, then consider that synthetic testing has uses outside of just a live product. The ability for your own team to use it during development will help reduce labour costs associated with software testing. This is especially true with the time-saving features of 2 Steps. So synthetic monitoring gets a slight edge if only one option is possible. Ideally, you'll want to have both once your product goes live though so that you have a complete monitoring and diagnostic solution. Let's dive further into this.
Why Do You Need App Monitoring?
Let's back up a bit. Why in the world do you need to monitor your apps in the first place? After all, you probably have some level of pre-launch automation baked into your security department.
Here are 10 reasons to use app monitoring.
- Helps you discover actionable insights into app issues
- Keeps your users happy (app crashes cause 71% of user uninstalls!)
- Boosts conversion rates (70% of users bounce if your app takes too long to load)
- Builds better brands (65% of users will think of your brand negatively if they run into app issues)
- Adds value to your app through better UI mapping (29% of users will uninstall your app if they don't find value)
- Lets you offload some testing to post-launch (47% of apps need more time for testing!)
- Gives you time to find bugs before users (44% of app issues are found by users)
- Provides fuel to your dev department to perform more code deployments (high-performing apps deploy code 200x more frequently)
- Finds the issues you won't ever know about (only 4% of users complain when they find issues!)
- Lowers QA costs (which takes up 25% of average teams budget!)
Why Synthetic Monitoring should come before Real User Monitoring?
Synthetic monitoring is much simpler. You literally plug-and-play the entire testing process. You connect with your synthetic monitoring solution, set up what you want to test, and run it. Then, you receive videos of issues, and you can direct the program where and what to test.
Here's the great thing — you can deploy it pre-launch. So, you don't have to put your buggy app/site out in front of users first — which will save you a few sweat buckets.
Let's skip to the chase — what kind of advantages does synthetic user monitoring offer over real user monitoring?
Here are 4 benefits to synthetic monitoring:
- You don't need users. Want to test pre-launch and check for performance issues? Go ahead! Need to see how this recent code deployment is going to impact load time? Do it! Or, maybe you just want to check the response time at 3 a.m. on a Tuesday. Now is your chance! Synthetic monitoring is always available. And it doesn't rely on external factors to provide tangible value.
- It provides easy tracking. You can waterfall chart every step, get videos of each interaction, and see EXACTLY what's happening during the synthetic monitoring test. With real users, data can get sloppy. And it can be hard to figure out what's going on.
- You don't need any code! Want to test an app that doesn't let you inject any code? Well, that was an easy decision! Looks like your using synthetic monitoring! This is also highly relevant for Ops teams. Does a C-level need to set up a monitoring process? Great! They can!
- It's 24/7. Synthetic monitoring is like the Energizer Bunny on a crack binge. It goes nonstop, and it doesn't need any pesky users to work its magic. You'll still get killer insights, and you'll definitely still receive critical feedback. So, while you're fast asleep, your synthetic monitor will be in the gym benching a hundred pounds of pure UI. 💪🏋
Synthetic Monitoring Use Cases
A lot of people haven't really heard of synthetic modelling and how it affects IT and application development. Yet, it's a technique that's extremely useful for companies to help them preserve up-time, reduce the time IT wastes on minor problems, and test new features easily.
Here are three use cases of synthetic monitoring that can help spot issues and prevent customer service problems later.
Synthetic monitoring can be used as part of the testing process for a new application, UI changes, etc. While it cannot always replace good software testers (who are, after all, actively trying to break the software), you can run a lot of synthetic monitoring processes to simulate how a new site, for example, will work under load. Synthetic monitoring in the test stage can save a lot of time and resources over trying to use human testers for the routine stuff and free up software testers to see if they can bring the application down with the weird edge cases.
Running a load test with synthetic monitoring, however, is one of the greatest uses of the technology. You can also run the same tests on the old and new parts of the site to verify whether you have actually improved performance and if so by how much. Synthetic monitoring ensures that the same actions are taking place in the same order and eliminates mistakes that might be made by a human tester.
You can also use synthetic monitoring to perform load tests on your regular site prior to, say, Black Friday or that date before Christmas when you know everyone puts their orders, helping you prepare for peak traffic. This can help avoid the embarrassingly common incidents where even large companies misjudge traffic and their site goes down. Do you need to temporarily increase your cloud server space or bandwidth? In this day and age, load crashes are not a failure of technology but of planning and management.
Synthetic Monitoring for Spotting Issues Before the Customer
Ideally, you should not be getting any bug reports from your customers. In the real world, of course, it is impossible to catch every single issue before a customer does. Synthetic monitoring, however, can often spot when a server has gone down before anyone else does, and allow you to get it back up and running quickly. You can also win points with customers by showing that you are aware of a site problem and working on it. Customers are more likely to wait for your site to come back up if they know that you are not ignoring them, and one of the best responses you can make to a bug report is "We already know and are working on it."
Early synthetic monitoring was merely automated pings that told you if your site was up or down, but modern systems can test the entire site at intervals, spotting whether a server you are linking to is down, whether your password server is timing out, etc. Newer tools can use Session Replay to capture actual user activity and then replay it, which is useful for troubleshooting a customer's problems. They can also help pinpoint exactly where the problem is, allowing it to be fixed quickly. The faster your issue is diagnosed, the faster it will be fixed. In today's climate, customers have come to expect 100% uptime on websites and e-commerce sites, and may well go elsewhere if your site is down and not fixed very rapidly indeed.
The monitoring can also help spot potential customer service issues before you get complaints, even in some cases before the site goes live and becomes available.
Targeting Resources for Improvement
Synthetic monitoring is awesome for baselining performance and working out where your issues are. For example, you can run scripts at times when nobody is using the site so as to determine the speed of APIs and object loads when there's little or no load on the network. This will tell you which parts of your site are slowest and need improvement.
A properly designed synthetic monitoring script can also do things like determining which parts of the user experience are creating lag. Synthetic monitoring, these days, can emulate a user logging in, doing a search, adding items to their cart and checking out, and can compare different times of day, different geographies, etc. Modern tools can even emulate user journeys that include two-factor authentication, helping companies balance security and convenience and improve two-factor login capabilities. Over time you can get a sense of what your real (not likely) peak times are. This can help you find days that you might not have expected to be busy. Do you have a peak of traffic on January 10-11 every year from people spending gift cards? If so, you might want to try and have extra customer service on board.
If logging in is slow, then you can direct resources to improve your password server. Combined with other forms of monitoring, synthetic monitoring can tell you exactly what parts of your site need the most work.
Synthetic monitoring used to just be a way of alerting you when your website crashed so you could restart it, but has now become much more sophisticated. Modern tools can do almost anything a real end-user can and can simulate typical user behaviour with enough accuracy to be useful. Modern browser simulators are fairly light code-wise and use relatively few resources, allowing them to run without interfering with anything else. Using synthetic monitoring can improve the reliability and performance of your website and get you customer service points with customers.
Should you leverage real people or robots to monitor your apps? Well, there are some pros and cons to both. Really, it will depend upon your business needs. But, if you want to test apps or sites at any point in your dev cycle and you want insights that are easy-to-understand, valuable 24/7, and completely controlled by YOU, you want synthetic monitoring.
Another unique feature of 2 Steps is its integration into the popular Splunk platform. We recently gave a presentation at SplunkLive detailing how we integrated the product into Splunk, why we choose to use Splunk, and how 2 Steps can help your company with its testing needs. Splunk is a fantastic and popular tool and 2 Steps combines several of its powerful features with our legacy IP to drive a new breed of synthetic monitoring.
Request a free demo or download our e-book. If you have any questions about anything you've read here or on our website, please do not hesitate to contact us.