Back in the day, a post such as this would begin with a short explanation on the importance of automated software testing. We’d list various reasons for investing in this crucial step, and tell you all about the damages suffered by companies who chose to skip it.
Thankfully, those days are gone automated software testing has rightfully earned its place in the must-have category, and there’s no need for us to go in that direction.
We can all finally agree that agile testing is a critical part of any software development process, unless you don’t really mind risking customers, money, security and your entire business reputation.
Market worth of $19.27 Billion by 2023
Not only does our industry agree on the need for software testing, we can also clearly see that many companies choose to conduct their testing in a similar manner. First, companies use automated testing solutions for obvious reasons that have to do with the volume of work vs. their manpower and time limitations. The global test automation market is expected to reach $19.27 billion by 2023.
Selenium is king
Second, data shows that the vast majority of companies opt for Selenium as their software testing automation tool of choice.
In fact, Selenium reaches a market share of 27.4%, well above any other solution in the field.
Coders seem to find in Selenium a lot of what they need in order to successfully perform automated testing. Beside, Selenium is a free open source environment. That helps.
And still, nobody’s perfect, Not even the most brilliant Selenium coder. More importantly, there are certain inherent flaws in Selenium that are significant enough to not only be annoying, but to also harm the ROI potential of automated testing.
Are there ways to optimize Selenium? Yes, there are, but first let’s briefly go over the main advantages of Selenium (always start with the good 🙂 ) and then move to some of its most significant flaws.
Thumbs up: Why we still love Selenium?
Selenium (mainly Selenium 2, also known as Selenium WebDriver) reached its leading status for many solid reasons that include the followings:
- Open sourced and free: Most test automation tools charge users, while Selenium remains 100% free and open source. This is not only a pricing issue, but also a strong statement to the developers’ community, which always prefers and supports open-source solutions.
- The community: Online support is where size really does matter, and the large user community offers plenty of help, resources and support to quickly solve any problem.
- Wide plugin support: The massive community of Selenium coders also creates plugins that add to the tool’s capabilities and cover some of its weaker spots.
- Easy installation and intuitive usage: Selenium’s UI and onboarding are easy to understand and follow, making it extremely user-friendly and efficient.
- Cross-browser support: Selenium supports Chrome, Safari, IE, Firefox, and more. Once again, this tool is more flexible than an olympic gymnast.
- Remote testing: Selenium Remote Control and Grid enable remote test execution.
- Multiple testing and parallel testing execution: Execution of parallel testing is made possible using Selenium Grid. By testing on several browsers and devices at once, companies save valuable time.
Yes, that’s quite a lot of pros…
Thumbs down: Selenium’s weak spots
OK. This is the hard part. Even Selenium’s biggest fans have to admit that there’s some room for improvement. These are the main issues that coders often struggle with:
- False- positive results: Flaky tests, fail test runs for a variety of reasons that don’t have anything to do with what we set out to test in the first place. For instance, a network connectivity issue can lead to an error, but that’s not because of a bug. When there’s a false positive result, the version release is delayed, the resources are aligned to solving a problem that does not exist, everybody is unhappy and money is lost. Need I say more? Selenium greatly suffers from false positive results. It’s super sensitive to this issue, and coders report it may get to 25% of their runs!! When false positives happen so frequently, teams start ignoring errors altogether, which of course misses the whole point of automated testing. This is a fault that can’t be ignored and must be solved. I am getting to it.
- Long test duration: Selenium can be extremely time-consuming when it comes to authoring the tests and the test run duration. That’s because of the different synchronizations layers should be done to accommodate the asynchronous nature of today’s Web technologies. Also, making sure tests are fully resilient is very time consuming and expertise is highly required
- No root-cause analysis: Selenium’s lack of reports and clear information about the root cause of errors is indeed a problem. This creates a major obstacle for the developers who essentially need to check their entire code in order to locate the error. This again causes a waste of time, which leads to annoyance and money loss.
The best test ever
So, what do automation coders do? Well, they do have a few options they choose from, some better than others:
- Bury their heads in the sand: I really have nothing to say about this one, other than that it is an option which more than one company uses. I don’t how or why they do, and I really can’t understand how industry pros would settle for that, but that’s the reality.
- Switch to alternative, 3rd party automation testing solutions: There are many interesting solutions for test automation, some of them offer a cure to the issues mentioned above. However, this would force Selenium coders to stop using Selenium coding and learn new environments.If you are already coding Selenium and enjoy its advantages, that’s a huge price to pay. Besides, these solutions can’t optimize your existing (legacy) tests, which means that you have lost all your past coding investments.
- DIY Selenium Optimization: Developing your own code editions to optimize Selenium and bypass its inherent limitations is always a possibility. The problem? It takes huge expertise, a lot of time, hard work, and of course, a large monetary investment. While this is a challenge that some companies do meet, in most cases it’s not optimal, to say the least.
- A new approach- Add an optimization layer on top of Selenium: New age solutions such as Shield34 (that’s us 🙂 ) enable Selenium optimization without abandoning it, and bring (many!) additional advantages. Imagine a whole new layer of code, fed by an accompanied AI engine, works hand in hand with Selenium, seamlessly solving its major problems and let coders continue working as before, but this time enjoy a smarter Selenium. Coders are able to maintain the same development environment, automatically optimize their existing (legacy) tests, improving test run speed by over 300% while eliminating false positive (flaky tests) results.
In other words, Selenium coders don’t want to change the way they work, they just want the problems to go away. That’s what we did differently in Shield34, compared to all other solutions.
When we encounter problems with tools we generally love and frequently use, it’s best to try and solve them using the existing foundations. Selenium managed to reach its position for a reason. Not only that, but their community took a long time to form and is a key factor in its success. Instead of abandoning great tools or giving them up altogether, we tried to develop an approach in which we embrace the good while fixing the less-than-great.
Test automation, specifically Selenium test automation, has a huge potential. In fact, you can’t really follow agile paradigms without relying on test automation.
But test automation is also young. And just like any young industry, there’s a huge room for improvement before the promise is fulfilled.
In many cases, and with mature technologies, innovation finds trouble to justify itself, and improvements may seem as ‘nice to have’ rather than mandatory. Just think of the smartphone that you hold in your hands.
But you can surely say that we are not at all there yet with test automation. We must insist on making needed improvements, otherwise, we will not be able to fully fulfill test automation potential