When you want to create a great web application and an application that will deliver the best value to your users, you need to be sure that the content is displayed correctly. There are a few possible ways to ensure this can happen. Firstly, test it manually as it’s a time-consuming solution, but at least it doesn’t require specific skills to cover basic cases. If you believe the stereotypes, programmers are lazy people (it fits with me) so I prefer to write end-to-end tests.
Of course, it is worth considering the business aspect of implementing a given solution. Manual tests don’t require any preparation at the beginning, however any new functionality that you will want to test will require a lot of time. Whereas end-to-end tests only require investment right from the very beginning (you need to write a few tests) however, it will pay off in the future with a smaller number of regressions. Also... It seems that it’s much easier to adapt tests to changes that you make in the application rather than editing scenarios that will be covered by the tester each time.
End-to-end (e2e) test are also called acceptance tests, and that is their goal. They ensure that we meet the expectations of the user. Unit tests or integration tests only partly check how a user can get the expected value of the application. In this scenario, end-to-end tests are used. It is easy to deduce from this that tests should be carried out at the user interface level, performing only such steps that are available. Thanks to this approach, we will be sure that our application meets the client's expectations.
Drawbacks of End-to-end tests
Unfortunately, just like every test, acceptance tests also have their downsides:
- the tests will be slower than unit or integration tests because we have to go through all layers of the application.
- Debugging requires more attention because we are able to notice the problem where it occurred and how it appeared, but at first glance, we can’t verify where the error occurred in the code.
Due to the above-mentioned problems, it is recommended to write the minimum necessary number of tests. And what is the minimum necessary number? This is an individual issue of the programmer / team.
In the title of this blog post, I mentioned what we use to test our web applications. However, it would be worth explaining what Canopy is exactly. According to the project documentation “Canopy is a web testing framework with one goal in mind, make UI testing simple”. The framework is based on Selenium which is well-known in the programming environment. However, if this name does not tell you anything ... then do not worry. you don’t need this knowledge to start working with Canopy. One key point of the framework documentation highlights that it’s "Quick to learn", and that’s backed up based on my own experience. The most complicated part of canopy is to adapt to the F # syntax. It doesn’t sound like a serious challenge, right? There also remains the question during the initial research, what about the license and what about the costs? Canopy is made available under the MIT license so you don’t have to worry about it, just use it.
How to install Canopy?
Nowadays almost everything is easy to install. The same thing is true with Canopy. you just need to install it form NuGet.
|PM> Install-Package canopy|
In addition I would recommend installing Argu parser. This will let you use parameters in a way, it is used in the Canopy Starter Kit (which can be downloaded here) and an example project which you can find below.
Elements of Canopy worth mentioning
Before writing the first tests, it is worth reading the framework documentation. It is short, concise and very readable. The information that I give below can of course be obtained from the mentioned documentation, but I will also add my comment and a possible explanation of why a given action, assertion or method is useful.
One of the most important elements of Canopy that I noticed while writing tests for our application is the possibility to use contexts. Context allows you to define the functions once / before / after / lastly and separately for a specific test sequence. Thanks to this solution, you will be able to define, for example, how to prepare an application for a specific batch of tests. One of the examples in which these functions seem to be useful is when you want to compare the state of the application and visible elements for the user when they are logged into the administration panel and when they view the site as a regular user. Remember to show the context clearly and sufficiently descriptive, it will be displayed in the console before the test sequence assigned to this context.
context “context name”
This is a function that runs at the very beginning of the context. The function should be used when the action being performed is not a step in the test but it is required to perform the test correctly. This is a good time to go to the website (! ^ Or url), and maybe login to the administrator panel.
once ( fun _ -> () )
This is a function that runs at the very beginning of each test. The function should be used when you know that previous tests change the state of the application differently to the one you want.
before ( fun _ -> () )
This is a function that runs at the very end of each test. The function should be used when you want to clean up after changes caused by a given test.
after ( fun _ -> () )
This is a function that runs at the very end of the context. The function should be used when the action being performed is not a step in the test but the following context may not require the state of the application obtained in this context. This is a good time to log off from the administration panel.
lastly ( fun _ -> () )
When we create an application, we want it to offer the same feeling to all recipients. One way to ensure this is to test your product on the target environment but when developing a web application is time-consuming. There are many different browsers available on the market, and you still need to take their versions into account. Canopy comes with the help of a mechanism to support multiple browsers. Of course the framework has no built-in browsers, but can use browsers available for your solution. In order to run a given browser, just enter the start instruction.
Selectors are written to match www.setapp.pl site
You can download an example solution from our bitbucket. This solution is based on the Canopy Starter Kit. To run this canopy project, open command line interpreter (f.e. command prompt) inside the solution directory and execute the command below. One test from Under Development Type will fail. It is intended - it shows how previous method works and why we need to be aware of our code.
Check if the element is displayed or not displayed
"Join us button is present" &&& fun _ -> displayed ".button#join-us-cta" "Random button is not present" &&& fun _ -> notDisplayed "#random-button"
Check if the element is enabled
"Join us button is enabled" &&& fun _ -> enabled "#join-us-cta"
"Join us button has correct value" &&& fun _ -> "#join-us-cta" == "Join us"
Verify number of elements
"What we do has 3 boxes" &&& fun _ -> count ".landing-what-we-do .images .landing-what-we-do-box" 3
Check if you can click the button and it will redirect you
"Join us button redirect to Career page" &&& fun _ -> displayed "#join-us-cta" click ("#join-us-cta") on "https://setapp.pl/career"
In the following example, I use Once and Lastly to prepare context for tests with a logged in user, to achieve this goal in the Once function we simulate the user's login process to the administrative panel and in the Lastly logout process function. It is not included in the example test solution.
module yourProject.Authorization open canopy.classic open canopy.runner.classic open Common let login (site: String) = url (site) "#LoginControl_UserName" << "admin login" "#LoginControl_Password" << "admin password" click "#LoginControl_Button1" let logout (site: String, returnToSite: String) = url (site) url (returnToSite) let setupAuthorization(site: String, returnToSite: String) = once (fun _ -> url (site) login(site) ) lastly (fun _ -> logout(site, returnToSite) )
An important issue during debugging is to mark tests as Work in Progress. This can be done in two ways (given below). Then as a developer, you will be able to follow the steps that the test performs and see any errors in your code (if you do not run headless browser tests). The important thing is that if you mark any test as Work In Progress, then only these tests will be run. Although in the debug mode, the tests are performed slower but selected elements are highlighted.
"Join us button is enabled" &&&& fun _ -> enabled "#join-us-cta" wip(fun _ -> enabled "#join-us-cta")
Canopy is a framework that allows you to perform acceptance tests of your web application in an easy, efficient and even pleasant way, in combination with F # 's syntax we also get a code that is easy to analyze by even a novice programmer. If you are looking for alternatives to Selenium, feel free to try Canopy. Although initially I wasn’t convinced by this solution, after a few weeks with this framework I don’t feel resentment when writing tests end-to-end.