Interlude and reflection on Selenium

I am putting my UI testing with Selenium series on hold for the moment. My goal at the start of the series was to be able to create UI tests and as I worked through learning how to, I blogged about what I was learning. My initial goal has been achieved and I can now write UI tests.

Before I continue the series I want to gain more experience UI testing, by writing tests for more advanced scenarios and allowing time for the way I write the tests to evolve. I also want to find out how robust (or brittle) the tests I create are when UI changes are made.

I think the blog series has been good. When I set out to learn Selenium, I personally found the available information confusing and had questions such as “which nuget packages do I actually need?”. I answered these questions and hopefully the series can help others who are looking to start using Selenium.

However, I do think the posts could have been better. It was not a conscious approach, but what I ended up doing was premeditating what I was going to blog about and then spend the week learning about that so that I could write the post. Deciding the goal in advance didn’t work out often and ended up splitting the blogging and learning into almost two separate activities.

I think a better approach would have been to have just blogged as I went about learning Selenium. For example something with a flow like the following bullet points.

  • I want to learn how to UI test with Selenium.
  • I can’t find any good guides for getting started.
  • I am attempting to create a UI test, but am confused about all the different nuget packages available.
  • Learn and describe the differences between them.
  • Create simple project with only the nuget required.

One of the things I struggled with was the structure of the posts and how to frame the information that I was learning. If I had written the posts as I went I think much of the background of why the information is important would have been taken care of. While writing the last post I started to make some changes to the example tests and the UI. Whilst making the changes I realised that I should add a page object to reduce code duplication and simplify any further changes. This led to me talking about page objects at the start of the last post, but I lost the context of why I introduced them, which would have help to demonstrate and explain the reason for the pattern.

I also think the posts require more code samples in the actual post. It would have been easier to add code snippets in if I was describing what I did in a more step by step manner.

My plans for the series are to look at libraries such as Seleno that simplify the UI tests, but as mentioned before I want to gain some more experience so that I can appreciate the benefits they provide. When I am ready to continue I will take the new approach to writing the blog posts.

Advertisements
Interlude and reflection on Selenium

ASP.NET UI Testing With Selenium in 2017: 3. Page Objects and using WebDriver

This is the third post in a series on creating automated UI tests for an ASP.NET app using C# and Selenium. My goal is to highlight and explain the key pieces required starting with Selenium WebDriver in 2017.

Last post I took a bit of a tangent from WebDriver and instead looked at starting the application automatically when tests are run. This time I have added some more realistic UI tests on a mock login page in the web app in the SeleniumExamples repo. To reduce code duplication and make the tests less brittle I have used the page object pattern, and to wait for the login request/response I used the WebDriverWait class.

Up until now I have not required the Selenium.Support nuget, but this week I have added it in as it provides classes for waiting and creating page objects.

Page Objects

Page objects are not specific to Selenium, but are a pattern for decoupling the logic for interacting with a UI element from your tests. This reduces the brittleness of UI tests, making updating your tests for UI changes quicker.

There are plenty of good resources available explaining page objects. Martin Fowler’s bliki is a must read, and the Selenium wiki has a good explanation and example.

In the PageObjects namespace in Selenium.Support there are classes to help simplify page object creation. Unfortunately there does not seem to be a lot information on them (the page in the Selenium wiki is not relevant for .NET). I found the best place to learn and get some examples were the test classes in the Selenium source code, but hopefully the page object class for the login page is a better example.

Waiting

When UI testing with Selenium there will be times when you need to wait for things such as responses to requests or animations to complete. A basic, but inefficient, approach would be to use Task.Delay (or an equivalent) to cause the test to wait. WebDriver provides a better approach for waiting where you set a max timeout and it polls the condition until it is reached, or the time expires.

For .NET there are 2 types of waits available in WebDriver – implicit and explicit. You may see another type, fluent waits, mentioned around the internet, but this is not available in .NET. As far as I can tell fluent waits behave the same as explicit waits, but are declared in a fluent manner.

The Selenium docs provide some details on implicit and explicit waits, but I found this answer on StackOverflow a better source of information. To summarise implicit waits are a one size fits all solution that only applies to finding elements, while explicit waits provide more flexibility and allow waiting for any condition to be met. As the official documentation mentions, implicit and explicit waits should not be mixed as they can cause unreliable wait times.

I used explicit waiting in the login page object to wait for the simulated login request/response to complete. I haven’t tried, but using implicit waits with the page object pattern seems like it would be difficult and I think having the explicit waits in the code to document where there is expected to be a delay is beneficial.

Wrap up

That’s it for this week. From here the plan is to go and create some tests and then look at libraries like Seleno to see how they make it easier.

ASP.NET UI Testing With Selenium in 2017: 3. Page Objects and using WebDriver

ASP.NET UI Testing With Selenium in 2017: 2. Running your application

This is the second post in a series on creating automated UI tests for an ASP.NET app using C# and Selenium. My goal is to highlight and explain the key pieces required starting with Selenium WebDriver in 2017.

In the last post I introduced WebDriver and created a bare minimum example for getting WebDriver running and using it to open the Google homepage in Chrome. The Selenium docs provides examples of the WebDriver API including fetching pages, locating elements and getting text values. These are worth a look to get an understanding of some of the functionality available.

I was planning on providing some WebDriver examples this week, but there is no point in me rehashing the ones provided in the docs. There are a few things worth highlighting, but to do this I will need an example web app and this week I am going to explain how to run the application before executing the tests.

Testing your application with Selenium (and xUnit)

You could start your web application manually before executing tests, but it does become tedious, so it makes sense to setup the tests to execute the application automatically.

A lot of the code I added to start the application comes from Michael Whelan’s post “Black-Box Testing ASP.Net: IIS Express and Selenium WebDriver”, which in turn comes from Seleno (which Michael contributes to). Take a look at Seleno as it will simplify your UI testing. I have not introduced it into my examples, because I wanted to build up an understanding of how to UI test without a particular framework, however in a real UI test solution I would use Seleno.

This week I added a test (Run_Application_And_Navigate_To_Home_Page in _2_Running_A_Web_App) that opens the homepage of the example web app and checks for the element containing “Hi!”. I’ve kept the test simple and haven’t added any asserts or waits, which would there would normally be, but the test will still fail as an exception is thrown if the element is not found.

 

The code for starting the web app is located in the WebAppFixture class. It uses the Process object to start an IIS Express process running the example web application. As mentioned before it is based on code from Michael Whelan and Seleno.

To run this code before the tests, I have setup WebAppFixture as a test collection fixture. This is xUnit specific, but other test frameworks will have similar ways of configuring code to run before tests. This requires an intermediate class (WebAppFixtureTestCollection) to configure the test collection.

Wrap up

That’s all for this post. I had started some more specific examples with WebDriver, but realised I needed an example web app for the tests. That led to automating the startup of it, which became the focus for this week.
Thanks again to Michael Whelan and the other Seleno contributors. Next post I intend to add some more specific examples now that I have an example web application for the tests.

ASP.NET UI Testing With Selenium in 2017: 2. Running your application

ASP.NET UI Testing With Selenium in 2017: 1. Introduction to WebDriver

In this blog series I am going to work through creating automated UI tests for an ASP.NET app using C# and Selenium. I was motivated to write this because I struggled to find relevant and accurate information online, which is I think is just a result of how long Selenium has been around and improved over time. My goal is to highlight and explain the key pieces required starting with Selenium WebDriver in 2017.

What is WebDriver? (WebDriver vs. RemoteControl)

WebDriver is the tool we are going to use to automate interacting with the UI. It is pretty powerful and can be used for any browser automation task, not just UI testing. It relies on each browser’s native support for automation and requires a driver for each browser. The official WebDriver documentation is actually pretty good, even if it seems like it might be out of date (ignore the disclaimer at the top).

RemoteControl (RC) was the original version of Selenium, but was replaced by WebDriver (the wikipedia article provides some history). RC injected custom JavaScript to perform the automation, required running Selenium Server and did not support dynamic pages as well as WebDriver does. That should be the extent of what you need to know about RC and you are safe to ignore any info or guides that refer to it.

Which nugets to use?

This was a point of confusion for me initially that nowhere explained. There are multiple official nuget packages, what do they all do and which ones do I need?

The ones you will require are Selenium.WebDriver and Selenium.Support. WebDriver is the WebDriver API (pretty straightforward) and Support provides better ways of interacting with the API for C#.

You can ignore the Selenium.RC nuget and also Selenium.WebDriverBackedSelenium as it provides WebDriver functionality but with the RC API. It exists to transition code using RC to WebDriver, without it to be rewritten to work with the WebDriver API.

Using WebDriver

To go along with this blog series I have created a SeleniumExamples repo on GitHub which will have code that I refer to. At the time of this post there is a very basic example of using WebDriver with the driver for Chrome to go to the Google homepage in the Open_Google_Chrome method of the _1_Bare_Minimum_Web_Driver_Example class.

To setup the project I added xUnit and Selenium.WebDriver through nuget.

xUnit is my personal preference to run tests and any other test framework, e.g. NUnit, would also work. If you are using xunit, don’t forget the Visual Studio runner.

Selenium.WebDriver gives us access to the WebDriver API. The project is not doing anything advanced at this stage, so I didn’t need Selenium.Support.

Since I decided to use Chrome, I needed to add the Chrome driver. There is a 3rd party nuget package that adds it, but I decided to download it and add it into a Dependencies directory in the project. I wanted to do this a bit out of habit (at One Model we follow this pattern for other required executables) and to make sure I could control where the driver gets added so that I can make them all live in the same folder (when I add others).

I also decided to set the project to copy the driver into the output (bin) directory on compile. I did this in case I want to deploy the test project elsewhere in the future to run tests, and this way all that is required is copying the entire bin directory. This might end up being unnecessary, but did not hurt to do now.

To create an instance of ChromeDriver requires that chromedriver.exe is somewhere in the computer’s PATH, or that you pass the full directory of where it is located as an argument. I prefer that the repo can be cloned and run without any system configuration, so I took the approach of providing the directory as an argument.

From there the code to open the browser and navigate to Chrome is pretty straight forward.

driver.Navigate().GoToUrl("https://www.google.com.au");

Wrap Up

That’s it for this post. In the next post I add an example web application for testing on and have it start automatically before the UI tests run.

ASP.NET UI Testing With Selenium in 2017: 1. Introduction to WebDriver

Adjusting disk size Windows Opsworks instances

At One Model we use AWS Opsworks and Chef to manage our servers. I had to increase the disk size of some Windows instances, but didn’t know how. The following is what I learnt.

I’ll start with the simple case. If you are using the standard version of the OS provided by AWS (i.e. not a custom AMI) all you have to do is start some instances with the desired size.

standard-ami

If, like us at One Model, you are using a custom AMI for the OS it is more complicated. You need to complete a circuit of launching multiple instances from AMIs to end up with a final AMI with the desired disk size. The steps to increase the volume size follow.

Note: I haven’t tried it, but for shrinking I think you will have to do steps 5 and 6 after step 1 (before step 2) to shrink the drive in the OS first before reducing the AMI’s volume size.

  1. Launch an EC2 instance from your custom AMI.
  2. When the instance has launched create an AMI from it, setting the disk size to the desired size. This will not increase the size of the disk in the OS, but we need this intermediate AMI so that we can create an instance with access to a larger disk.
  3. Once the intermediate AMI has created terminate the instance it was created from.

    create-ami

  4. Launch an instance from the intermediate AMI.
  5. When it is ready, connect to the new instance through RDP.
  6. If you open Windows explorer you will notice that the disk size in the OS is unchanged, e.g. if you increased the root volume from 30GiB to 50GiB C drive in the OS will still have a max volume of 30GiB. Use Windows disk management to adjust the volume of the drive.
  7. Follow these instructions (Create a Custom Windows AMI with/without Sysprep) to prepare the instance to run as a custom AMI in Opsworks.
  8. Create a final AMI from the instance.
  9. When the final AMI has finished creating you can deregister the intermediate AMI and terminate the intermediate EC2 instance.

This final AMI can now be used to create new Opsworks instances.

It may be tempting to skip launching the first EC2 instance (step 1) by creating the intermediate image from an instance you already have running in Opsworks, but if you do this the AMI will have any additional changes that have been made (e.g. anything Chef may have done) and will not be as clean as the original custom AMI.

I hope this has helped you. Let me know in the comments if this doesn’t work for you.

Adjusting disk size Windows Opsworks instances

Why you should blog

For the last few years I have been a career mentor for I.T. students at QUT. As part of that I try to inspire the students to start blogging. In this post, I’m going to detail reasons why you should blog. It’s written with students and graduates in mind, but I think everyone should be blogging. I regret that I was slow to start.

It is intimidating to start. You might think that you don’t have anything worth blogging about, or that your posts won’t be good enough for others. Forget about what others might think.

You should blog for your own personal development.

Blogging will help you work through ideas, you will learn more as you research your posts, and your communication will improve.

Blogging is good for your career. It will show your interest, that you enjoy learning. It will also show that you can communicate and share ideas. It will differentiate you from other students and graduates without a blog.

Forget that others might read your blog and start blogging for your own benefit. Blog about what you are learning, the parts you found challenging and what helped you to overcome that. Blog about what you have built and how you have applied what you are learning.

You are just starting out in your career and that gives you a different perspective from experienced bloggers. Writing from your perspective might allow you to create content that helps others starting out in their career. Sooner or later people will start reading your blog and you will have contributed to the community. You might even find looking at your view count addictive!

Hopefully this convinces you to start and I am always happy to help and review.

If you need more convincing I recommend you read Scott Hansleman’s or Steve Yegge’s or Erik Dietrich’s opinions.

Why you should blog

Fixing a CloudFormation rollback loop

Disclaimer: Apologies if some of the details of this are incorrect, I am working off my notes and recollection of what happened at the time. I started to blog about this and had a couple of attempts to confirm, but couldn’t reproduce the issue. If you’ve worked with CloudFormation you will understand how slow the attempts to reproduce were.

Problem

I had an issue which resulted in a CloudFormation stack ending up stuck. It failed to update and then failed to rollback the failed update. Attempting the rollback a second time produced the same result. This is how the events appeared in the AWS console.

failure

I am not sure what the changes we made were that resulted in it getting stuck. I think I was trying to update the AMI and the new AMI was not set up to work correctly in Opsworks.

I couldn’t see a way to resolve this issue in the AWS console so I did some searching and found a way to get out of the rollback look.

Solution

The AWS command line tools often have more functionality than what is available in the web console. I had a look at the commands available for CloudFormation and found continue-update-rollback, which has a “–resources-to-skip” parameter. Using that command with the ID of the stuck instance, ProcessorOpsworksInstance1, got the rollback to complete, however I wasn’t quite done.

success

The following warning is provided with the “–resources-to-skip” parameter:


Warning

Specify this property to skip rolling back resources that AWS CloudFormation can’t successfully roll back. We recommend that you troubleshoot resources before skipping them. AWS CloudFormation sets the status of the specified resources to UPDATE_COMPLETE and continues to roll back the stack. After the rollback is complete, the state of the skipped resources will be inconsistent with the state of the resources in the stack template. Before performing another stack update, you must update the stack or resources to be consistent with each other. If you don’t, subsequent stack updates might fail, and the stack will become unrecoverable.


What you need to do here depends on what changes the failed update was making and what state your stack ended up left in.

For me the AMI failed to update and a new instance was not started. At the end of the rollback CloudFormation would be expecting a ProcessorOpsworksInstance1 in Opsworks running with the original AMI, but I had an instance that would not start. I decided the safest thing to do would be to recreate the instance so that I would be working from a clean slate in the future. To do this I deleted the Opsworks instance by removing it from the template and updating the stack. After that I then added it back into the template and updated the stack again. This got me back to where I started.

Fixing a CloudFormation rollback loop