Watir-WebDriver support in TestWise2

By mark.

Selenium and Watir, as we know, are the two most popular open-source testing frameworks. The next big thing is WebDriver, which is now merged with Selenium 2. Thanks Jari Bakken (who is also the creator of Celerity), we have Ruby bindings for Selenium 2 and even Watir wrapper of it: watir-webdriver, which some might refer it as Watir 2.0. With watir-webdriver, users can pretty much reuse existing Watir scripts, with minor changes.

What are the benefits of using watir-webdriver comparing existing Watir 1.6.5?

  • Run same test scripts in IE, Firefox, Google Chrome and HTMLUnit (headless)
  • No JSSH plugin required for running tests in Firefox
  • Synergy of combination of Selenium and Watir

In last appearance at Watir Podcast, I said future TestWise versions will support WebDriver. Now you can try web-driver in IDE!

  1. Download TestWise-2.0b1-setup.exe and install
  2. In TestWise, open sample project c:\program files\testwise\samples\demo\demo.tpr (if you are using TestWise first, it shall be pre-loaded)
  3. Select the first test script file (left mouse click the file name on the left): ajax_rwebspec_spec.rb,
    choose your browser (IE, Firefox or Chrome), then run it

Our initial impression with Watir-WebDriver is good and very promising, with a few issues currently:
  • Can't reuse existing browser window, has to start up a new one
  • Test execution is slower than Watir 1.6
  • Opening browser operation seems quite CPU intensive

Please note that both Selenium 2 and TestWise2 are at early development stage, so please use Watir 1.6.5 and TestWise 1.9 for production use.

Test sites with native Windows authentication

By mark.

For some sites (especially in .NET world), users are presented a native windows' authentication window (like below) before can access to it.

The standard open_browser in Watir won't work (get stuck), as it will be waiting response from the browser.

  open_browser("http://secure.site.com") # will stuck 

To add a bit more complexity, we expect the same tests to pass whether user has already logged in or not.

The solution is:

  • Detecting Login Window using Ruby's timeout
  • Use AutoIt to fill user name and password in native Windows Login Window.
      Timeout::timeout(5) {
    rescue Timeout::Error : e
      # debug "Timeout error on get to site, maybe asking for login"
      autoit = WIN32OLE.new('AutoItX3.Control')
      win_title = "Connect to edam.bluefirems.com.au"
      ret = autoit.WinWait(win_title, '', 10)
      if ret
        autoit.ControlSetText(win_title, "", "Edit2", username)
        autoit.ControlSetText(win_title, "", "Edit3", password)
        autoit.ControlClick(win_title, "", "Button3")

The above test scripts will work, but not readable. By applying 'Extract Function' and "Move to Helper" Refactorings in TestWise, we get:

    # ...
    rescue Timeout::Error : e
      # debug "Timeout error on get to site, maybe asking for login"
      # calling helper function in test_helper.rb
      login_as("zhimin@secure.site.com", "Secret")    

Database Reset in Automated Testing

By mark.

At TIST conference, after my demonstration, on tester asked me "We saw you running the same test (changing password) twice, before and after refactoring, how could it work second time? The data has been changed from first run.". The answer was database reset, ie. the database is reset to the known state before running each test. As I have been doing this for a number of years now, it almost become second nature to me in terms of automated testing. To make the concept of database reset easy to remember, I call it "The Simpsons Pattern".

Benefits of database reset is so important that you probably will appreciate only after experience it. Once I asked a business analyst to rate his view of importance on database reset in scale of 1 - 10. He answered "9.... No, 10".

  • Develop tests without worrying of data changes caused by tests, what a feeling!
  • Faster test execution. Use seeded data records instead of creating data from scratch every time.
  • Always know data record states, great even for manual inspections of applications.
My typical RWebSpec test scripts like this:
suite "Group of related test cases" do
  include TestHelper

  before(:all) do

  before(:each) do

  after(:each) do
    failsafe{ logout }

  test "One test" do
    # might change application records

  test "Another test case" do
    # doesn't matter the above test is run or not, as we start fresh

Now comes the hard question, how could you implement database reset as testers? No, it has to been done by developers. If your team is doing Agile (or claiming so), go ask them, and it will be of great help for them as well. From experience, the better architecture, easier to implement database reset and the earlier the better.

Watir Podcast on TestWise

By mark.

My Podcast interview with Željko Filipin (the host of Watir Podcast) and Alister Scott (WebMaster of Watir.com) is now published: "Watir Podcast #34 Zhimin Zhan on TestWise":http://watirpodcast.com/34-zhimin-zhan-on-testwise/.

Ant today is 2nd birthday for Watir Podcast, so Happy Birthday! Thanks Željko and other contributors!

StoryWise is released

By mark.

StoryWise, one of Agileway's Agile Testing Suite product, is a flexible web-based project management tool.

StoryWise is free and open source, released under the terms of the GNU General Public License v2 (GPL).

  • Requirement Management
  • Defect Tracking
  • Project Wiki
  • Time Tracking
  • Gantt chart and calendar
  • Requirement Traceability

Apart from being free, simple and easy to use, StoryWise distinguishs itself from others with one unique feature: Requirement Traceability, a term nowdays gradually fades out due to high hype with low benefits but lots of maintainence work.

StoryWise is different, it allows Requirement Traceability to be achieved simply and effortlessly. StoryWise was first demonstrated at 3rd Tianjin International Conference on Software Testing with very positive feedbacks.

Check it out!

3rd TIST Conference Slides and Videos

By mark.

ANZTB 2010 Conference Slides and Videos

By mark.

ANZTB 2010 Conference

By mark.
I will be delivering a presentation "Testers and Programmers Collaboration in Automated UI Testing" at upcoming ANZTB 2010 conference, see you there! !http://www.iceaustralia.com/anztb2010/pix/gui/pix_splash.jpg!

AJAX testing in iTest2

By mark.
Web 2.0 sites such as GMail grow rapidly, which brings big challenges to web test automation. Different from traditional web sites with "synchronous request/response model", new web sites handle requests asynchronously using AJAX. For example, in GMail, clicking 'Send' button does not land you on 'Email Sent' page, instead you see a '_Sending..._' banner appears and replaced with '_Email sent successfully_' later. The difficulty of testing AJAX web sites are not knowing when the request is processed and verify dynamic updated page contents. The approach used iTest2 is simple, after AJAX call, keep trying next operation with specific time. During that time frame, operation error will be ignored (and try again after interval),
    # try up to 10 seconds, waiting receipt date 'showing up'
    try(10) { label_with_id("date").should == today }

    # try up to 60 seconds every 10 seconds.
    try (60, 10) { click_button('Next') }
To assert specific HTML element is shown or hidden
    assert_hidden(:div, "info")  # 
is hidden assert_visible(:div, "info")
To assert specific HTMl element is added or removed in a web page
    assert_exists("label", "email_sent") # 

False positive virus alert on TestWise.exe

By mark.

We use free "Bat to Exe Converter"" to create an exe file (with icon) to launch TestWise. Unfortunately, a few Anti-Virus software reported false positive on it, please be assured that TestWise is virus free. Virus Total, an online virus scanning service, reports :

"5 out of 41" Ant-Virus software detect TestWise positive, they are Comodo, Jiangmin, McAfee+Artemis, McAfee-GW-Edition and TrendMicro.

If you happen to have the annoyance, you can try to start TestWise with batch file included.

If you are interested in why exe file generated by "Bat to Exe Converter" may cause false virus alert, please read this.