Testing complex sites

By mark.
We are not living in a perfect world, some web sites are created without automated testing in mind. So we have some sites with complex javascripts, elements without IDs or duplicated IDs. . Here is one, the test scripts generated from TestWise/Watir recorder won't work.
  scenario "Site with complex JavaScripts: Recorded" do
    select_option("intFrom", "Brisbane")
    click_link_with_id("toSYD")
    select_option("intDepMonthYear", "Jan 2010")
    select_option("intDepDay", "Sun 03")
    select_option("intRetMonthYear", "Feb 2010")
    select_option("intRetDay", "Wed 03")
    select_option("intAdults", "2")
    click_button("Go")
  end
1. Line 3: _click_link_with_id("toSYD")_ failed with error 'Unable to locate element, using :id, "toSYD"' *Reason:* a javascript window pops up as user enters text in destination text field, however that event is not fired when running the test scripts. *Solution:* Click the image to bring up the javascript popup window manually.
   image(:id, "toBoxPlusSign").click 
# Line 10: _click_button("Go")_ had no effect *Reason:* There are more than one button with caption 'Go' on the page
 
 
 
*Solution:* Click the specific button by using :index
    button(:text : "Go", :index : 2).click # click 2nd 'Go' button
Here is the complete test script:
  scenario "Site with complex JavaScripts:  Working version" do
    select_option("intFrom", "Brisbane")
    image(:id, "toBoxPlusSign").click # bring up the popup    
    click_link_with_id("toSYD")
    select_option("intDepMonthYear", "Jan 2010")
    select_option("intDepDay", "Sun 03")
    select_option("intRetMonthYear", "Feb 2010")
    select_option("intRetDay", "Wed 03")
    select_option("intAdults", "2")
    button(:text : "Go", :index : 2).click # click 2nd 'Go' button
  end

Work with frames

By mark.
TestWise Recorder does not support operations in frames yet. The work around is simple: Ues Watir syntax for operations in frames. Let's walk through with an example. Here is a sample page with an iframe embedded, its html source fragment of frames as below:

Data-Driven Testing in TestWise

By mark.

Data-Driven Testing means tests' input and output are driven from external sources, quite commonly in Excel or CSV files. For example, if you have a list of user logins with different roles, and login process is the same but with different assertions. In this case, you can extract the pure test data out in an Excel spreadsheet, use the same test script to load test data and execute one by one. Because RWebSpec/Watir tests are in fact Ruby scripts, it is quite easy to do so.

For impatient readers, you can see and run the sample by opening the demo project C:\Program Files\TestWise\samples\demo\demo.tpr in iTest, run test script file: database_spec.rb.

The Excel file: C:\Program Files\TestWise\samples\demo\testdata\users.xls contains 3 username-password combination.

DESCRIPTION LOGIN PASSWORD EXPECTED_TEXT
Valid Login agileway agileway Login successful!
User name not exists nonexists smartass Login is not valid
Password not match agileway badpass Password is not valid

Test Script

require 'rwebspec'
require 'spreadsheet'

spec "Use Excel for data driven web tests" do
  include RWebSpec::RSpecHelper

  before(:all) do
    open_browser("http://travel.agileway.net")

    # Load Excel file
    excel_file = File.join(File.dirname(__FILE__), "testdata", "users.xls")
    excel_book = Spreadsheet.open excel_file
    @excel_sheet1 = excel_book.worksheet "Users" # or use 0 for first sheet
  end

  before(:each) do
  end

  after(:all) do
    close_browser unless debugging?
  end

  scenario "Load user list from an Excel spreadsheet to test multiple user logins" do
    # Iterate each row in the spreadsheet, use data for test scripts
    @excel_sheet1.each_with_index do |row, idx|
      next if idx == 0 # ignore first row
      login, password, expected_text = row[1], row[2], row[3]
      goto_page("/")
      enter_text("userName", login)
      enter_text("password", password)
      click_button("Sign In")
      page_text.should include(expected_text)
      failsafe{ click_link("SIGN-OFF") } # if logged in OK, try log out
    end
  end

end

New feature: Script Library

By mark.

‘Script Library’ (a collection of searchable test scripts) provides a convenient way to learn writing RWebSpec or Watir test scripts. To use it, click ‘View’ -> ‘Script Library’ (or clicking the ‘Library’ tool on toolbar in v1.6.5)

The script library will be shown on the right (as below).

You can view all or browse by categories (RWebSpec, Watir and assertions), also can search by text description. Once an entry is selected, the test script is below, clicking ‘Insert’ button (or double clicking it in upcoming v1.6.5) will insert it into current editor (at the current caret).

New feature: Run to this line

By mark.

‘Running test up to specific line’ (or ‘breakpoint’ alike) is one of the most requested feature, it is now available in v1.6.4. Right mouse click on a test statement in editor, then select ‘Run to this line’. TestWise will execute this test case just as ‘running an individual test case’ except the execution will be paused on the line. (If selected line is an assertion statement, TestWise will choose next available operation line, such as click_link, enter_text)

You can resume the test execution by clicking ‘Resume’ button.

TestWise Tips - Test Report

By mark.

After (or during) test execution, click the Excel icon

to get test report in Excel

For formal reporting, it is best to be generated as a part of continuous integration.

TestWise 1.6 released

By mark.

Releases

TestWise 1.6 introduced many new features and enhancements, check it out!

Also there is minor update to TestWise 1.5.6 community edition to support unicode, RWebSpec v1.4.0.2 and TestWise/Watir recorder v1.1.1.

There are also updates to the documentations, particularly ‘Productivity’ section.

TestWise in the news

TestWise Tips - Capture web pages from test execution

By mark.

Quite often, testers/developers want to capture web pages from executing automated test cases. There are a number reasons (or benefits):

  • Manual Inspection / Verification
  • Capture application error text or stack traces
  • Check or looking for certain text (such as payment receipt number, then reconcile manually)
  • Auditing

By using TestWise, you can do it effortlessly if using Page Objects (ie. if you are new to it, please have a look at this paper)

1. Before test execution, open the preferences dialog box, enable the ‘Dump page sources while running the specs’

2. After test execution completes, the camera alike button will be available (bottom-left), Click it.

3. Here you can see text view of each page captured.

4. To get a better view of page, click ‘Open in browser’ button.

PRoR-191 preview: Watir on Ruby 1.9.1

By mark.

The long waiting One-Click Ruby Installer for Windows is coming closer, it's first preview release has been available over 2 months now. Based on it, we packaged a preview release of PRoR with Jari Bakken's fork of Watir for Ruby 1.9.1.

You can download it here, WARNING: this is a preview software.

Here is comparison of running a fairly complex 3-minute Watir test case using two versions of PRoR-191 and PRoR-186.

Ruby 1.9.1 (mingw32)

C:\sandobx>ruby -v
ruby 1.9.1p129 (2009-05-12 revision 23412) [i386-mingw32]

C:\sandobx>spec -fs lodgement_spec.rb
loading win32ole from stdlib on 1.9
Loading pages ...Cost: 1.640552
Lodgement 
 - [Story 123] A complete end-to-end test happy case
Finished in 171.78125 seconds

Ruby 1.8.6 (mswin32)

C:\sandobx>ruby -v
ruby 1.8.6 (2008-03-03 patchlevel 114) [i386-mswin32]

C:\sandobx>spec -fs lodgement_spec.rb
Loading pages ...Cost: 0.656
Lodgement 
 - [Story 123] A complete end-to-end test happy case
Finished in 174.983 seconds

Here are some observations:

  • No noticeable performance differences, as expected, as test execution time is most depend on web application server response time and browser rendering time.
  • Page class loading time (Ruby's require/load classes) using Ruby 1.9 is longer than in Ruby 1.8.6
  • close_others (Watir) does not work