Sample projects: RWebSpec, Watir, Selenium, RSpec, and Cucumber

By mark.

Quite often, we receive the following questions:

  • Which test framework do you suggest, RWebSpec or Watir or Selenium?
  • Which BDD framework is good for automated test scripts, xUnit or RSpec or Cucumber?

These kind of questions are not easy to answer, as people have different preferences. Now we created a public GitHub repository, you can try out and compare them yourself:
https://github.com/zhimin/adminwise-ui-tests

A test web site is created for this, you just checkout (or download) the test projects (six of them), open one in TestWise 2.

HP TouchPad marketing irony

By mark.

Two days ago, I received a marketing email from HP with subject "Be the first to own the HP TouchPad". Inside, "Works Like Nothing Else", "Everybody on HP TouchPad", "There is nothing like it", you may still visit its web version on HP.

Guess how much how much could change in 2 days? Today, 2011-08-19, news come out that "HP will discontinue its TouchPad as well as its WebOS phones". I feel sorry for people who just bought it.

This is just another example of sales/marketing guys will sell you dying animals. It happens so common in our industry that whole project team suffers from some manager/architect's decision on a product (usually expensive), knowing there are much cheaper or free products out there are a lot better. I am sure these managers/architects won't do grocery like this (as that will be own money), he/she will try and pick up most value for money products.

My point in the context of testing, when choosing a testing tool/framework, try it out for a week, see how much effort for the team to learn and how it cope with application changes? This is common sense really, isn't it?

Are Software Testing same as VCS, worst are most expensive ones?

By mark.

Today I came cross to the Martin Fowler's Version Control Software Survey. Ironically, commercial and expensive VCS such as TFS and ClearCase are behind, correction, far far behind free ones such as Git and Subversion rated by thoughtworks consultants. Given the nature of version control software, it is hard to be critical against free good working ones, as they are widely approved by large software projects, why there are people still paying money for more problematic ones? Sounds crazy, but this happens all the time, remember old expensive J2EE containers, it turns out free Tomcat suits most projects, free and more stable.

This kind lack of common sense: prefer very expensive and useless rather than working and affordable happens to our testing world as well. Do you know projects who buy very expensive testing tools which generate unmaintainable test scripts, eventually left on shelves, I bet you do, as I heard all the time during testing conferences. But there is a hope, I see increasingly more testing job ads on Watir and Selenium.

Zero Bug Tolerance

By mark.

Occasionally we got questions on use of defect tracking system, my answer is "try not using them if the team is co-located". To back up my claim, I quoted from the two classic books below:

From 'Agile Testing' book, "New agile teams usually have a hard time believing it can be done." "The objective is not to get ride of a defect tracking system -- it is to not need to a defect tracking system. There is a big difference. This trick is to expose a defect the moment it occurs."

From 'Extreme Programming 2nd Edition', "At first, American workers can't believe this. Chet Hendrickson told me the story of his brother-in-law who worked at a Toyota plant in Kentucky. He saw a defective door go by. His buddy said, "Pull the cord." Nah, he didn't want to get in trouble. Another defective door. Another. Finally, he pulled the cord. He was praised for telling the truth and pointing out flaws. Unlike mass-production lines where someone "down the line" is responsible for quality, in TPS the goal is to make the quality of the line good enough that there is no need for downstream quality assurance. This implies that everyone is responsible for quality."

Google lost to scripting languages?

By mark.

News: Oracle Seeks xE2x80x98BillionsxE2x80x99 From Google in Java Patent Lawsuit for use of Java in Andriod. When Google acquired Android, why tied themselves to a technology belong to a potential competitor (Sun, then Oracle). With talents in Google, it shouldn't be too difficult to implement the idea in free languages such as Ruby or Python. Surely it will be of some work extending the language, but it won't cost 'billions', let alone on-going license fees. Apple is smarter on this.

To extend the thought further, It seems Google, with all respect, is not winning against companies embracing free and scripting languages, namely:

  • FaceBook, implemented in PHP
  • Twitter, originally created in Ruby on Rails

One less technology to worry about: Microsoft dumps Sliverlight

By mark.

Today I came to this news: Windows SkyDrive Says Sayonara (Goodbye in Japanese) to Sliverlight, Embraces HTML 5. The rumour has been around for while (see this: Microsoft is Starting to Dump Silverlight, Phased Out of Microsoft Services). Now it sounds like poor Sliverlight is being dumped by its parent.

It saves us from answering the enquiry: "Does TestWise support Sliverlight?". I had a look on Sliverlight before, my impression: complex and hard to test. The designers didn't leverage the success factors of Web(HTML): simple and "Power of Text" (a principle of the classic book: Pragmatic Programmer).

Update (2011-06-22): Wired magazine just published an article on future of Adobe Flash, one developer says "itxE2x80x99s dead in the water".

Continuous Integration Steps

By mark.

Months ago, I promised to blog about each of ten steps in Continuous integration, here they are, all in one go. They are in a format of brief introduction and hints, I hope you find it useful!

Click on picture to enlarge

Test Scripts shall be in syntax of scripting languages

By mark.

Occasionally I got questions like

  • Why you didn't build TestWise as Eclipse plugin? Why you built a new IDE from scratch?
  • Do you support test scripts in Java or C#?

First question is easy to answer: testers don't need the complexity of Eclipse. They just need a tool simple, small, view/edit/run test scripts. Also another important human factor, if testers are using the same tool as developers, it puts programmers on a dominant position. From my experience, minor details count.

The same argument (sharing same tool/language with programmers) goes for the second question as well. Test scripts are shall be simple, concise, ideally one line for one operation. It is good to know people (with some programming skills) now move away from expensive record/playback tool for open-source test frameworks, Selenium is a popular choice. However, if the choice is made purely for the benefits of programmers, then I disagree.

Recently at the TIST conference, two people from separate companies told me the similar story: despite of a lot of effort programmers spent providing various ways (such as page-objects, environment selection, data-driven,... etc) to help non-tech testers get engaged, outcome not ideal. That's because the whole approach was programmer-centric.

I think test scripts shall be in scripting languages such as Ruby, Python, not compiled ones such as Java or C#. Why, because it is called 'test scripts' for a reason.

Continuous Integration Officer = CIO

By mark.

In last post, I stated the vital importance of Continuous Integration by quoting Google CEO's speech and Lisa Crispin's blog. Then I found for this such important role, there is no conventional title. Well, I will give it one: Continuous Integration Officer, in short, CIO.

I recently browsed my book collection and found the paragraph below on CI in this classic book: Implementing Lean Software Development: From Concept to Cash:

"We have been told many times that this (Continuous Integration) is a very difficult discipline to put in place, but once it is working, no one would ever think of going back to the old way of doing things." (page 203)

TIST2011 Conference Slides and Videos

By mark.

第四届天津国际软件测试会议演讲用的视频演示 (Quicktime格式)



为什么TestWise被认为是下一代功能测试工具?
敏捷联盟在2007年10月11日和12日对下一代功能测试展望:整合的开发环境帮助我们:重构测试元素,自动完成命令,增量式的语法检验(基于测试领域特定语言),快捷键支持,调试等等。这些正是TestWise的主要功能,在实践中被用户所喜爱(2006年AgileWay已开发成功)