app development

Why not follow me on twitter? @xinsight

Calabash Tips

05 Sep 2015

For a recent iOS project, I’ve been using calabash for running cucumber acceptance tests. It’s been working fairly well so far, but there were some stumbling blocks, so I thought I’d share a few tips that I’ve learned.

A quick overview of how calabash works: It embeds a webserver in your app. A feature file with scenarios and steps like “Given I enter a valid username” is parsed by cucumber. The steps are then converted to queries that are sent to the app. The queries inspect the view hierarchy, and can also perform gestures (e.g. swipes) and interact with views (e.g. selecting a button).

It’s a bit of a crazy system. The feature files are written in ‘cucumber’. The code to handle the features is ruby, and of course, the app itself is written in swift or objective-c. But it does work.

Wait is better than Sleep

Most feature files start off with something like this:

Background:
  Given I launch the app

Since calabash launches your app automatically at the start of each scenario, there’s nothing to be done in this step. However, it would occasionally fail if the app took too long to load. I initially wrote the step handler like this:

Given(/^I launch the app$/) do
  # give the app some time to be responsive
  sleep(3)
end

That mostly worked - but not always. And when the tests were running on a slower machine, the sleep duration wasn’t long enough.

Since the first screen is always a login screen, an explicit wait test is much more reliable:

Given(/^I launch the app$/) do
  wait_for(:timeout => 20) { element_exists("UITextField") }
end

Case Insensitive Queries

The designers were going back and forth on whether buttons should be titlecase or uppercase. I decided the easiest approach was to make the button tests case insensitive with the [c] modifier.

Given(/^I select the \"\" button$/) do |name|
  selector = "button label {text ==[c] '#{name}'} parent button"
  touch(selector)
end

Now a test will pass if the feature file has “Let’s go!” and the UI is “LET’S GO!”. Of course, the test would still pass if the UI is “lEt’s go!”, but that seems like an acceptable trade-off.

Partial Text

Sometimes you don’t quite know what you’re looking for. Here is a step for finding part of a string:

Then(/^I should see the partial text "(.*?)"/) do |partialText|
  selector = "label {text CONTAINS '#{partialText}'}"
  if !element_exists(selector)
    screenshot_and_raise "Partial text [#{partialText}] not found."
  end
end

Query can do more than just query

Something that wasn’t obvious at first, is that you aren’t limited to querying the properties that are returned when you run query("view") in the calabash-ios console.

You can call any arbitrary method on a view – even ones with parameters.

# no parameters
value = query("UISlider", :value).first

# one parameter
query("UISlider", {:setValue => 1})

# multiple parameters
query("UISlider", [{:setValue => 1}, {:animated => true}])

This allows you to create useful “backdoors” in your code to test things that wouldn’t be easy to test. For example, if you needed to create an object 7 days ago.

In the slider example, when you set a UIControl value manually, Apple’s convention is that no UIControlEvent is created. So calabash can set the value of a slider, but there will not be a UIControlEventValueChanged posted which might be needed by the UI if a button only becomes enabled once the slider has been set.

However, you can manually trigger a UIControlEvent:

UIControlEventValueChanged = 1 << 12
query("UISlider", {:sendActionsForControlEvents, UIControlEventValueChanged})

This is a bit of a hack – synthesizing a touch event to drag the slider would be a more accurate test to how the end-user would experience the app – but that’s not always practical.

Method Chaining

I didn’t see this documented anywhere, but Calabash seems to support basic method chaining. I had a custom view with an array of strings, and I wanted to extract the first string. Amazingly, this worked:

firstItem = query("ContainerView", :items, [{:objectAtIndex => 0}).first

Cross platform

There is also an Android app being developed in parallel with the iOS app, and since calabash is also available for Android, we can use the same feature files for both platforms. The actual code behind each step is platform specific - but it is still possible to share steps between the two platforms. As long as a step only uses other steps, it is platform agnostic.

For example:

Given(/^I login to the app$/) do
  step "I enter \"test\" in the \"username\" textfield"
  step "I enter \"pass\" in the \"password\" textfield"
  step "I select the \"Login\" button"
end

This was a huge benefit for testing the signup process, since the two platforms could share the test user data.

So, those are a few tips. I’ll add more as I find them. Happy testing!

Permanent link to this post: http://xinsight.ca/blog/calabash/

Older: Launch Images and Splash Screens

View Comments

Related Posts

Recent Posts