test automation (78)


Be your own SaaS

You take a few products from open source, integrate, customise. It’s still called development, and it still requires skill – that is is you want your test framework to be stable and grow. It needs skill, as you need to minimise product-dependencies ongoing, or you can quickly get lost in spiral of dependency failures, as you update core framework.

You can chase rainbows for the ultimate test automation tool – there are those using open source, with same dire “all your problems are over” style taglines for their SaaS.

While you are chasing your rainbow, I will have built a framework and maintained it for resusability and open to all to use – tech and non-tech people alike. By trusting a third-party vendor or SaaS, you constrict your team immediately to work a certain way. It limits creativity and relevance.




Testing REST API with Cukes




Elementor: An improved element finder for protractor

elementor – An improved element finder for protractor

Source: andresdominguez/elementor: An improved element finder for protractor




UI/API Test Framework Github Repos

  • March 30, 2016
  • BDD, behat, ...

CucumberJS/Protractor

Build Status

README

PREP MAC/LINUX

  • Install NodeJS
  • npm install -g phantomjs
  • npm install -g gulp
  • cd /repo root folder
  • node_modules/.bin/webdriver-manager update
  • node_modules/.bin/webdriver-manager start

PREP WINDOWS

  • Install NodeJS
  • Intsall GitSCM
  • Install Visual Studio Express
  • (probably good point for reboot)
  • From gitbash prompt:
    • npm install -g node-gyp
    • npm install -g gulp
  • node_modules/.bin/webdriver-manager update
  • node_modules/.bin/webdriver-manager start

    To run headless phantomjs

    • Download window phantomjs binary
    • Extract somewhere and rename phantomjs.exe to phantomjs
    • Copy to \Users[user id]\AppData\Roaming\npm
    • Change browserName: 'phantomjs' in conf.js file

How do I run tests?

from repo root folder run:

  • npm install (just once)
  • Symlink to api module: ln -s node_modules/apickli/apickli-gherkin.js e2e/step-definitions/api/apickli-gherkin.js
  • Always run export NODE_ENV={environment} prior to running any tests (e.g. export NODE_ENV=development) _By default, if this value of NODEENV is not set before running tests, the tests will run against development url. Currently the two settings available are 'development' and 'localhost'

API

  • Run gulp api to run API tests

UI (Using webdriver)

  • gulp webdriver-update
  • gulp webdriver-standalone
  • Run gulp ui to run UI tests
  • Run gulp api to run headless-browser UI tests

Run UI tests in parallel

  • Run gulp parallel-ui

Browserstack

  • Run gulp bs for regression tests on browserstack service

To view the the report after running the above gulp commands, run the following gulp tasks to generate the html report.

  • gulp clean-json
  • gulp protractor-report

** Or use shell scripts provided

  • Run ./run_api_tests.sh to run api tests
  • Run ./run_ui_tests.sh to run ui tests
  • Run ./run_ui_tests_parallel.sh to run ui tests in parallel mode
  • Run ./run_api_tests_parallel.sh to run api tests in parallel mode

Run selected test with optional tag

  • An optional tag can be passed during when the gulp task is invoked e.g gulp ci --tag @tagNameHere will only run tests tagged with @tagNameHere.
  • If the optional tag is missing the gulp task will run all tests set in the {{gulTaskConfig}}.config.js file as a fallback

Behat/Mink

Build Status

Key components:

Behat 3 Mink Extension PageObjects WebAPIContext

SETUP

From Behat repo root folder run following commands:-

RUNNING SELENIUM BROWSER TESTS

Before running behat to test the feature files in features directory, ensure the following commands are executed :-

  • java -jar selenium-server-standalone-[version].jar

To run tests (open another terminal window):-

  • bin/behat features

Second test runs using Guzzle (for API), the rest using Firefox

RUNNING PHANTOMJS TESTS

  • phantomjs --webdriver=4444
  • bin/behat -p phantomjs features

PERFORMANCE/PARALLEL TESTING

  • apt-get install parallel
  • java -jar selenium-server-standalone-2.43.1.jar --role hub
  • find features -iname '*.feature'| parallel --gnu -j5 --group bin/behat --ansi {}

CROSS BROWSER

Using saucelabs service, you can run tests against most OS/browser combinations and mobile platforms too.

I added an example profile for IE8, as example. To run it, first run sauceconnect config:-

  • bin/sauce_config saucelabs_user_id saucelabs_api_key

Now try running the tests ....

  • bin/behat -p saucelabs_ie8 features/

If you want to use saucelabs service against a localhost url, or any url behind a firewall, then follow these steps (assuming linux):

  1. sudo wget https://saucelabs.com/downloads/sc-4.3.8-linux.tar.gz
  2. sudo tar -xvf sc-4.3.8-linux.tar.gz
  3. cd sc-4.3.8-linux
  4. bin/sc -u saucelabs_user_id -k saucelabs_api_key

REPORTING

As well as a html style report, there is a graphical report-based version using Twig. These are generated in the "reports" folder. Below is example of the Twig report output.

Jasmine/Protractor

Build Status

Demo test application and protractor tests.

Setup

git clone https://github.com/jaffamonkey/protractor-jasmine.git
cd protractor-demo
npm install

To run

Get ChromeDriver set up: Run ./node_modules/.bin/webdriver-manager update.

Start the test application server with node app/expressserver.js Or if you're feeling lazy, just npm start.

Run the tests with node_modules/.bin/protractor test/conf.js Or npm test.

CucumberJS/Protractor(Basic)

Behat with Docker

docker-behat

This repository is the source of docker-behat which brings :

  • a basic shell with oh-my-zsh
  • php5-cli with PHP 5.5
  • behat 3.0 / mink 1.5
  • all needed dependencies

Install

docker pull jaffamonkey/docker-behat

Usage

docker run -t -i -h docker-behat-1 -v $(pwd):/home/behat:rw jaffamonkey/docker-behat /bin/zsh

You should see a prompt containing [ docker-behat ] and have behat command available.

Build

If you need adapt the project to your needs, clone, modify the Dockerfile and from the source directory, run:

docker build -t docker-behat .

behat

I am running simple test from the behat 3 kickstart (as you can see in last line in Dockerfile) - please read following for more guidance on running behat tests:

https://github.com/jaffamonkey/behat-3-kickstart/blob/master/README.md



Waiting for elements to appear/disappear with cucumberjs/protractor

The cucumberjs/protractor combination can be awkward, testing angularjs, and using chai-as-promised helped, but reliability still can become an issue. So I started to lean towards more purist protractor code in these circumstances, to get round issue that waits are not always intelligent in cucumberjs and chai-as-promised didnt always provide solution (but is still great extension!). This following code has proved a reliable wait function.


this.Given(/^I wait for something to finish$/, function () {
function waitForElementToNotBePresent(elementToWaitFor) {
		var EC = protractor.ExpectedConditions;
		var el = element(by.id(elementToWaitFor));
		return browser.wait(EC.visibilityOf(el));
// Or to check if element is not there, return browser.wait(EC.invisibilityOf(el));
	}
});

What is happening here is that protractor is waiting for the element to be be present or not, on page, before completing current step action.