behat (5)


UI/API Test Framework Github Repos

  • March 30, 2016
  • BDD, behat, ...

CucumberJS/Protractor

Build Status

README

PREP MAC/LINUX

  • Install NodeJS
  • npm install -g phantomjs
  • npm install -g gulp
  • cd /repo root folder
  • node_modules/.bin/webdriver-manager update
  • node_modules/.bin/webdriver-manager start

PREP WINDOWS

  • Install NodeJS
  • Intsall GitSCM
  • Install Visual Studio Express
  • (probably good point for reboot)
  • From gitbash prompt:
    • npm install -g node-gyp
    • npm install -g gulp
  • node_modules/.bin/webdriver-manager update
  • node_modules/.bin/webdriver-manager start

    To run headless phantomjs

    • Download window phantomjs binary
    • Extract somewhere and rename phantomjs.exe to phantomjs
    • Copy to \Users[user id]\AppData\Roaming\npm
    • Change browserName: 'phantomjs' in conf.js file

How do I run tests?

from repo root folder run:

  • npm install (just once)
  • Symlink to api module: ln -s node_modules/apickli/apickli-gherkin.js e2e/step-definitions/api/apickli-gherkin.js
  • Always run export NODE_ENV={environment} prior to running any tests (e.g. export NODE_ENV=development) _By default, if this value of NODEENV is not set before running tests, the tests will run against development url. Currently the two settings available are 'development' and 'localhost'

API

  • Run gulp api to run API tests

UI (Using webdriver)

  • gulp webdriver-update
  • gulp webdriver-standalone
  • Run gulp ui to run UI tests
  • Run gulp api to run headless-browser UI tests

Run UI tests in parallel

  • Run gulp parallel-ui

Browserstack

  • Run gulp bs for regression tests on browserstack service

To view the the report after running the above gulp commands, run the following gulp tasks to generate the html report.

  • gulp clean-json
  • gulp protractor-report

** Or use shell scripts provided

  • Run ./run_api_tests.sh to run api tests
  • Run ./run_ui_tests.sh to run ui tests
  • Run ./run_ui_tests_parallel.sh to run ui tests in parallel mode
  • Run ./run_api_tests_parallel.sh to run api tests in parallel mode

Run selected test with optional tag

  • An optional tag can be passed during when the gulp task is invoked e.g gulp ci --tag @tagNameHere will only run tests tagged with @tagNameHere.
  • If the optional tag is missing the gulp task will run all tests set in the {{gulTaskConfig}}.config.js file as a fallback

Behat/Mink

Build Status

Key components:

Behat 3 Mink Extension PageObjects WebAPIContext

SETUP

From Behat repo root folder run following commands:-

RUNNING SELENIUM BROWSER TESTS

Before running behat to test the feature files in features directory, ensure the following commands are executed :-

  • java -jar selenium-server-standalone-[version].jar

To run tests (open another terminal window):-

  • bin/behat features

Second test runs using Guzzle (for API), the rest using Firefox

RUNNING PHANTOMJS TESTS

  • phantomjs --webdriver=4444
  • bin/behat -p phantomjs features

PERFORMANCE/PARALLEL TESTING

  • apt-get install parallel
  • java -jar selenium-server-standalone-2.43.1.jar --role hub
  • find features -iname '*.feature'| parallel --gnu -j5 --group bin/behat --ansi {}

CROSS BROWSER

Using saucelabs service, you can run tests against most OS/browser combinations and mobile platforms too.

I added an example profile for IE8, as example. To run it, first run sauceconnect config:-

  • bin/sauce_config saucelabs_user_id saucelabs_api_key

Now try running the tests ....

  • bin/behat -p saucelabs_ie8 features/

If you want to use saucelabs service against a localhost url, or any url behind a firewall, then follow these steps (assuming linux):

  1. sudo wget https://saucelabs.com/downloads/sc-4.3.8-linux.tar.gz
  2. sudo tar -xvf sc-4.3.8-linux.tar.gz
  3. cd sc-4.3.8-linux
  4. bin/sc -u saucelabs_user_id -k saucelabs_api_key

REPORTING

As well as a html style report, there is a graphical report-based version using Twig. These are generated in the "reports" folder. Below is example of the Twig report output.

Jasmine/Protractor

Build Status

Demo test application and protractor tests.

Setup

git clone https://github.com/jaffamonkey/protractor-jasmine.git
cd protractor-demo
npm install

To run

Get ChromeDriver set up: Run ./node_modules/.bin/webdriver-manager update.

Start the test application server with node app/expressserver.js Or if you're feeling lazy, just npm start.

Run the tests with node_modules/.bin/protractor test/conf.js Or npm test.

CucumberJS/Protractor(Basic)

Behat with Docker

docker-behat

This repository is the source of docker-behat which brings :

  • a basic shell with oh-my-zsh
  • php5-cli with PHP 5.5
  • behat 3.0 / mink 1.5
  • all needed dependencies

Install

docker pull jaffamonkey/docker-behat

Usage

docker run -t -i -h docker-behat-1 -v $(pwd):/home/behat:rw jaffamonkey/docker-behat /bin/zsh

You should see a prompt containing [ docker-behat ] and have behat command available.

Build

If you need adapt the project to your needs, clone, modify the Dockerfile and from the source directory, run:

docker build -t docker-behat .

behat

I am running simple test from the behat 3 kickstart (as you can see in last line in Dockerfile) - please read following for more guidance on running behat tests:

https://github.com/jaffamonkey/behat-3-kickstart/blob/master/README.md



Behat 3 Skeleton Repo

Behat 3 Kickstart Repo




Behat/Selenium/Parallel

  • October 6, 2014
  • BDD, behat, ...

This exercise demonstrates how to run behat tests (feature files) in parallel, to reduce test run time.  It is important to ensure you have no inter-dependencies between tests before you start.  It also assumes Linux or macOS platform.

First install parallel, a shell tool for executing jobs in parallel using one or more computers:-




Behat And Testing Design Errors/Changes

Comparing screenshots can prove effective way to verify design errors/changes, using CasperJS in conjunction with Resemblejs makes comparisons and reporting easy. PhantomCSS does exactly this, and the repo is available here. Or you can take my forked version here, in which I removed the PhantomJS junk, and altered scripts to just process the screenshots directory. It was only the comparison tools I was looking for .

The trick is now to generate the screenshots (the baseline, and the one for comparison).

As I use Behat/Mink, I didnt need most of the PhantomCSS repo, which assumes use of phantonJS are test driver.

I generate screnshots after certain steps to create baseline for comparison (best to do this in context of a test suite).

/**
* @Then /^I take a screenshot of current page “([^”]*)”$/
*/
public function takeAScreenshotOfCurrentPage($pagename) {
$image_data = $this->getSession()->getDriver()->getScreenshot();
file_put_contents(‘PhantomCSS/screenshots/behat_’.$pagename.’.diff.png’, $image_data);
}

Then I use PhantomCSS to do the screenhot comparison grunt work. There is a manual step, which which you establish the baseline screenshots. Then each subsequent test run, a new “diff” file is generated and comapred with baseline version. If differences, then an almalgamated image is generated to highlight difference (filename appended with “fail”).

 

  • brew install casperjs
  • (or) npm install -g casperjs
  • Clone repo
  • Assuming you follow my behat route ..
  • Run your test once to get baseline images for pages.
  • Rename files by removing the “.diff” part
  • Each time test runs, the newer comparison file will be removed and updated.
  • The command to do comparison is:- casperjs test demo/testsuite.js

 Rudimentary end to end process (todo: add verification step function, and call the casperjs command from within Behat function)




Behat CI

Behat handily generates JUnit reports, handy because Build tools sucj as Jenkins can process these reports to generate ongoing statistics and graphs. As often is with tech, solutions are simple but usually come with path of undefined length to reach that point 🙂 So here is timesaver guide to quick integration of Behat tests.