BDD (60)


What did they mean?

  • October 2, 2016
  • Agile, BDD, ...

iknowwhatyoumean

Let’s take the user story from previous post, and start with the granularity (i.e. the scenario steps). While specifying steps is an even better guide for developers to follow in their implementation, it’s important to remember anything we say can be misinterpreted!

 

Feature: As a online shopper,
I want to be able to add multiple items to my shopping cart,
in order to progress through payment process quickly

Scenario: Add single item to shopping cart
Scenario: Add multiple items to shopping cart
Scenario: Add multiple of same item to shopping cart
Scenario: Remove items from shopping cart




What do you mean?

  • October 2, 2016
  • Agile, BDD, ...

After working on several BDD (Behaviour Driven Development) projects, I was struck firstly by what a great development approach it is, if only from a test engineering viewpoint. While on the tech side, BDD is more clearly understood, clients may find the style of writing requirement commonly expected with this approach, more involved than they are used to.

There is still value in defining user stories only, if you are happy with trusting team to take care of implementation, and ask relevant questions. But perhaps you want to be more involved … So here is a guide aimed largely at Product Owners, on how to write requirements for not only BDD projects, but as alternative style to more traditional Agile user stories.




CucumberJS/Protractor: Testing on multiple environments

There are many ways to run your cucumberjs/protractor tests on different environment.  I wanted to avoid adding another package (or two), so settled on the following direction – using the NODE_ENV system variable.  Below is the code for my main file for keeping page url definitions, as well as the base urls.  I have added two as examples, but you can add as many as you like.




UI/API Test Framework Github Repos

  • March 30, 2016
  • BDD, behat, ...

CucumberJS/Protractor

Build Status

README

PREP MAC/LINUX

  • Install NodeJS
  • npm install -g phantomjs
  • npm install -g gulp
  • cd /repo root folder
  • node_modules/.bin/webdriver-manager update
  • node_modules/.bin/webdriver-manager start

PREP WINDOWS

  • Install NodeJS
  • Intsall GitSCM
  • Install Visual Studio Express
  • (probably good point for reboot)
  • From gitbash prompt:
    • npm install -g node-gyp
    • npm install -g gulp
  • node_modules/.bin/webdriver-manager update
  • node_modules/.bin/webdriver-manager start

    To run headless phantomjs

    • Download window phantomjs binary
    • Extract somewhere and rename phantomjs.exe to phantomjs
    • Copy to \Users[user id]\AppData\Roaming\npm
    • Change browserName: 'phantomjs' in conf.js file

How do I run tests?

from repo root folder run:

  • npm install (just once)
  • Symlink to api module: ln -s node_modules/apickli/apickli-gherkin.js e2e/step-definitions/api/apickli-gherkin.js
  • Always run export NODE_ENV={environment} prior to running any tests (e.g. export NODE_ENV=development) _By default, if this value of NODEENV is not set before running tests, the tests will run against development url. Currently the two settings available are 'development' and 'localhost'

API

  • Run gulp api to run API tests

UI (Using webdriver)

  • gulp webdriver-update
  • gulp webdriver-standalone
  • Run gulp ui to run UI tests
  • Run gulp api to run headless-browser UI tests

Run UI tests in parallel

  • Run gulp parallel-ui

Browserstack

  • Run gulp bs for regression tests on browserstack service

To view the the report after running the above gulp commands, run the following gulp tasks to generate the html report.

  • gulp clean-json
  • gulp protractor-report

** Or use shell scripts provided

  • Run ./run_api_tests.sh to run api tests
  • Run ./run_ui_tests.sh to run ui tests
  • Run ./run_ui_tests_parallel.sh to run ui tests in parallel mode
  • Run ./run_api_tests_parallel.sh to run api tests in parallel mode

Run selected test with optional tag

  • An optional tag can be passed during when the gulp task is invoked e.g gulp ci --tag @tagNameHere will only run tests tagged with @tagNameHere.
  • If the optional tag is missing the gulp task will run all tests set in the {{gulTaskConfig}}.config.js file as a fallback

Behat/Mink

Build Status

Key components:

Behat 3 Mink Extension PageObjects WebAPIContext

SETUP

From Behat repo root folder run following commands:-

RUNNING SELENIUM BROWSER TESTS

Before running behat to test the feature files in features directory, ensure the following commands are executed :-

  • java -jar selenium-server-standalone-[version].jar

To run tests (open another terminal window):-

  • bin/behat features

Second test runs using Guzzle (for API), the rest using Firefox

RUNNING PHANTOMJS TESTS

  • phantomjs --webdriver=4444
  • bin/behat -p phantomjs features

PERFORMANCE/PARALLEL TESTING

  • apt-get install parallel
  • java -jar selenium-server-standalone-2.43.1.jar --role hub
  • find features -iname '*.feature'| parallel --gnu -j5 --group bin/behat --ansi {}

CROSS BROWSER

Using saucelabs service, you can run tests against most OS/browser combinations and mobile platforms too.

I added an example profile for IE8, as example. To run it, first run sauceconnect config:-

  • bin/sauce_config saucelabs_user_id saucelabs_api_key

Now try running the tests ....

  • bin/behat -p saucelabs_ie8 features/

If you want to use saucelabs service against a localhost url, or any url behind a firewall, then follow these steps (assuming linux):

  1. sudo wget https://saucelabs.com/downloads/sc-4.3.8-linux.tar.gz
  2. sudo tar -xvf sc-4.3.8-linux.tar.gz
  3. cd sc-4.3.8-linux
  4. bin/sc -u saucelabs_user_id -k saucelabs_api_key

REPORTING

As well as a html style report, there is a graphical report-based version using Twig. These are generated in the "reports" folder. Below is example of the Twig report output.

Jasmine/Protractor

Build Status

Demo test application and protractor tests.

Setup

git clone https://github.com/jaffamonkey/protractor-jasmine.git
cd protractor-demo
npm install

To run

Get ChromeDriver set up: Run ./node_modules/.bin/webdriver-manager update.

Start the test application server with node app/expressserver.js Or if you're feeling lazy, just npm start.

Run the tests with node_modules/.bin/protractor test/conf.js Or npm test.

CucumberJS/Protractor(Basic)

Behat with Docker

docker-behat

This repository is the source of docker-behat which brings :

  • a basic shell with oh-my-zsh
  • php5-cli with PHP 5.5
  • behat 3.0 / mink 1.5
  • all needed dependencies

Install

docker pull jaffamonkey/docker-behat

Usage

docker run -t -i -h docker-behat-1 -v $(pwd):/home/behat:rw jaffamonkey/docker-behat /bin/zsh

You should see a prompt containing [ docker-behat ] and have behat command available.

Build

If you need adapt the project to your needs, clone, modify the Dockerfile and from the source directory, run:

docker build -t docker-behat .

behat

I am running simple test from the behat 3 kickstart (as you can see in last line in Dockerfile) - please read following for more guidance on running behat tests:

https://github.com/jaffamonkey/behat-3-kickstart/blob/master/README.md



How I stopped hating QA

  • September 25, 2015
  • Agile, BDD, ...

Guerilla test management – it became my accidental niche area.  From 2000-2011, I was working in primarily creative, publishing, mobile and media companies in QA/Test Management capacity – usually hands-on (asides pure strategy/management based contracts).  Either welcome or not, I was hired for my pragmatism and multi-disciplined QA skills.  I have built test environments on the sly (not kidding), experienced sabotage, back-stabbing, blame-game defences.  I have managed distributed teams, and have been a lone QA  on multiple projects.  I often had to learn new things, at speed.  What never changes (rarely) is my good-natured optimism, and refusal to allow others negativity to affect me.  But, a few years ago, I had grown tired of my management direction and tired of unhealthy chaotic projects I was hired to improve QA within.  On a personal level, the work felt vague with no focus, and increasing dissatisfaction.