continuous integration (5)


OWASP Dependency-Check Plugin on Jenkins CI

The OWASP Dependency-Check Plugin will locate npm, maven, php, jar packages and analysze them for known security vulnerabilities (full support list is on the website). To use, you need to create a build step on the app build job you have, after all dependencies installed, then publish the report in a post-build step.




OWASP ZAP on Jenkins CI

OWASP ZAP is a very established and useful test tool, and there is a Jenkins plugin ZAProxy to enable you to easily include it in CI.  You can add it as a step to an exiting job, and create a job specifically to run ZAP.  Instead of using a lot of screenshots, I have done it as a step-by-step text-only guide.




Behat/Selenium/Parallel

  • October 6, 2014
  • BDD, behat, ...

This exercise demonstrates how to run behat tests (feature files) in parallel, to reduce test run time.  It is important to ensure you have no inter-dependencies between tests before you start.  It also assumes Linux or macOS platform.

First install parallel, a shell tool for executing jobs in parallel using one or more computers:-




Behat CI

Behat handily generates JUnit reports, handy because Build tools sucj as Jenkins can process these reports to generate ongoing statistics and graphs. As often is with tech, solutions are simple but usually come with path of undefined length to reach that point 🙂 So here is timesaver guide to quick integration of Behat tests.




BDD & Continuous Integration

BDD demands a lot of attention, and is definitely not for the faint-hearted.  Be prepared to have all your flaws exposed, and sometimes in ruthless succession. It is why I think more and more than Kanban style is the way forward in management of development, for two key reasons:-

1. Acceptance of concept of human error, and plans to mitigate that predictability.

2. Acceptance that plans made today may have to change tomorrow.

Continuous Integration (CI) is the part that is supposed to mitigate regression issues – that does not mean regression issues are not possible.It can be a single point of truth for the state of the application, if managed honestly.

A strange way to put it? Well …

One of the more ridiculous mantras of over-excited Agile developers, was that regression testing was dead, as the new development approach avoided the need for it.  Largely this assumption was made around increased use Continuous Integration, but comically, the mantra was still issued by those who didn’t observe CI principles.  CI practice can mitigate may regression issues, but it has to be GOOD practice.

A frequently standard CI setup involves many unit tests, testing code in isolation too often, instead of integration tests.  This presents a misleading picture of progress, as the actual benchmark of progress is at business logic level, not code level. Introduce your BDD tests into CI, and you can expect failures every single time, there is no such thing as 100% from an acceptance test point of view.  If it is, then the tests are not extensive enough or unit tests are too low level for any useful test results.

Management love hearing such positive reports “95% of our CI tests are passing”, without ever questioning what that actually means in terms of delivery. It could mean absolutely nothing, beyond verifying framework modules and/or code (that may or may not be relevant to requirements).

Lastly, there is no reason not to have more than one CI process running – one for unit tests, one for BDD tests.

BDD and Continuous Integration