The future of functional testing in Drupal 8 - Drupalcon Amsterdam

The presentation

Talk and BOF outcomes

Common library with drupalextension

The maintainer of the drupalextension library, Jonathan Hedstrom (@jhedstrom) came to both the session and the BOF. Jonathan discussed the viability of having a common library of steps that we could use in both core and contrib instead of reinventing the wheel twice. We will be looking into this once we have our existing work in core.

Use Mink asserts instead of writing our own

Everzet asked if we had looked at using the Mink library of asserts and use those instead of our own custom wrappers with PHPUnit. We have investigated this since and see itas a good way forward and much less code to maintain. There is a lot of value for Mink on this front as well, if there are bugs we find in Mink, we contribute upstream!

Explore if we can run this with run-tests.sh as a wrapper

One of our blockers right now is the testbot. Since the current testing infrastucture is so tightly bound to run-tests.sh we might not be able to run this as of yet. Not all hope is not lost thought, Simpletest already runs PHPUnit tests, so we might be able to leverage the same approach for the time being, thus unblocking this issue.

Write a test suite to test itself

We are currently in the process of writing a test suite to "test itself". This testsuites goal is to perform basic operations that would be performed in a real test. We could then looking at running this testsuite before all others and fail fast, if the testsuite itself fails why run any other functional tests?

Javascript

Another contributor to the BOF was Moshe Weitzman (@moshe-weitzman). He feels that we should have a test that uses javascript as a part of the initial patch. For us to have this we need to implement drivers so tests can declare if they require a javascript browser. It was decided that we should look at Behat and see if we could leverage a component in there to handle the switching of the drivers from Goutte (Guzzle client) to Javascript.

ConsoleTable for CLI

This item is all about making the CLI output look pretty, want to know which line the test failed at? This is what we are looking at doing here. We plan to leverage ConsoleTable so that way we don't have to worry about writing our own table output library (less work!).

Conclusion

Myself and Cameron want to take this opportunity to thank the community for coming to our talk, we had quite a few come to our BOF after the talk to map out the above requirements and it is greatly appreciated. We also want to say thankyou to @everzet whose insight fueled the discussion and provided valuable "off the island" opinions.

To keep track of this work please subscribe to the following Drupal.org issue: https://www.drupal.org/node/2232861