PHPackages sandstorm/e2etesttools - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. sandstorm/e2etesttools ActiveNeos-package sandstorm/e2etesttools ====================== 8.3.2(2mo ago)1040.4k↓31.8%2[6 issues](https://github.com/sandstorm/Sandstorm.E2ETestTools/issues)[1 PRs](https://github.com/sandstorm/Sandstorm.E2ETestTools/pulls)MITPHP Since May 25Pushed 2mo ago7 watchersCompare [ Source](https://github.com/sandstorm/Sandstorm.E2ETestTools)[ Packagist](https://packagist.org/packages/sandstorm/e2etesttools)[ RSS](/packages/sandstorm-e2etesttools/feed)WikiDiscussions main Synced 1mo ago READMEChangelog (10)Dependencies (11)Versions (18)Used By (0) End-To-End Test Tools ===================== [](#end-to-end-test-tools) ... for Neos and Flow Projects. **for SYMFONY projects, see [README.Symfony.md](./README.Symfony.md).** We use [Playwright](https://playwright.dev) as browser orchestrator for tests involving a real browser. We use Behat as the test framework for writing all kinds of BDD tests. - a way to test Fusion code with Behat - Utilities for integrating the [Playwright](https://playwright.dev) browser orchestrator and the Behat test framework - [End-To-End Test Tools](#end-to-end-test-tools) - [Installation and Setup Instructions](#installation-and-setup-instructions) - [Setting up Playwright](#setting-up-playwright) - [Creating a FeatureContext](#creating-a-featurecontext) - [Loading CSS and JavaScript for the Styleguide](#loading-css-and-javascript-for-the-styleguide) - [Pipeline Setup](#pipeline-setup) - [Running Behat Tests](#running-behat-tests) - [Style Guide](#style-guide) - [Writing Behat Tests](#writing-behat-tests) - [Fusion Component Testcases](#fusion-component-testcases) - [Fusion Integration Testcases](#fusion-integration-testcases) - [Full-Page Snapshot Testcases](#full-page-snapshot-testcases) - [Architecture](#architecture) Installation and Setup Instructions =================================== [](#installation-and-setup-instructions) - either copy .mise.toml and update it, then run `mise run e2e:setup` - or do the following: ``` composer require sandstorm/e2etesttools @dev ./flow behat:setup ./flow behat:kickstart Your.SitePackageKey http://127.0.0.1:8081 rm bin/selenium-server.jar # we do not need this ``` - you can delete `behat.yml` and only keep `behat.yml.dist` - in `behat.yml.dist`, remove the `Behat\MinkExtension` part completely. > Mink is generic a "browser controller API" which in our experience is a bit brittle to use and adds unnecessary complexity. We recommend to instead use Playwright directly. - You should configure the Flow/Neos `Configuration/Testing/Behat/Settings.yaml` and copy the production `Settings.yaml`there; to ensure that Behat is accessing the same Database like the production application. - You should create a `Configuration/Development/Docker/Behat/Settings.yaml` with the following contents: ``` Neos: Flow: persistence: backendOptions: dbname: '%env:DB_NEOS_DATABASE_E2ETEST%' ``` - You should create a `Configuration/Production/Kubernetes/Behat/Settings.yaml` with the following contents: ``` Neos: Flow: persistence: backendOptions: dbname: '%env:DB_NEOS_DATABASE_E2ETEST%' ``` Setting up Playwright --------------------- [](#setting-up-playwright) We suggest copying `Resources/Private/e2e-testrunner-template` of this package to the root of the Git Repository and name the folder `e2e-testrunner` (in our projects, usually one level ABOVE the Neos Root Directory). Also, make sure you have installed playwright on your device. `npx playwright install` Creating a FeatureContext ------------------------- [](#creating-a-featurecontext) The `FeatureContext` is the PHP class containing the step definitions for the Behat scenarios. We provide base traits you should use for various functionality. We provide a working skeleton for you to use under `Tests/Behavior/Bootstrap/FeatureContext.php.default`Copy this file as your `FeatureContext.php` to your project `Tests/Behavior/Bootstrap/FeatureContext.php``cp -R ./Packages/Application/Sandstorm.E2ETestTools/Tests/Behavior/Bootstrap/FeatureContext.php.default ./DistributionPackages//Tests/Behavior/Bootstrap/FeatureContext.php`Inside the file, check the paths to the provided Sandstorm traits and update if necessary. Loading CSS and JavaScript for the Styleguide --------------------------------------------- [](#loading-css-and-javascript-for-the-styleguide) In your Fusion code, add the JavaScript and CSS of your page to the `Sandstorm.E2ETestTools:StyleguideStylesheets`and `Sandstorm.E2ETestTools:StyleguideJavascripts` prototypes, e.g. in the following way: ``` prototype(Sandstorm.E2ETestTools:StyleguideStylesheets) { headerAssets = PACKAGEKEY:Resources.HeaderAssets } ``` > Additionally, the base URL needs to be configured correctly. This package sets it to "/" in the `Testing/Behat`context which will work in most cases out of the box. Pipeline Setup -------------- [](#pipeline-setup) We provide a skeleton to run e2e tests in your gitlab pipeline. Add the provided lines from our `.gitlab-ci.yml` to yours and adjust accordingly. Every related service (like redis, database, ...) needs to be started using a `servives` entry. Ensure the Docker image version of the service matches the development and production image from `docker-compose.yml`. The *environment variables* of the job are passed on to *all services* - so all connected services and the main job share the same environment variables. Thus, you need to add the environment variables for BOTH the SUT (which is the main job) and all related services to the `variables` section of the test job. Running Behat Tests =================== [](#running-behat-tests) > This is MANDATORY to read for everybody. We suggest that this section is COPIED to the readme of your project. First, you need to start the **Playwright Server** on your development machine. For that, go to `e2e-testrunner`in your Git Repo, and do: ``` npm install node index.js # now, the server is running on localhost:3000. # Keep the server running as long as you want to execute Behavioral Tests. You can leave the server # running for a very long time (e.g. a day). ``` Second, **ensure the docker containers are running**; usually by `docker-compose build && docker-compose up -d`. Then, enter the `neos` container: `docker-compose exec neos /bin/bash` and run the following commands inside the container: ``` ./flow behat:setup bin/behat -c Packages/Sites/[SITEPACKAGE_NAME]/Tests/Behavior/behat.yml.dist ``` Behat also supports running single tests or single files - they need to be specified after the config file, e.g. ``` # run all scenarios in a given folder bin/behat -c Packages/Sites/[SITEPACKAGE_NAME]/Tests/Behavior/behat.yml.dist Packages/Sites/[SITEPACKAGE_NAME]/Tests/Behavior/Features/Fusion/ # run all scenarios in the single feature file bin/behat -c Packages/Sites/[SITEPACKAGE_NAME]/Tests/Behavior/behat.yml.dist Packages/Sites/[SITEPACKAGE_NAME]/Tests/Behavior/Features/WebsiteRendering.feature # run the scenario starting at line 27 bin/behat -c Packages/Sites/[SITEPACKAGE_NAME]/Tests/Behavior/behat.yml.dist Packages/Sites/[SITEPACKAGE_NAME]/Tests/Behavior/Features/WebsiteRendering.feature:27 ``` In case of exceptions, it might be helpful to run the tests with `--stop-on-failure`, which stops the test cases at the first error. Then, you can inspect the testing database and manually reproduce the bug. Additionally, `-vvv` is a helpful CLI flag (extra-verbose) - this displays the full exception stack trace in case of errors. **For hints how to write Behat tests, we suggest to read [Sandstorm.E2ETestTools README](./Packages/Application/Sandstorm.E2ETestTools/README.md).** Style Guide ----------- [](#style-guide) If you use the Style Guide feature (`Then I store the Fusion output in the styleguide as "Button_Component_Basic"`), then your tests need to be annotated with `@playwright` and the playwright dev server needs to be running. You can then access the style guide using [127.0.0.1:8080/styleguide/](http://127.0.0.1:8080/styleguide/). The style guide contains BOTH HTML snapshots; and rendered images of the HTML. Writing Behat Tests =================== [](#writing-behat-tests) Here, we try to give examples for common Behat scenarios; such that you can easily get started. Fixture Setup ------------- [](#fixture-setup) The Sandstorm.E2ETestTools Package provides inline, delegated and hybrid fixture setups. We recommend using a hybrid approach. ### Delegated / Hybrid [](#delegated--hybrid) In your Neos Backend, select a node you want to test, go to the meta tab and click "export node". This will download a yaml file containing all the selected node's parents and all descendants of the nearest document parent (in case of dependencies as such references). Afterwards, move the downloaded yaml file into test directory and use them in your .feature file like such: ``` Given I have a site for Site Node "www-my-site" with name "www.my.side" And I have the following nodes from file "relative-path-from-test-file-to.yaml" ``` Also, you can override node properties inline: ``` Given I have the following nodes from file "relative-path-from-test-file-to.yaml" with overwrites | identifier | property | value | | 5cb3a5f7-b501-40b2-b5a8-9de169ef1105 | title | Foo | ``` ### Inline [](#inline) ``` Given I have the following nodes: | Identifier | Path | Node Type | Properties | Language | | 5cb3a5f7-b501-40b2-b5a8-9de169ef1105 | /sites | unstructured | {} | de | ``` Fusion Component Testcases -------------------------- [](#fusion-component-testcases) You can use a test case like the following for testing components - analogous to what you usually do with Monocle. Some hints: - We need to set up a minimal node tree, as otherwise we cannot render links. ``` @fixtures @playwright Feature: Testcase for Button Component Background: Given I have a site for Site Node "site" Given I have the following nodes: | Identifier | Path | Node Type | Properties | Language | | 5cb3a5f7-b501-40b2-b5a8-9de169ef1105 | /sites | unstructured | {} | de | | 5e312d5b-9559-4bd2-8251-0182e11b4950 | /sites/site | PACKAGEKEY:Document.Page | {} | de | | 9cbaa2e2-d779-4936-aa02-0dab324da93e | /sites/site/nested | PACKAGEKEY:Document.Page | {"uriPathSegment": "nested"} | de | Given I get a node by path "/sites/site" with the following context: | Workspace | Dimension: language | | live | de | Scenario: Basic Button (external link) When I render the Fusion object "/testcase" with the current context node: """ testcase = PACKAGEKEY:Component.Button { text = "External Link" link = "https://spiegel.de" isExternalLink = true } """ Then in the fusion output, the inner HTML of CSS selector "a" matches "External Link" Then in the fusion output, the attributes of CSS selector "a" are: | Key | Value | | class | button | | href | https://spiegel.de | | target | _blank | Then I store the Fusion output in the styleguide as "Button_Component_Basic" ``` Fusion Integration Testcases ---------------------------- [](#fusion-integration-testcases) It is especially valuable to not just test the Fusion component (which is more or less like a pure function), but instead test that a given *Node* renders in a certain way - so that the *wiring between Node and Fusion component*is set up correctly. A test case can look like the following one: ``` @fixtures @playwright Feature: Testcase for Button Integration Background: Given I have a site for Site Node "site" Given I have the following nodes: | Identifier | Path | Node Type | Properties | Language | | 5cb3a5f7-b501-40b2-b5a8-9de169ef1105 | /sites | unstructured | {} | de | | 5e312d5b-9559-4bd2-8251-0182e11b4950 | /sites/site | PACKAGEKEY:Document.Page | {} | de | | 9cbaa2e2-d779-4936-aa02-0dab324da93e | /sites/site/nested | PACKAGEKEY:Document.Page | {"uriPathSegment": "nested"} | de | Scenario: Secondary Button Given I create the following nodes: | Path | Node Type | Properties | Language | | /sites/site/main/testnode | PACKAGEKEY:Content.Button | {"type": "secondary", "link": "node://9cbaa2e2-d779-4936-aa02-0dab324da93e"} | de | Given I get a node by path "/sites/site/main/testnode" with the following context: | Workspace | Dimension: language | | live | de | When I render the Fusion object "/testcase" with the current context node: """ testcase = PACKAGEKEY:Content.Button """ Then in the fusion output, the attributes of CSS selector "a" are: | Key | Value | | href | /de/nested | Then I store the Fusion output in the styleguide as "Button_Integration_Secondary" ``` Full-Page Snapshot Testcases ---------------------------- [](#full-page-snapshot-testcases) This tests a complete page rendering, and not just single components. It is meant mostly for visual checking; and most likely you'll work less with specific assertions. In this case, the rendering depends on many more nodes - so setting up the behat fixture with all the relevant nodes can be a bit tedious. Luckily, there are helpers in this package to help with the process. We suggest writing a CommandController like the following: ```