Running tests
In order to allow us to make changes quickly, often and with a high level of confidence, we heavily rely on tests within the web
repository.
All the steps below require you to have the web
repo cloned locally and dependencies installed.
This can be achieved by running
$ git clone https://github.com/owncloud/web.git
$ cd web
$ pnpm install
We have a steadily growing coverage of unit tests. You can run them locally via
$ pnpm test:unit
You can also specify which tests to run by giving a path param, like so: pnpm test:unit packages/<app-name>/tests/unit/path/to/test.spec.js
.
Our unit tests spec files follow a simple structure:
- fixtures and mocks at the top
- helper functions at the bottom
- tests in between
We usually organize tests with nested describe
blocks. If you would like to get feedback from the core team about
the structure, scope and goals of your unit tests before actually writing some, we invite you to make a pull request
with only describe
blocks and nested it.todo("put your test description here")
lines.
Our end-to-end test suite is built upon the Playwright Framework, which makes it easy to write tests, debug them and have them run cross-browser with minimal overhead.
Please make sure you have installed all dependencies and started the server(s) as described in tooling.
Bundle the web frontend with the following command:
$ pnpm build:w
Our compose setup automatically mounts it into an oCIS backend, respectively. Web also gets recompiled on changes.
The following command will run all available e2e tests:
$ pnpm test:e2e:cucumber 'tests/e2e/cucumber/**/*.feature'
To run a particular test, simply add the feature file and line number to the test command, e.g. pnpm test:e2e:cucumber tests/e2e/cucumber/features/smoke/admin-settings/users.feature:84
Various options are available via ENV variables, e.g.
BASIC_AUTH=true
use basic authorization for api requests.RETRY=n
to retry failuresn
timesSLOW_MO=n
to slow the execution time byn
millisecondsTIMEOUT=n
to set tests to timeout aftern
millisecondsHEADLESS=bool
to open the browser while the tests run (defaults to true => headless mode)BROWSER=name
to run tests against a specific browser. Defaults to chromium, available are chromium, firefox, webkit, chromiumADMIN_PASSWORD
to set administrator password. By default, theadmin
password is used in the test
For debugging reasons, you may want to record a video or traces of your test run. Again, you can use the following ENV variables in your command:
REPORT_DIR=another/path
to set a directory for your recorded files (defaults to “reports”)REPORT_VIDEO=true
to record a video of the test runREPORT_HAR=true
to save request information from the test runREPORT_TRACING=true
to record traces from the test run
To then open e.g. the tracing from the REPORT_DIR
, run
$ npx playwright show-trace path/to/file.zip
Run the following command to find out the lint issues early in the test codes:
$ pnpm lint
And to fix the lint problems run the following command:
$ pnpm lint --fix
If the lint problems are not fixed by --fix
option, we have to manually fix the code.
We’ve decided to switch to playwright for end-to-end tests. As we steadily increase the coverage of our playwright based e2e tests we keep the existing nightwatch based e2e tests maintained. However, we decided to not add new scenarios to the nightwatch based e2e tests anymore.
In other words: only continue reading about our nightwatch based acceptance tests below if you need to debug a failing test.
At ownCloud, we have decided to adopt Docker as the main environment for developing our application. This also applies for running our acceptance tests.
Please make sure you have installed all dependencies and started the server(s) as described in tooling.
Bundle the web frontend, which then gets mounted into the respective backends. It also gets recompiled on changes.
$ pnpm build:w
The acceptance tests need additional docker containers to be running.
$ docker compose up ocis selenium middleware-ocis
and make sure there are no conflicting ports and everything runs smoothly. You can check if everything has worked by opening https://host.docker.internal:9200 and logging in using the demo user (admin/admin).
If you’re using a M1 Mac, you need to use seleniarm/standalone-chromium:4.7.0-20221206
for now. To do so, export SELENIUM_IMAGE=seleniarm/standalone-chromium:4.7.0-20221206
.
-
Change the directory to
tests/acceptance
-
Install all the test dependencies with
pnpm install
command -
Run the tests
$ pnpm test:acceptance:ocis features/path/to/test
To watch the tests while running, open http://host.docker.internal:7900/ in the browser to access your VNC client.
The cucumber library is used as the test runner for both e2e and acceptance tests. The report generator script lives inside the tests/e2e/cucumber/report
folder. If you want to create a report after the tests are done, run the command:
node tests/e2e/cucumber/report --report-input=tests/e2e/cucumber/report/report.json
--report-input
is the path to the report file generated by the test runner. If you want to generate a report for the acceptance tests, you can run the command
node tests/e2e/cucumber/report --report-input=tests/acceptance/report/report.json
By default, the report gets generated to reports/e2e/cucumber/releaseReport/cucumber_report.html.
The location can be changed by adding the --report-location
flag.
To see all available options run
node tests/e2e/cucumber/report --help