In the previous post I covered what E2E (end-to-end) tests are.

In this post I do the implementation via this PR: https://github.com/k-candidate/selenium-book-search-slack-alerts/pull/40.

What I am basically doing here is running the actual code for real, except for the Slack part.
By that I mean that I am using the actual browser. It’s a tradeoff: speed (seconds) for confidence.

Why not the Slack part?

Because:

  • it is very unlikely to change.
  • the actual code runs every day. So if something changes from Slack’s side, I’d know because I won’t receive a notification in my Slack.

So what does the e2e test do here?

It checks for 2 books:

  • one that exists (in this case 1984), and asserts that it is “available”.
  • one that does not exist (a random string of characters: dfvtvrbg) and asserts that it is “not_found”.

This covers all the scenarios, including when something changes from the webserver’s side because the 2 book queries will fail and I will know that I have to adapt the code.

This e2e test will run in every PR.

PR tests

How to run the e2e test locally?

# source .venv/bin/activate
export WEBSITE_URL=https://www.theactualwebsite.com/
export SLACK_WEBHOOK_URL=https://hooks.slack.com/services/blablabla
pytest tests/e2e/ --no-cov -v -s

What new things have I learned?

Capturing output in pytest. See the -s in the command (above for the local run, and in the gha workflow file), and the print() in the test file.

For more information, see https://docs.pytest.org/en/stable/how-to/capture-stdout-stderr.html.

e2e pytest output

Final thoughts

With this, I think I am comfortable merging PRs if I see the tests pass. I mean I could consider auto-merging renovate bot’s PRs if all tests pass. But I won’t do that for now. I want to do some reading about it first.