If you went to Playwright.dev and looked up the documentation on how to setup playwright to run on Circle CI you are mostly given the option to run a docker image provided by microsoft and thats about it.

However, the specific codebase I was working on had some special nodeJS engine requirements which made it all the harder to use the docker image by Microsoft which was lacking any documentation of configuration options, literally.

All you see in the screenshot above is all you get in terms of documentation.

If you want to learn how to load and run your first test on the chrome extension read here

Going the highway

This left me no option but to manually setup the configuration needed for running playwright on CircleCI.

The result is the following

version: 2.1

executors:
  plugin-e2e-tester:
    docker:
      - image: cimg/node:14.15.0-browsers
    working_directory: ~/repo
    resource_class: medium+

aliases:
  - &cache_restore
    restore_cache:
      keys:
        - v2-dependencies-{{ checksum "yarn.lock" }}

  - &cache_save
    save_cache:
      paths:
        - node_modules
        - ~/.cache/yarn
      key: v2-dependencies-{{ checksum "yarn.lock" }}

  - &install_dependencies
    run: yarn install --frozen-lockfile --cache-folder ~/.cache/yarn
      
jobs:      
  plugin-e2e-test:
    executor: plugin-e2e-tester
    parallelism: 1
    steps:
      - checkout
      - *cache_restore
      - *install_dependencies
      - *cache_save

      - run:
          name: Build the plugin
          command: |
            yarn plugin build:e2e     

      - run:
          name: Install playwright
          command: |
            npx playwright install --with-deps chromium

      - run:
          name: Run E2E tests
          command: |
            yarn plugin e2e
          environment:
            PLAYWRIGHT_JUNIT_OUTPUT_DIR: ~/repo/apps/plugin/e2e/reports/junit/
            PLAYWRIGHT_JUNIT_OUTPUT_NAME: results.xml

      - store_test_results:
          path: ~/repo/apps/plugin/e2e/reports/junit

      - store_artifacts:
          path: ~/repo/apps/plugin/e2e/recordings
          destination: artifacts


workflows:
  version: 2
  all:
    jobs:
      - plugin-e2e-test

The crucial bit that got me some time to figure out, is that Playwright is using patched versions of chromium, webkit and firefox and hence using a random browser orb from Circle CI wouldn't do the trick. So this line shown below is very important

npx playwright install --with-deps chromium

Video recordings and reporter

To setup the reporter and video recordings apart from doing so above in config.yml you also need to set the paths in playwright.config.js so don't forget to do that.

Conclusion

All in all, it was an interesting journey but I would love for microsoft to give more attention to the documentation for CI/CD.

Petros Kyriakoupersonal blog

How to setup Playwright on Circle CI

Going the highway

Video recordings and reporter

Conclusion