Era of Clarity and Efficiency in Reporting

The playwright has several built-in reporters to generate reports, including:

• HTML Reporter: Generates a self-contained folder with the test run report, which can be served as a web page.
• JSON Reporter: Generates a JSON file with the test run results.
• JUnit Reporter: Generates a JUnit XML file that can be imported into a continuous integration (CI) system.
• GitHub Actions Annotations: Add annotations to your GitHub Actions workflow to track the status of your tests.

Reporter usage in test automation has a long and evolving history. Early frameworks lacked built-in reporting capabilities, leading to a time-consuming and error-prone manual report generation.

Trends in test automation reporting include:

• Real-time reporting for immediate test results
• Advanced visualization techniques for better understanding of test results
• Integration with continuous integration (CI) systems
• Collaborative design to share reports with stakeholders

The following features in Playwright can significantly enhance the effectiveness and clarity of your tests. By leveraging these features, you can provide additional context and information about your tests, making them more informative and easier to understand for anyone reviewing them.

import { test, Page, expect } from "@playwright/test";
test('Validate Report features', async ({ page }) => {
    test.info().annotations.push({
        type: 'feature',
        description: 'Use these features to provide context and help reviewers understand tests quickly.'
    });
    await test.step('Navigate to the page', async () => {
        await page.goto('<https://the-internet.herokuapp.com/login>');
    })
    await test.step('Entering the user name', async () => {
        await expect(page.getByLabel('Username'), "Username text box not visible").toBeVisible()
        await page.getByLabel('Username').fill("tomsmith");
    })
    await test.step('Entering the password', async () => {
        await expect(page.getByLabel('Password'), "Password text box not visible").toBeVisible()
        await page.getByLabel('Password').fill("SuperSecretPassword!");
    })
    await test.step('Clicking the submit', async () => {
        await expect(page.getByRole('buttons', { name: ' Login' }), "Login button not visible").toBeVisible()
        await page.getByRole('buttons', { name: ' Login' }).click();
    })
})

In Playwright, custom metadata can be added to tests as annotations. These annotations are key/value pairs that can be accessed through test.info().annotations. They can indicate the feature being tested or provide other relevant details for debugging or issue tracking.

The test.step() function declares a step by accepting the name of the step and a callback function with the code to be tested as arguments. The function returns the value returned by the step callback. The screen capture of the report generated for the same tests is written using test.step().

Using an expect function before an action statement allows you to set a custom error message as a second argument. This message will appear in the report in case of failure, making it easier to identify what went wrong. Separating the expectation from the action also helps with code organization and improves test case readability.

To attach the trace file, modify the playwright.config.ts file. The available options are:

• off: No trace is recorded.
• on: Trace is recorded for each test.
• retain-on-failure: Trace is recorded for each test, but removed for successful runs.
• on-first-retry: Trace is recorded only on the first test retry.

My recent publication compiled a comprehensive collection of 100 similar typescript programs. Each program not only elucidates essential Typescript concepts and expounds upon the significance of test automation but also provides practical guidance on its implementation using Playwright. This resource will undoubtedly be valuable if you look deeper into similar topics.

Playwright's Cookbook: Recipes with Typescript