agent-skills/dogfood/references/playwright-qa.md

Playwright Python for QA Testing

Environment Setup

Playwright Python is available on this system:

• Package: /home/ubuntu/.hermes/hermes-agent/venv/lib/python3.11/site-packages/playwright/
• Chromium: ~/.cache/ms-playwright/chromium-1217/
• Import: from playwright.sync_api import sync_playwright
• Run: python3 script.py (not node — Playwright Node module may not be installed)

Basic Pattern

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    context = browser.new_context()
    page = context.new_page()

    # Login
    page.goto('https://auth.example.com/login')
    page.wait_for_load_state('networkidle')
    page.fill('#username', 'admin_user')
    page.fill('#password', 'password')
    page.click('button[type="submit"]')
    page.wait_for_url('**/login-success**', timeout=10000)

    # Navigate to target
    page.goto('https://www.example.com/admin')
    page.wait_for_load_state('networkidle')

    # Interact and test...

    browser.close()

Event Monitoring (Critical for QA)

Console Messages and JS Errors

console_msgs = []
page_errors = []
page.on("console", lambda m: console_msgs.append(f"[{m.type}] {m.text}"))
page.on("pageerror", lambda e: page_errors.append(str(e)))

# After interactions:
for m in console_msgs:
    print(f"  {m}")
for e in page_errors:
    print(f"  ERROR: {e}")

Network Requests and Responses

api_responses = []
def on_response(r):
    if '/api/' in r.url or '/admin/' in r.url:
        api_responses.append({"url": r.url, "status": r.status})
page.on("response", on_response)

# After interactions:
for r in api_responses:
    print(f"  {r['url']} -> {r['status']}")

Failed Resource Loads

failed_resources = []
page.on("requestfailed", lambda r: failed_resources.append({"url": r.url, "error": r.failure}))

Element Query Patterns

# By text content
btn = page.query_selector('button:has-text("Save")')
links = page.query_selector_all('a:has-text("Login")')

# By CSS selector with attribute
input_el = page.query_selector('input[name="csrf_token"]')
form = page.query_selector('#contentForm')

# By role
submit = page.query_selector('button[type="submit"]')

# Get all buttons (for debugging)
all_btns = page.query_selector_all('button')
btn_texts = [b.inner_text().strip() for b in all_btns]

JavaScript Evaluation

# Check if function is defined
is_defined = page.evaluate("typeof saveDraft === 'function'")

# Get element properties
val = page.evaluate("document.getElementById('contentJson').value")

# Get page content
page_html = page.content()
body_text = page.inner_text('body')

# Execute arbitrary JS
result = page.evaluate("() => { return document.title; }")

Screenshots

# Full page
page.screenshot(path='/tmp/screenshot.png', full_page=True)

# Then analyze with vision tool

Cookie Inspection

cookies = context.cookies()
for c in cookies:
    print(f"{c['name']}: domain={c['domain']}, httpOnly={c['httpOnly']}, "
          f"secure={c['secure']}, sameSite={c.get('sameSite','N/A')}")

Testing with Custom Headers (e.g., Bearer Token)

# Create separate context with extra headers
context2 = browser.new_context(extra_http_headers={"Authorization": "Bearer fake_token"})
page2 = context2.new_page()
page2.goto('https://www.example.com/admin')
# Check if redirected to login
print(f"URL: {page2.url}")
page2.close()
context2.close()

CSP Bug Detection Pattern

When buttons with onclick="fnName()" do nothing:

1. Check console for CSP violation: "Executing inline script violates the following Content Security Policy directive 'script-src-elem'"
2. Verify function availability: page.evaluate("typeof fnName") returns "undefined"
3. Confirm script tag exists but is blocked by CSP
4. Check CSP header: curl -sI URL | grep content-security-policy
5. Look for script-src-elem directive that overrides script-src

Pitfalls

• Use Python, not Node.js: The playwright npm module may not be installed. Python works.
• expect_response timeout: Don't use broad URL patterns. Use specific path matches or handle timeout gracefully.
• expect_navigation for SPA: Single-page apps may not trigger navigation events. Use wait_for_timeout or check state changes instead.
• Rate limit testing: Don't try to trigger rate limits via Playwright — too slow. Use curl for rate limit tests.
• page.on("console") misses CSP errors: CSP violations appear as pageerror events, not console messages. Listen to both.