Building intent-driven UI testing with Amazon Nova Act

Building Intent‑Driven UI Testing with Amazon Nova Act

Epsilon, the data‑centric marketing arm of Publicis Groupe, needed a way to scale UI testing across dozens of product teams without drowning them in browser‑infra management. Their existing Playwright regression suite worked well for deterministic checks but broke on cosmetic UI changes and required each team to maintain its own browser environment. By pairing Amazon Nova Act with the Amazon Bedrock AgentCore Browser Tool, Epsilon created a managed, multi‑tenant service that lets engineers author tests in plain English while AWS handles the underlying browser sessions.

---

1. Why Intent‑Driven Testing?

• Resilience to UI churn – Natural‑language steps such as “Confirm the rewards balance is visible” stay valid even when CSS selectors change.
• Broader accessibility – Non‑engineers can read and contribute to test definitions, improving collaboration between product, design, and QA.
• Zero‑maintenance browser layer – The Bedrock AgentCore Browser Tool supplies ephemeral, headless Chromium instances, eliminating the need to ship binaries, manage display servers, or patch browsers.

Epsilon’s goal was not to replace Playwright but to layer intent‑based validation on top of it, providing a safety net for visual and semantic checks that Playwright alone cannot guarantee.

---

2. Core Architecture

Epsilon’s UI‑testing service consists of three logical layers:

1. Platform API – A Django‑REST interface exposed through API Gateway. It accepts workflow definitions (target URL + ordered natural‑language steps) and queues execution requests.
2. Task Runner – An ECS‑hosted service running Celery workers. Each worker pulls jobs from SQS, spins up a Bedrock AgentCore Browser session, and drives the browser via a Chrome DevTools Protocol (CDP) WebSocket.
3. Nova Act Integration – Nova Act interprets each natural‑language step, translates it into CDP actions, and returns pass/fail status. Results, screenshots, and session metadata are persisted to S3; audit trails flow to CloudTrail.

The service is exposed as a standard agent (`ui-test-agent`) with two skills:

• `ui-test-recorder` – Interactive recording of steps that are automatically converted to natural‑language instructions.
• `ui-test-executor` – Execution of recorded steps, returning a structured JSON report.

Because authentication relies on the AWS credential chain (IAM roles and STS AssumeRole), each product team can run tests in its own AWS account with scoped permissions, preserving isolation and auditability.

---

3. An 18‑Step Test Workflow (Illustrative)

1. Create workflow via POST `/workflows` with `target_url` and step list.
2. Platform registers a Nova Act workflow definition.
3. User triggers execution via POST `/executions`.
4. Execution request is placed on SQS.
5. Celery worker pulls the job and launches a Bedrock AgentCore Browser session.
6. Browser navigates to `target_url`.
7. Nova Act receives step “Accept cookies if banner appears”.
8. Conditional logic checks for the banner; clicks Accept if present.
9. Step “Login with test credentials” – Nova Act fills form fields and submits.
10. Wait for MFA prompt; live view is enabled for optional human takeover.
11. Step “Verify dashboard loads” – Nova Act asserts presence of key element.
12. Capture screenshot.
13. Step “Confirm rewards balance is visible”.
14. Assert text content matches expected pattern.
15. Capture screenshot.
16. Step “If a tour overlay appears, close it”.
17. Conditional skip logic executed.
18. On completion, worker stores JSON results, screenshots, and session ID in S3, then fires a webhook to update the platform UI.

The workflow demonstrates how natural‑language steps, conditional branches, and live‑view fallbacks combine to produce robust, maintainable tests.

---

4. Best Practices & Measured Benefits

| Practice | Reason | |---|---| | Keep Playwright for deterministic regression | Playwright remains the fastest way to verify exact DOM states and API contracts. | | Use Nova Act for high‑level intent checks | Natural‑language steps survive UI tweaks and are readable by product owners. | | Leverage the recorder skill for rapid authoring | Teams can capture a user flow once and generate reusable steps without writing code. | | Enable session recording to S3 | Provides a visual audit trail for flaky failures and compliance reviews. | | Scope IAM roles per team | Guarantees isolation, limits blast radius, and satisfies security audits. |

Since rollout, Epsilon reports a 30 % reduction in test maintenance time and a 15 % increase in UI coverage across its product portfolio, while keeping Playwright as the backbone for low‑level regression.

---

5. Getting Started

1. Provision the Nova Act service – Deploy the provided CloudFormation template to create the API, SQS queue, and ECS task definition.
2. Enable Bedrock AgentCore Browser – Grant the task role `bedrock:InvokeAgentCoreBrowser` permissions.
3. Create a cross‑account role for each product team with `ui-test-agent` access.
4. Integrate the SDK – Use the Python `boto3` client to call `CreateWorkflow` and `StartExecution`.
5. Record a test – Invoke `ui-test-recorder` from the UI console, perform the flow, and save the generated steps.
6. Run and monitor – Execute via `ui-test-executor`; view results in the platform dashboard or fetch the JSON report from S3.

By following these steps, organizations can quickly extend their existing test suites with intent‑driven validation, reduce brittle selector maintenance, and offload browser infrastructure to AWS.

---

Conclusion

Amazon Nova Act paired with the Bedrock AgentCore Browser Tool gives enterprises a pragmatic path to intent‑driven UI testing that complements, rather than replaces, traditional regression frameworks. Epsilon’s multi‑tenant service shows how a shared platform can deliver resilient, natural‑language tests at scale while leveraging AWS’s managed security, identity, and browser capabilities. Teams looking to modernize their UI validation can adopt the same pattern, gaining faster feedback, lower maintenance overhead, and clearer collaboration across engineering and product stakeholders.