TEST STRATEGY

VietGara — Multi-Tenant SaaS Platform for Auto Garage Management & Maintenance

Version 1.6 — October 12, 2026 · Scope: Phase 1, Release 1.0–1.4; Phase 2 mobile BFFs; Phase 4 Quality Foundation

1. Test levels

Level	What it checks	Where it lives	Who runs it	When
Unit	One package or component in isolation, with mocks	Each repository (`make test-unit` in the backend, 90% coverage gate; `vitest` in the owner web and admin web; `flutter test` in the mobile apps)	CI	Every push
Integration	Backend repositories and migrations on a real Postgres, as the RLS-bound application role	`vietgara-backend`, next to the code (`//go:build integration`, `make test-integration`; database from `internal/platform/db/pgtest`)	CI	Every push
Design system	Token contrast (text 4.5:1, control and focus 3:1) in the light and dark themes; every component and state rendered in the gallery	`vietgara-web-ui` (`npm test`, `npm run dev`); `vietgara_core` (`test/theme_test.dart`)	CI	Every push
Contract	Proto lint, format and breaking-change checks; OpenAPI and FE types generated from the same contracts	`vietgara-proto` CI	CI	Every contract change
API (blackbox)	The deployed backend through its public REST API: workflows across modules, auth, tenant isolation, error codes	`vietgara-backend/testing/blackbox` (`make blackbox-stubbed`, `make blackbox-staging`)	CI (stubbed); QA (staging)	Every push (stubbed); every staging deploy (staging)
End to end	Complete user flows through the real owner web and backend	E2E_Scenarios/ (source of truth for scenarios); automated in `vietgara-e2e` (Playwright)	QA / CI	Every build given to QA; before every release
UAT	The release’s Definition of Done, checked by the product owner	UAT checklist (to be written per release)	Product owner	Before release sign-off

From Phase 4 (Quality Foundation) the shared test dataset lives in the public vietgara-testdata repository (YAML, JSON Schema, Go and TypeScript loaders; disposable test credentials only, consumers pin dataset tags without deploy keys) with short names (owner1, gara1-1, advisor1-1); a seeder creates it through the public API and a test-only purge removes it afterwards, so the stubbed stack and staging start and end clean. The blackbox suite targets every P1 scenario and at least 80% of API operations; Playwright covers the P1 scenarios of both web consoles; mobile device tests are deferred. Every release adds its own scenarios (Definition of Done). Performance testing (NFR-PF-06) is Release 9.0 (k6 at about 50 garages) and the security review and penetration test are Release 8.0; each gets its own document in this folder. The settlement scenario (08_Settlement.md) is automated in Release 4.1, and each later release adds its scenarios as listed in the E2E README, Section 4.

2. API (blackbox) testing

The blackbox suite calls the backend the way its clients do: REST/JSON under /api/v1, through Kong, with a bearer token. It knows nothing about the backend’s code.

Only the public contract. The suite is a separate Go module, so the compiler stops it from importing backend internals. Requests and responses are the vietgara-proto messages, so a contract change shows up as a compile error or a failing decode. Assertions are limited to what a client can see: HTTP status, the error.code in the body, and response fields. They never check database rows or logs.

One suite, two targets. The same scenarios run against either deployment. BLACKBOX_TARGET selects which:

Target	Deployment	Third parties	Runs
`stubbed`	The backend image built from the commit, behind Kong with the real routes, on Postgres, Redis and Kafka in containers	Stubbed: Mailpit for e-mail (the suite reads OTPs from it), a Google stub that publishes signing keys and mints ID tokens, local disk for files	CI, every push; locally with `make blackbox-stubbed`
`staging`	The staging deployment	Real (Mailjet, Google), except that test users’ e-mail goes to a Mailpit test inbox (Section 4)	After every staging deploy

Infrastructure we run ourselves (Postgres, Redis, Kafka) is never stubbed; the bugs worth catching live in it. Each third party is replaced by a stub that the backend reaches through a configurable URL (SMTP_HOST, GOOGLE_CERTS_URL, GOOGLE_*_ENDPOINT).

Each scenario creates what it needs. It signs up fresh users on unique e-mail addresses through the API, so scenarios run in parallel and can be repeated on a long-lived stack without a reset.
Staging runs every scenario except Google sign-in. The suite signs up test users there and reads their OTPs from the test inbox (Section 4). Google sign-in needs the Google stub, so it runs on the stubbed target only. Without the test inbox, the sign-up scenarios skip, and the read-only smoke checks can use a pre-provisioned owner.

3. API tests or E2E tests?

Every client (the owner web, the admin web, the mobile apps) goes through the same API. A rule tested at the API therefore holds for all of them, and it runs in seconds without a browser.

Behaviour	Tested in	Why
Business rules, validation, error codes, state transitions (quote → repair order → stock deduction)	API	Same for every client; fast; an exact error code is easier to check than its translated message
Authentication, sessions, OTP rules, token rotation	API	Includes cases a UI can’t produce (replayed refresh token, forged or expired Google token)
Tenant isolation and permissions (one garage’s data never reachable from another)	API	Must hold for any caller, including one that skips the UI
Mobile-only endpoints (BFFs, Google ID-token sign-in)	API	No web UI exercises them; mobile E2E comes later
Screens, navigation, forms, what the user sees after an action	E2E	Only the UI can show it
A short happy path per P1 flow, through the UI	E2E	Proves the UI is wired to the API; the rules behind it are already covered at the API

When a workflow is covered by API tests, its E2E spec covers only the UI part and does not repeat every rule. E2E scenarios stay the source of truth for UI flows (Section 5). API scenarios are documented by their test names and package comments in testing/blackbox/scenarios/<area>, one package per module.

4. Test users and data on staging

Automated tests can’t read Mailjet e-mail. On staging, the third party is therefore replaced for test users only, the same way the stubbed target replaces it for everyone. Authentication is not changed: test users get random OTPs with the usual expiry, 5-attempt limit and resend cooldown, and the suite reads them like a user would.

Test accounts are the accounts on TEST_ACCOUNTS_EMAIL_DOMAIN (blackbox.vietgara.test). The backend accepts only a domain under a TLD reserved for testing (.test, .example, .invalid, .localhost). No real person can own such an address, so the routing below never applies to a real user. The setting is never set in production.
Test inbox: every e-mail to the test domain goes to a Mailpit on the staging VM instead of Mailjet. All other recipients still get Mailjet. The suite reads Mailpit’s API on its own host (MAILPIT_ADDRESS), behind basic auth.
Cleanup: every hour, the backend deletes test accounts older than TEST_ACCOUNTS_RETENTION (24 h) with everything they own. Deletion cascades to their garages and those garages’ data, memberships and subscriptions. Every scenario creates fresh accounts, so nothing has to be cleaned up by hand.
Running it: on the staging VM, make blackbox-env (vietgara-infra) prints the suite’s settings. Then run make blackbox-staging in vietgara-backend, or the manual Blackbox tests (staging) GitHub workflow, which reads the same values from repository secrets.

Google sign-in can’t be stubbed on staging, so its scenarios stay stubbed-only.

5. End-to-end testing

Scenarios are the source of truth. Every scenario is written in E2E_Scenarios/ first, with a stable ID, named test data and exact expected results; vietgara-e2e automates it under that same ID (test('E2E-IAM-007 ...', ...)), never the reverse — a behaviour with no scenario gets one before it gets a test.
What is automated first. P1 scenarios marked Candidate, starting with sign-up and verification, garage creation, staff invitation and plan limits: the flows every release depends on. Quote → Repair Order → Inventory’s stock deduction (E2E-QUO-010, E2E-REP-008, E2E-INV-008) is the next priority once 1.0–1.1 P1s are stable, since it is the platform’s core business flow end to end.
What stays manual. The owner web’s Google sign-in popup (needs a real Google account; the API it ends in is covered by the API suite), and checks that need human judgement (layout, translations reading naturally). Each release gets a visual pass at 360, 768 and 1280px in the light and dark themes (loading, empty, error and success states included), plus a keyboard-only walkthrough of sign-in → quote → payment, including the sidebar: collapse and expand it, check that it survives a reload, opens on hover (after the short rest) and on Tab focus, and never shifts the content.
E2E in CI runs from the published images. The E2E workflow in vietgara-e2e starts the full stack (Postgres, Redis, Kafka, Mailpit, backend, both web apps, Kong) from the public ghcr.io/viettechno/* images with ci/compose.yml, so it needs no source checkout or credentials and tests what ships. Image tags default to the newest release tag of each package and can be pinned per run (backend_tag, owner_web_tag, admin_web_tag), or the suite can target an already running environment (web_base_url). A change in a web app or the backend is therefore covered after it merges and its image is published; the suite’s copy of the Kong config (ci/kong.yml) must follow vietgara-backend/kong/kong.yml.
Locators. vietgara-e2e finds elements by role, label and visible text (getByRole('option'), getByLabel), never by styling classes, so a visual change does not break it. Tables are found as table tbody tr, search-select options as option roles.
Test data. Only the named records in 00_Test_Data.md, on the vietgara.test e-mail domain, created by the baseline setup (or a scenario’s own self-contained setup, e.g. a dedicated low-limit plan for a transaction-limit test). Never real people’s e-mails or phone numbers.

6. Environments

Environment	Used for	E-mail	Data
Local (Docker Compose)	Manual E2E by QA, automated E2E in CI later	Mailpit: nothing leaves the machine	Reset freely
Blackbox stack (`testing/blackbox/compose.yml`)	API tests, locally and in CI	Mailpit	Disposable: created and destroyed with each run
Staging	Release candidate check, UAT, API tests after each deploy	Mailjet: real delivery to QA-owned mailboxes; API test users’ e-mail to the Mailpit test inbox (Section 4)	Test data only; reset per release; API test users purged after 24 h (Section 4)
Production	—	—	No test data, no E2E or API test runs

Every database runs PostgreSQL 18.4: local Docker Compose, the blackbox stack, the integration tests (pgtest) and staging, so RLS and query behaviour match across them. Production’s Cloud SQL will use the same major version. A version upgrade changes all of them together (vietgara-infra README, Rules).

7. Entry and exit criteria for a release

Entry (QA starts): CI is green on the build, including the API (blackbox) suite on the stubbed stack; the backend and owner web versions are recorded; the baseline setup completes without errors.

Exit (release can ship):

every P1 scenario passes;
every P2 scenario passes, or its failure has a bug accepted by the product owner for a later release;
the API (blackbox) suite passes on staging for the release candidate;
no open bug of severity Critical or High in the release scope;
the release’s Definition of Done (Release Plan) is shown in UAT.

8. Defects

Every bug found by an E2E run names the scenario ID and the failing step; a bug found by the API suite names the test (TestOwnersCannotReachEachOthersGarages/...) and the target. Both record the build versions and the environment, and have a severity: Critical (data loss, security or tenant isolation breach, no workaround), High (a P1 flow blocked), Medium (a P2 flow broken or a workaround exists), Low (cosmetic, wording).