# Conformance suites

One list of checks, run by a small app per stack (`integrations/<stack>/`, each with its own `test.sh`; the suites
are listed below), in CI
on PostgreSQL 16 for each push, and on 17 and 18 every night (`PG_MAJOR=17` or `18` runs a suite on another
version). Every night they also run on 16 with the app connecting through PgBouncer in transaction mode
(`POOLER=pgbouncer`, `pooler.sh`; the owner and the change feed stay direct). The tests are numbered after
this list.

1. Signed in, a list shows only the user's rows.
2. Signed out on purpose, only what `anyone` may see.
3. Not signed in: the not-signed-in error (AZ701), naming `act_as`.
4. With a pool of one connection, a request as Ann, then one as Bob: Bob never sees Ann's rows.
5. Concurrent requests never mix users.
6. A refused insert: 403 with the problem body naming the rule.
7. An update of a hidden row: 404. Of a visible row the user may not edit: 403, with the reason.
8. Insert, then read back, works when the select rule allows it, and is explained when not.
9. The generated names type-check, and a wrong permission name doesn't: in the generated client, and in the
   SDK's own calls.
10. A background job signs in as a service principal.
11. A fresh database: migrate, and the policy tests pass. Then the tool's own diff shows no change.
12. The framework's test database has the policy: a copy of the migrated one for each test worker (Vitest's,
    pytest-xdist's; for Go, each test package, which `go test` runs in a process of its own). A copy's name longer
    than Postgres's 63 bytes is shortened the same way in every SDK: its first 54 bytes (fewer, not to cut a
    character in two), `_`, and the first 8 hex digits of the SHA-256 of the whole name.
13. The app refuses to start on a connection that skips row-level security.
14. After a policy change and a new migration, the app works with the new generated names.
15. Next.js only: a signed-in page is never served from a cache to another user, and a signed-in read inside
    `unstable_cache` or `"use cache"` fails instead of being cached.
16. A call the database turns down that is no refusal answers as its code's page says, with the database's
    words: what it names isn't there (AZ708) 404, a missing or wrong argument (AZ710) 400, a call that needs
    someone signed in (AZ714) or a login refused (AZ703) 401, a move inside itself (AZ713) 409. A refusal by the
    database's own code (AZ704, AZ705, AZ715) is 403. The app's mistakes stay errors, a 500: a share the policy
    doesn't declare (AZ706), a name not in the policy (AZ707), who is signed in changed by hand (AZ702). In
    every integration that answers HTTP, each code through each (Next.js: `route()` and `action()`), and through
    `authzRoutes()` the codes its calls can meet (AZ705, AZ706, AZ707, AZ708, AZ710, AZ714).

Every SDK reads the database's errors by the same rules: the code from the HINT alone, the not-signed-in error
only with its code, Postgres's own refusal in any language, and one spelling for nobody
([Using it from app code](/reference/app-code.md)). Each SDK's own tests (`test_sdk.py`, `sdk.test.ts`,
`sdk_test.go`, `SdkTests.cs`, `sdk_test.rb`, `SdkTest.java`, `SdkTest.kt`, `sdk.rs`, `sdk_test.exs`,
`SdkTest.php`) check them.

`fastapi/` runs the background jobs' checks (10, and 4, 6 and 7 for a job) in Celery's worker and RQ's, on a Redis of
its own (`tests/test_jobs.py`, preview): a job started by a request acts for its service, or for the user who
asked, given to it as an argument, and the next job on the worker's one connection never acts for the one before.

`fastapi/` also mounts the SDK's route for an identity provider's SCIM requests (`tests/test_scim.py`): what the
database answers is `core/tests/scim.sh`'s to check, and these check what a route passes on and sends back. A
request without the provider's API key is 401, whatever else it says of who asks. The requests a provider sends
before it starts are answered, the query string as it was sent. One PATCH that adds a member of a group and
removes another changes the members' rows, and the notes each of the two reads follow. A user made is answered 201
with its address. The app's own function for users (`app.scim_user`, which the stack pages show with the settings
that name it) keeps what a request says nothing of, and a user the provider deactivates stays, not active. A body
that is no JSON is 400 in SCIM's words, and an error 500, with the database's words in the
app's log. The app's own `user(request)` isn't asked about a provider's request, in an app mounted in another
too, and a sync engine answers as the async one does.

`nextjs/` mounts the TypeScript SDK's (`app/scim/v2/[...scim]/route.ts`, `tests/scim.test.ts`) and sends the
same requests, each twice: through the server, as a provider reaches it, and to the same handlers in the
test's own process, where what runs of the SDK is measured. Beside the above: a HEAD is the database's to
refuse, a server with a pool of one connection answers, the address of what was made is on the host a proxy
handed on, and each driver's `scim()` asks the database the same way (Prisma, pg, postgres.js, Drizzle).

The Next.js suite (`integrations/nextjs`) also runs checks 1, 2, 4 to 7 and 9 through Neon's serverless driver, on
the Node.js runtime and on the edge (`edge/`, an app of its own), behind Neon's `wsproxy`, and check 10 through a
BullMQ queue on Redis: a preview.

The suites: `fastapi` (the Python SDK: FastAPI, SQLAlchemy, Alembic), `nextjs` (the TypeScript SDK: Next.js,
Prisma, pg, postgres.js, Drizzle, React, Vitest), and, each a preview: `node` (a plain Express server on pg, Drizzle
and postgres.js, Drizzle Kit's migrations, its tests on Jest; `authzRoutes()` is `@rowstile/next`'s), `go` (the Go
SDK: `net/http`, pgx, sqlc, goose), `django` (the Python SDK's Django module: Django's ORM on psycopg 3, Django's
migrations, pytest-django), `aspnetcore` (the .NET SDK: ASP.NET Core minimal APIs, EF Core on Npgsql, EF Core's
migrations, xUnit), `rails` (the Ruby SDK: Rails 8, Active Record, Puma, Minitest), `spring` (the Java SDK: Spring
Boot, Spring Security, JPA and JdbcClient over HikariCP, Flyway, JUnit), `ktor` (the Java SDK's Kotlin module: Ktor,
Exposed over HikariCP, Flyway, JUnit; the Java SDK's names, used from Kotlin), `axum` (the Rust SDK: Axum on sqlx,
sqlx's migrations, tokio-postgres, cargo test), `phoenix` (the Elixir SDK: Phoenix on Bandit, Ecto on Postgrex,
Ecto's migrations, Oban, ExUnit) and `laravel` (the PHP SDK: Laravel 13, Eloquent on pdo_pgsql, the queue's
database worker, PHPUnit with paratest, PHPStan). Check 15 is Next.js's alone.

`electric` (a preview) is the sync engine's suite, on the sync index: Electric behind a small Express app, whose
shape routes are `@rowstile/client`'s `electricShape` on the shapes `rowstile client electric` writes, nginx as a
cache in front of it, and the checks on Vitest with Electric's own client. Every user's shapes of the files and the
folders stay open, live, and after each step each holds exactly the ids `authz.list` gives that user, once a write
after the step (a rename of one of its rows) has reached it, so a row that comes late is seen:

- E1. The first snapshot.
- E2. A share made: the row arrives live. Revoked: it leaves.
- E3. A folder moved out of reach: the files below it leave. Moved back: they return.
- E4. A team left: the rows reached only through it leave, and one also reached another way (a share) stays.
- E5. A sub-team moved under another team: its members' rows follow.
- E6. A share that ends: the row leaves at the next `authz.sync_tick()`, and not before.
- E7. `confidential` set: the file leaves those who read it only through its folder.
- E8. Every user at once, in rounds: each gets their own rows. A client's own `where`, user id (`params`), `table`
  or secret is refused (400), a subset only narrows, and a POST's body that is more than a subset is refused.
- E9. Not signed in: 401 with the problem body, and nothing is asked of Electric.
- E10. Answers vary on who asks, and nginx never gives one user's answer to another.
- E11. Writes through the app arrive in the shapes; a refused one is 403, a hidden row 404.
- E12. Lint finds Electric's role without BYPASSRLS, a member of the app role, or reading a masked column;
  `rowstile apply`, `reapply` and a changed policy leave the shapes as they are (the files' and the folders': the
  same handles, no `must-refetch`), and `authz.sync_verify()` is true.
- E13. A masked column is never in a shape, even when the client asks for it (in `columns` or a subset's filter).
- E14. A user deleted: their keys go, and their shape empties.
- E15. A session limited by scopes (an API key's), or viewing as someone, is refused (403).
- E16. Keyed by folder: a folder moved out of reach and back writes no file's readers, and the shapes follow.
- E17. After commit: with the worker stopped, a share waits; started again, it arrives.
- A client that leaves while an answer streams (a live shape over SSE) ends that answer alone: the app answers the
  next request.
- The worker (the same in the three sync engines' apps, which a unit test holds alike): a run that fails waits as
  long as between two runs before the next, and a stop settles whoever waits for a run.

And the stack's checks that apply: 1 (E1), 4 (a pool of one), 5 (E8), 11 (a fresh database migrates with the
app's SQL migrations, the policy's tests pass, the lock file is current) and 13 (the app refuses to start on the
owner's connection); 16 as the routes meet it (AZ703, AZ709, a hidden row).

The suite runs whole in each of the index's modes, which `SYNC_MODE` names: `copied` (the default), `keyed` (`by
folder` on the files' and the folders' sync lines) and `after` (`sync after commit`). In another mode than the
default, `test.sh` writes that mode's policy from `db/policy.authz` and the migration `rowstile migrate` writes to
it from the lock file, and runs it after the app's migrations, as an app changing its sync lines would. Keyed by
folder, the files' and the folders' filters take a second way in, through two more of the index's tables: the run
writes the app's shapes again (`src/shapes.gen.ts`, put back after) and adds those tables to the publication. After
commit, a worker beside the app runs `authz.sync_work()` in a loop, and a check that needs the index to hold every
change so far waits for its next run. E16 runs keyed by folder only, E17 after commit only, and a first check says
the index in force is in the mode named. Each pull request runs the three modes, a job each; every night they run
on 17 and 18, and on 16 with the app through PgBouncer (Electric's own connection stays direct).

`zero` (a preview) is the other sync engine's suite, on the sync index: zero-cache on a role of its own (no
superuser: Zero runs without event triggers), the app's publication, a small Express app with Zero's query and
mutate endpoints on the module `rowstile client zero` writes, and the checks on Vitest with Zero's own client in
Node. Every user's queries of the files, the folders and the notes stay open, live, and after each step each holds
exactly the ids `authz.list` gives that user, once a write after the step has reached it, as Electric's do. The checks
are Electric's, numbered alike, where Zero meets them:

- Z1 to Z7. As E1 to E7: the first sync, a share made and revoked, a folder moved out and back, a team left, a
  sub-team moved, a share that ends (at the next `authz.sync_tick()`), `confidential` set.
- Z8. A client that says it is someone else in its own context is sent its token's rows: the query endpoint makes
  the context from the request.
- Z9. Not signed in: the endpoint's 401, which Zero shows as `needs-auth`, and no row.
- Z10. What the client holds of the index is its own: its keys, the readers rows they lead to (keyed by folder, the
  `given` and `inherits` rows on its way), never another user's key.
- Z11. Writes through Zero's mutators, in a transaction the mutate endpoint signed in: an allowed one arrives in the
  queries; a refused one comes back as an error, in the database's words, and changes nothing.
- Z12. Lint finds Zero's role without BYPASSRLS, a member of the app role, or reading a masked column; `rowstile
  apply`, `reapply` and a changed policy leave the clients connected all along (no reset: zero-cache's replication
  slot is the same) and their queries following changes.
- Z13. The masked body is in neither the publication nor a client; a publication that carries it is in lint.
- Z14. A user deleted: their keys go, and their queries empty.
- Z15. A session limited by scopes (an API key's) gets `needs-auth`, and so does a key that isn't one; a key
  without scopes syncs its user's rows. Revoked, it is refused to a client that connects with it, while an open
  client's queries go on following changes (zero-cache asks the endpoints again only for a new query or connection).
- Z16. Keyed by folder: a folder moved out of reach and back writes no file's readers, and the queries follow.
- Z17. After commit: with the worker stopped, a share waits; started again, it arrives.

And the stack's checks that apply: 1 (Z1), 4 (the app again with a pool of one, its endpoints called as one user, then
nobody, then another: each query's filter and each mutation's sign-in is the asker's), 5 (Z8), 11 and 13. It runs
whole in each of the index's modes, as Electric's does, a job each on every pull request; every night on 17 and 18,
and on 16 with the app through PgBouncer (zero-cache's own connection stays direct).

`powersync` (a preview) is the third sync engine's suite, on the sync index's mirrors (`mirrored`): the PowerSync
service on a role of its own, its bucket storage in a database of its own, the publication `powersync` (the mirrors
and the keys), the streams `rowstile client powersync` writes, a small Express app that gives the clients their
tokens and takes their writes, and the checks on Vitest with PowerSync's own client in Node. Every user's files,
folders and notes stay open, live, and after each step each holds exactly the ids `authz.list` gives that user, once
a write after the step has reached it, as Electric's do:

- P1 to P7. As E1 to E7: the first sync, a share made and revoked, a folder moved out and back, a team left, a
  sub-team moved, a share that ends (at the next `authz.sync_tick()`), `confidential` set.
- P8. The app's token, which lasts five minutes, is taken by the service; the same signed with another key is refused.
- P9. Not signed in: the app gives no token, and the client gets no row.
- P10. The client is synced no `readers` and no masked column, and its buckets are the keys its user holds.
- P11. Writes through the app arrive; a refused one from a client's upload queue is the app's 403, with AZ709.
- P12. Lint finds the service's role without BYPASSRLS, or a member of the app role; `rowstile apply`, `reapply`
  and a changed policy keep the mirrors, and the clients go on syncing.
- P14. A user deleted: their keys go, and their tables empty; the app gives them no token.
- P15. A session limited by scopes (an API key's) gets no token, and a key that isn't one is 401 (AZ703); a key
  without scopes syncs its user's rows. Revoked, it gets no new token, while the one it has goes on syncing until it
  ends.
- P17. After commit: with the worker stopped, a share waits; started again, it arrives.

And the stack's checks that apply: 1 (P1), 3 (P9), 4 (the app again with a pool of one, asked for tokens and sent
writes as one user, then nobody, then another: each token names the asker, each write is judged as theirs), 11 (with
the streams `rowstile client` writes from the migrated database, as committed) and 13 (and the app refuses to start
without a key of 32 characters at least to sign tokens with). It runs whole in the two modes a mirror takes, `copied` and `after` (a mirrored line
can't be keyed by folder), a job each on every pull request; every night on 17 and 18, and on 16 with the app through
PgBouncer (the service's own connection stays direct).

A new stack gets its own folder here, with every check that applies to it.
