Skip to content

Conformance suites ​

One list of checks, run by a small app per stack (integrations/<stack>/, each with its own test.sh; the suites are listed below), in CI on PostgreSQL 16 for each push, and on 17 and 18 every night (PG_MAJOR=17 or 18 runs a suite on another version). Every night they also run on 16 with the app connecting through PgBouncer in transaction mode (POOLER=pgbouncer, pooler.sh; the owner and the change feed stay direct). The tests are numbered after this list.

  1. Signed in, a list shows only the user's rows.
  2. Signed out on purpose, only what anyone may see.
  3. Not signed in: the not-signed-in error (AZ701), naming act_as.
  4. With a pool of one connection, a request as Ann, then one as Bob: Bob never sees Ann's rows.
  5. Concurrent requests never mix users.
  6. A refused insert: 403 with the problem body naming the rule.
  7. An update of a hidden row: 404. Of a visible row the user may not edit: 403, with the reason.
  8. Insert, then read back, works when the select rule allows it, and is explained when not.
  9. The generated names type-check, and a wrong permission name doesn't: in the generated client, and in the SDK's own calls.
  10. A background job signs in as a service principal.
  11. A fresh database: migrate, and the policy tests pass. Then the tool's own diff shows no change.
  12. The framework's test database has the policy: a copy of the migrated one for each test worker (Vitest's, pytest-xdist's; for Go, each test package, which go test runs in a process of its own). A copy's name longer than Postgres's 63 bytes is shortened the same way in every SDK: its first 54 bytes (fewer, not to cut a character in two), _, and the first 8 hex digits of the SHA-256 of the whole name.
  13. The app refuses to start on a connection that skips row-level security.
  14. After a policy change and a new migration, the app works with the new generated names.
  15. Next.js only: a signed-in page is never served from a cache to another user, and a signed-in read inside unstable_cache or "use cache" fails instead of being cached.
  16. A call the database turns down that is no refusal answers as its code's page says, with the database's words: what it names isn't there (AZ708) 404, a missing or wrong argument (AZ710) 400, a call that needs someone signed in (AZ714) or a login refused (AZ703) 401, a move inside itself (AZ713) 409. A refusal by the database's own code (AZ704, AZ705, AZ715) is 403. The app's mistakes stay errors, a 500: a share the policy doesn't declare (AZ706), a name not in the policy (AZ707), who is signed in changed by hand (AZ702). In every integration that answers HTTP, each code through each (Next.js: route() and action()), and through authzRoutes() the codes its calls can meet (AZ705, AZ706, AZ707, AZ708, AZ710, AZ714).

Every SDK reads the database's errors by the same rules: the code from the HINT alone, the not-signed-in error only with its code, Postgres's own refusal in any language, and one spelling for nobody (Using it from app code). Each SDK's own tests (test_sdk.py, sdk.test.ts, sdk_test.go, SdkTests.cs, sdk_test.rb, SdkTest.java, SdkTest.kt, sdk.rs, sdk_test.exs, SdkTest.php) check them.

fastapi/ runs the background jobs' checks (10, and 4, 6 and 7 for a job) in Celery's worker and RQ's, on a Redis of its own (tests/test_jobs.py, preview): a job started by a request acts for its service, or for the user who asked, given to it as an argument, and the next job on the worker's one connection never acts for the one before.

fastapi/ also mounts the SDK's route for an identity provider's SCIM requests (tests/test_scim.py): what the database answers is core/tests/scim.sh's to check, and these check what a route passes on and sends back. A request without the provider's API key is 401, whatever else it says of who asks. The requests a provider sends before it starts are answered, the query string as it was sent. One PATCH that adds a member of a group and removes another changes the members' rows, and the notes each of the two reads follow. A user made is answered 201 with its address. The app's own function for users (app.scim_user, which the stack pages show with the settings that name it) keeps what a request says nothing of, and a user the provider deactivates stays, not active. A body that is no JSON is 400 in SCIM's words, and an error 500, with the database's words in the app's log. The app's own user(request) isn't asked about a provider's request, in an app mounted in another too, and a sync engine answers as the async one does.

nextjs/ mounts the TypeScript SDK's (app/scim/v2/[...scim]/route.ts, tests/scim.test.ts) and sends the same requests, each twice: through the server, as a provider reaches it, and to the same handlers in the test's own process, where what runs of the SDK is measured. Beside the above: a HEAD is the database's to refuse, a server with a pool of one connection answers, the address of what was made is on the host a proxy handed on, and each driver's scim() asks the database the same way (Prisma, pg, postgres.js, Drizzle).

The Next.js suite (integrations/nextjs) also runs checks 1, 2, 4 to 7 and 9 through Neon's serverless driver, on the Node.js runtime and on the edge (edge/, an app of its own), behind Neon's wsproxy, and check 10 through a BullMQ queue on Redis: a preview.

The suites: fastapi (the Python SDK: FastAPI, SQLAlchemy, Alembic), nextjs (the TypeScript SDK: Next.js, Prisma, pg, postgres.js, Drizzle, React, Vitest), and, each a preview: node (a plain Express server on pg, Drizzle and postgres.js, Drizzle Kit's migrations, its tests on Jest; authzRoutes() is @rowstile/next's), go (the Go SDK: net/http, pgx, sqlc, goose), django (the Python SDK's Django module: Django's ORM on psycopg 3, Django's migrations, pytest-django), aspnetcore (the .NET SDK: ASP.NET Core minimal APIs, EF Core on Npgsql, EF Core's migrations, xUnit), rails (the Ruby SDK: Rails 8, Active Record, Puma, Minitest), spring (the Java SDK: Spring Boot, Spring Security, JPA and JdbcClient over HikariCP, Flyway, JUnit), ktor (the Java SDK's Kotlin module: Ktor, Exposed over HikariCP, Flyway, JUnit; the Java SDK's names, used from Kotlin), axum (the Rust SDK: Axum on sqlx, sqlx's migrations, tokio-postgres, cargo test), phoenix (the Elixir SDK: Phoenix on Bandit, Ecto on Postgrex, Ecto's migrations, Oban, ExUnit) and laravel (the PHP SDK: Laravel 13, Eloquent on pdo_pgsql, the queue's database worker, PHPUnit with paratest, PHPStan). Check 15 is Next.js's alone.

electric (a preview) is the sync engine's suite, on the sync index: Electric behind a small Express app, whose shape routes are @rowstile/client's electricShape on the shapes rowstile client electric writes, nginx as a cache in front of it, and the checks on Vitest with Electric's own client. Every user's shapes of the files and the folders stay open, live, and after each step each holds exactly the ids authz.list gives that user, once a write after the step (a rename of one of its rows) has reached it, so a row that comes late is seen:

  • E1. The first snapshot.
  • E2. A share made: the row arrives live. Revoked: it leaves.
  • E3. A folder moved out of reach: the files below it leave. Moved back: they return.
  • E4. A team left: the rows reached only through it leave, and one also reached another way (a share) stays.
  • E5. A sub-team moved under another team: its members' rows follow.
  • E6. A share that ends: the row leaves at the next authz.sync_tick(), and not before.
  • E7. confidential set: the file leaves those who read it only through its folder.
  • E8. Every user at once, in rounds: each gets their own rows. A client's own where, user id (params), table or secret is refused (400), a subset only narrows, and a POST's body that is more than a subset is refused.
  • E9. Not signed in: 401 with the problem body, and nothing is asked of Electric.
  • E10. Answers vary on who asks, and nginx never gives one user's answer to another.
  • E11. Writes through the app arrive in the shapes; a refused one is 403, a hidden row 404.
  • E12. Lint finds Electric's role without BYPASSRLS, a member of the app role, or reading a masked column; rowstile apply, reapply and a changed policy leave the shapes as they are (the files' and the folders': the same handles, no must-refetch), and authz.sync_verify() is true.
  • E13. A masked column is never in a shape, even when the client asks for it (in columns or a subset's filter).
  • E14. A user deleted: their keys go, and their shape empties.
  • E15. A session limited by scopes (an API key's), or viewing as someone, is refused (403).
  • E16. Keyed by folder: a folder moved out of reach and back writes no file's readers, and the shapes follow.
  • E17. After commit: with the worker stopped, a share waits; started again, it arrives.
  • A client that leaves while an answer streams (a live shape over SSE) ends that answer alone: the app answers the next request.
  • The worker (the same in the three sync engines' apps, which a unit test holds alike): a run that fails waits as long as between two runs before the next, and a stop settles whoever waits for a run.

And the stack's checks that apply: 1 (E1), 4 (a pool of one), 5 (E8), 11 (a fresh database migrates with the app's SQL migrations, the policy's tests pass, the lock file is current) and 13 (the app refuses to start on the owner's connection); 16 as the routes meet it (AZ703, AZ709, a hidden row).

The suite runs whole in each of the index's modes, which SYNC_MODE names: copied (the default), keyed (by folder on the files' and the folders' sync lines) and after (sync after commit). In another mode than the default, test.sh writes that mode's policy from db/policy.authz and the migration rowstile migrate writes to it from the lock file, and runs it after the app's migrations, as an app changing its sync lines would. Keyed by folder, the files' and the folders' filters take a second way in, through two more of the index's tables: the run writes the app's shapes again (src/shapes.gen.ts, put back after) and adds those tables to the publication. After commit, a worker beside the app runs authz.sync_work() in a loop, and a check that needs the index to hold every change so far waits for its next run. E16 runs keyed by folder only, E17 after commit only, and a first check says the index in force is in the mode named. Each pull request runs the three modes, a job each; every night they run on 17 and 18, and on 16 with the app through PgBouncer (Electric's own connection stays direct).

zero (a preview) is the other sync engine's suite, on the sync index: zero-cache on a role of its own (no superuser: Zero runs without event triggers), the app's publication, a small Express app with Zero's query and mutate endpoints on the module rowstile client zero writes, and the checks on Vitest with Zero's own client in Node. Every user's queries of the files, the folders and the notes stay open, live, and after each step each holds exactly the ids authz.list gives that user, once a write after the step has reached it, as Electric's do. The checks are Electric's, numbered alike, where Zero meets them:

  • Z1 to Z7. As E1 to E7: the first sync, a share made and revoked, a folder moved out and back, a team left, a sub-team moved, a share that ends (at the next authz.sync_tick()), confidential set.
  • Z8. A client that says it is someone else in its own context is sent its token's rows: the query endpoint makes the context from the request.
  • Z9. Not signed in: the endpoint's 401, which Zero shows as needs-auth, and no row.
  • Z10. What the client holds of the index is its own: its keys, the readers rows they lead to (keyed by folder, the given and inherits rows on its way), never another user's key.
  • Z11. Writes through Zero's mutators, in a transaction the mutate endpoint signed in: an allowed one arrives in the queries; a refused one comes back as an error, in the database's words, and changes nothing.
  • Z12. Lint finds Zero's role without BYPASSRLS, a member of the app role, or reading a masked column; rowstile apply, reapply and a changed policy leave the clients connected all along (no reset: zero-cache's replication slot is the same) and their queries following changes.
  • Z13. The masked body is in neither the publication nor a client; a publication that carries it is in lint.
  • Z14. A user deleted: their keys go, and their queries empty.
  • Z15. A session limited by scopes (an API key's) gets needs-auth, and so does a key that isn't one; a key without scopes syncs its user's rows. Revoked, it is refused to a client that connects with it, while an open client's queries go on following changes (zero-cache asks the endpoints again only for a new query or connection).
  • Z16. Keyed by folder: a folder moved out of reach and back writes no file's readers, and the queries follow.
  • Z17. After commit: with the worker stopped, a share waits; started again, it arrives.

And the stack's checks that apply: 1 (Z1), 4 (the app again with a pool of one, its endpoints called as one user, then nobody, then another: each query's filter and each mutation's sign-in is the asker's), 5 (Z8), 11 and 13. It runs whole in each of the index's modes, as Electric's does, a job each on every pull request; every night on 17 and 18, and on 16 with the app through PgBouncer (zero-cache's own connection stays direct).

powersync (a preview) is the third sync engine's suite, on the sync index's mirrors (mirrored): the PowerSync service on a role of its own, its bucket storage in a database of its own, the publication powersync (the mirrors and the keys), the streams rowstile client powersync writes, a small Express app that gives the clients their tokens and takes their writes, and the checks on Vitest with PowerSync's own client in Node. Every user's files, folders and notes stay open, live, and after each step each holds exactly the ids authz.list gives that user, once a write after the step has reached it, as Electric's do:

  • P1 to P7. As E1 to E7: the first sync, a share made and revoked, a folder moved out and back, a team left, a sub-team moved, a share that ends (at the next authz.sync_tick()), confidential set.
  • P8. The app's token, which lasts five minutes, is taken by the service; the same signed with another key is refused.
  • P9. Not signed in: the app gives no token, and the client gets no row.
  • P10. The client is synced no readers and no masked column, and its buckets are the keys its user holds.
  • P11. Writes through the app arrive; a refused one from a client's upload queue is the app's 403, with AZ709.
  • P12. Lint finds the service's role without BYPASSRLS, or a member of the app role; rowstile apply, reapply and a changed policy keep the mirrors, and the clients go on syncing.
  • P14. A user deleted: their keys go, and their tables empty; the app gives them no token.
  • P15. A session limited by scopes (an API key's) gets no token, and a key that isn't one is 401 (AZ703); a key without scopes syncs its user's rows. Revoked, it gets no new token, while the one it has goes on syncing until it ends.
  • P17. After commit: with the worker stopped, a share waits; started again, it arrives.

And the stack's checks that apply: 1 (P1), 3 (P9), 4 (the app again with a pool of one, asked for tokens and sent writes as one user, then nobody, then another: each token names the asker, each write is judged as theirs), 11 (with the streams rowstile client writes from the migrated database, as committed) and 13 (and the app refuses to start without a key of 32 characters at least to sign tokens with). It runs whole in the two modes a mirror takes, copied and after (a mirrored line can't be keyed by folder), a job each on every pull request; every night on 17 and 18, and on 16 with the app through PgBouncer (the service's own connection stays direct).

A new stack gets its own folder here, with every check that applies to it.