Thumper

Thumper can be used as an artificial traffic generator or/and as an availability probe for SpiceDB instances.

It can issue CheckPermission and CheckBulkPermission requests, Read/Write Relationships, ExpandPermissionTree and LookupResources. It also can expose Prometheus metrics about those operations.

Usage

1. Install thumper:

   git clone https://github.com/authzed/thumper.git
   cd thumper
   go build -o thumper ./cmd/thumper
   sudo mv thumper /usr/local/bin/  # Optional if you want to move thumper into $PATH

2. Write your script in a YAML file (see script format down below.)

3. If your script contains schema or relationship writes, run the migration step to set that data up first:

   thumper migrate --endpoint grpc.authzed.com:443 --token t_some_token ./scripts/schema.yaml

4. Run your script as in the following examples:

   # 5 requests per second against Authzed's hosted SpiceDB with a secure connection:
   thumper run --qps 5 --endpoint grpc.authzed.com:443 --token t_some_token ./scripts/example.yaml
   
   # 1 request per second against local SpiceDB with an insecure connection:
   thumper run --token presharedkeyhere --insecure ./scripts/example.yaml

Script Format

Thumper config files are YAML files. These files support Go template preprocessing supported.

The final YAML generated by the templates must validate with the schema in schema.yaml.

Example

name: create org, tenant, and add client
weight: 1
steps:
- op: CheckPermission
  resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  permission: write_relationships
  expectNoPermission: true
  consistency: AtLeastAsFresh
- op: LookupResources
  resource: {{ .Prefix }}tenant
  permission: view_tenant
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  numExpected: 0
  consistency: AtLeastAsFresh
- op: WriteRelationships
  updates:
  - op: TOUCH
    resource: {{ .Prefix }}organization:org_{{ randomObjectID }}
    subject: {{ .Prefix }}platform:plat_{{ randomObjectID }}
    relation: platform
  - op: TOUCH
    resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
    subject: {{ .Prefix }}organization:org_{{ randomObjectID }}
    relation: organization
  - op: TOUCH
    resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
    subject: {{ .Prefix }}client:client_{{ randomObjectID }}#token
    relation: writer
  - op: TOUCH
    resource: {{ .Prefix }}client:client_{{ randomObjectID }}
    subject: {{ .Prefix }}token:t_{{ randomObjectID }}
    relation: token
    caveat:
      name:  {{ .Prefix }}caveat_name
      context:
        bool_field: true
        string_field: value
        int_field: 4
        float_field: 3.14159
        null_field: null
        nested_object:
          abc: def
        nested_list:
        - 1
        - 2
        - 3
- op: CheckPermission
  resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  permission: write_relationships
  consistency: AtLeastAsFresh
- op: CheckPermission
  resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  permission: permission_with_caveat
  consistency: AtLeastAsFresh
  context:
    field_name: field_value
- op: LookupResources
  resource: {{ .Prefix }}tenant
  permission: view_tenant
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  numExpected: 1
  consistency: AtLeastAsFresh
- op: WriteRelationships
  updates:
  - op: DELETE
    resource: {{ .Prefix }}organization:org_{{ randomObjectID }}
    subject: {{ .Prefix }}platform:plat_{{ randomObjectID }}
    relation: platform
  - op: DELETE
    resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
    subject: {{ .Prefix }}organization:org_{{ randomObjectID }}
    relation: organization
  - op: DELETE
    resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
    subject: {{ .Prefix }}client:client_{{ randomObjectID }}#token
    relation: writer
  - op: DELETE
    resource: {{ .Prefix }}client:client_{{ randomObjectID }}
    subject: {{ .Prefix }}token:t_{{ randomObjectID }}
    relation: token
- op: LookupResources
  resource: {{ .Prefix }}tenant
  permission: view_tenant
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  numExpected: 0
  consistency: AtLeastAsFresh
- op: CheckPermission
  resource: {{ .Prefix }}tenant:ps_{{ randomObjectID }}
  subject: {{ .Prefix }}token:t_{{ randomObjectID }}
  permission: write_relationships
  expectNoPermission: true
  consistency: AtLeastAsFresh

Types

The following common types are used in various operations:

Type	Example(s)	Used In
Permission/Relation Name	reader, writer, view	*
Object Reference	objecttype:objectid	CheckPermission, ExpandPermissionTree, LookupSubjects, WriteRelationships
Subject Reference	subjecttype:subjectid, subjecttype:subjectid#optionalrelation	CheckPermission, ReadRelationships, DeleteRelationships, ExpandPermissionTree, WriteRelationships
Object Type	objecttype	LookupResources, LookupSubjects
Object Filter	objecttype, objecttype:objectid	ReadRelationships, DeleteRelationships

Go Template Properties

The following properties are available to be used from within go templates:

enumerate(length)

This function will generate an array with the specific length filled with the natural numbers. This array can be ranged over to repeat a script fragment a number of times with a varying identifier.

Example:

name: many checks
steps:
{{- range $i := enumerate 100 }}
- op: CheckPermission
  resource: document:{{ $i }}
  subject: user:stacy
  permission: read
{{- end }}

randomObjectID

This function returns a different random object ID per worker allowing many workers to work on the same flow in parallel. Because this function returns a randomObjectID per worker, it will require you to load a set of scripts for every worker. This can significantly increase the Thumper initialization time for high QPS tests.

Example:

name: check permissions on random document
weight: 1
steps:
- op: WriteRelationships
  updates:
  - op: TOUCH
    resource: document:{{ randomObjectID }}
    subject: user:stacy
    relation: reader
- op: CheckPermission
  resource: document:{{ randomObjectID }}
  subject: user:stacy
  permission: read

Prefix

This parameter contains the value of the --prefix command line parameter followed by a /, and can be used to isolate schemas and data between different instances of thumper.

Example:

name: check permissions on random tenant
weight: 1
steps:
- op: WriteRelationships
  updates:
  - op: TOUCH
    resource: {{ .Prefix }}document:1
    subject: {{ .Prefix }}user:stacy
    relation: reader
- op: CheckPermission
  resource: {{ .Prefix }}document:1
  subject: {{ .Prefix }}user:stacy
  permission: read

IsMigration

This parameter contains a boolean that specifies whether the script is being run under the thumper migrate command. This can be used to write a script that contains both a migration and the actual test scripts.

Example:

{{- if .IsMigration }}
---
name: write schema
steps:
- op: WriteSchema
  schema: |
    definition user {}
    definition document {
      relation reader: user
    }
- op: WriteRelationships
  updates:
  - op: TOUCH
    resource: document:1
    subject: user:stacy
    relation: reader
{{- else }}
---
name: check permissions
weight: 1
steps:
- op: CheckPermission
  resource: document:1
  subject: user:stacy
  permission: reader
{{- end }}