Setting up Sync

Warrant Enterprise Only

Warrant Sync is a service that automatically manages warrants for your application by mapping its database tables to object types in Warrant. This automatic syncing is done by the Sync Agent, a lightweight service that captures changes to your data and sends them to Warrant. Learn more about Warrant Sync here.

This guide will walk you through the steps to get Warrant Sync running with a Sync Agent.

Configure your Database

Before setting up a Sync Agent to capture changes to your data, you must configure your database to track these changes. We also recommend creating a new database user with only the privileges necessary to capture changes to your data. This user will be used by the Sync Agent. Currently only MySQL is supported, with support for more databases coming soon. If you have a database you'd like us to support, drop us a note.

1. Start by creating a MySQL user

CREATE USER 'user'@'dbhostname' IDENTIFIED WITH mysql_native_password BY 'password';

2. Grant the user the required permissions

GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT, LOCK TABLES ON *.* TO 'user'@'dbhostname';

note

The LOCK TABLES permission is also required if you have a hosted database such as Amazon RDS or Amazon Aurora

3. Finalize the user's permissions

FLUSH PRIVILEGES;

4. Enable binary logging and GTIDs in the database by adding the following to your MySQL configuration file

log_bin                  = ON
binlog_format            = ROW
binlog_row_image         = FULL
expire_logs_days         = 10
gtid_mode                = ON
enforce_gtid_consistency = ON

note

You may have to restart MySQL for the changes to take effect.

5. Confirm the changes by checking your configuration

SHOW VARIABLES LIKE '%log_bin%';
SHOW VARIABLES LIKE '%gtid%';

Setup the Sync Agent

The Sync Agent can currently be deployed on Docker or Kubernetes, with support for more platforms coming soon. If there's a platform you'd like us to support, drop us a note.

Docker

1. Pull Docker image

docker pull warrantdev/sync-agent

2. Create a file to configure your properties, i.e. warrant.properties, and add the following properties with your own values

# warrant.properties
API_KEY=your-api-key-here
AGENT_NAME=my-warrant-sync-agent
DATABASE_HOSTNAME=db-hostname
DATABASE_DBNAME=my-db
DATABASE_PORT=3306
DATABASE_USER=some-database-user
DATABASE_PASSWORD=my-password
DATABASE_CONNECTION_TIMEZONE=America/Los_Angeles
DATABASE_WATCHED_TABLES=my-db.users,my-db.accounts

note

• The list of tables in DATABASE_WATCHED_TABLES should be comma separated in the format db_name.table_name, i.e. my_db.users,my_db.tenants.

• Use your local timezone for DATABASE_CONNECTION_TIMEZONE. For example, common timezones in the continental U.S. include America/Los_Angeles, America/New_York, America/Chicago, and America/Denver. You can also check this complete list of timezones.

• If running on MacOS, use host.docker.internal as the value for DATABASE_HOSTNAME.

For more information on the properties, take a look at our reference.

3. Start the docker container

From the directory where you created warrant.properties, start the docker container with the following command, using the --env-file flag to pass in the properties file:

docker run --name warrant-sync-agent --env-file warrant.properties warrantdev/sync-agent

Local Testing

To run the agent image locally, set DATABASE_HOSTNAME in your properties file to host.docker.internal.

If you are running the agent on your local machine, include the --network host flag:

docker run --name warrant-sync-agent --env-file warrant.properties --network host warrantdev/sync-agent

If you are running the agent on Apple Silicon, include the --platform linux/amd64 flag:

docker run --name warrant-agent --env-file warrant.properties --platform linux/amd64 warrantdev/sync-agent

4. Confirm status in Warrant dashboard

After successfully starting a Sync Agent, you should see a new agent in your dashboard with the status indicating that Warrant is receiving events from the agent.

Kubernetes

1. Create deployment config

Create a deployment.yaml file with the following configuration, replacing each environment variable's REPLACE_ME with the appropriate value

apiVersion: apps/v1
kind: Deployment
metadata:
  name: warrant-sync-agent
  labels:
    app: warrant-sync-agent
spec:
  replicas: 1
  selector:
    matchLabels:
      name: warrant-sync-agent
  template:
    metadata:
      labels:
        name: warrant-sync-agent
    spec:
      containers:
        - name: agent
          image: warrantdev/sync-agent:latest
          command: ["/bin/sh"]
          args: ["./run.sh"]
          env:
            - name: API_KEY
              value: REPLACE_ME
            - name: AGENT_NAME
              value: REPLACE_ME
            - name: DATABASE_HOSTNAME
              value: REPLACE_ME
            - name: DATABASE_DBNAME
              value: REPLACE_ME
            - name: DATABASE_PORT
              value: REPLACE_ME
            - name: DATABASE_USER
              value: REPLACE_ME
            - name: DATABASE_PASSWORD
              value: REPLACE_ME
            - name: DATABASE_CONNECTION_TIMEZONE
              value: REPLACE_ME
            - name: DATABASE_WATCHED_TABLES
              value: REPLACE_ME

note

The list of tables in DATABASE_WATCHED_TABLES should be comma separated in the format db_name.table_name, i.e. my_db.users,my_db.tenants.

For more information on the properties, take a look at our reference.

2. Apply the deployment.yaml configuration to your Kubernetes cluster

kubectl apply -f deployment.yaml

3. Confirm status in Warrant dashboard

After successfully starting a Sync Agent, you should see a new agent in your dashboard with the status indicating that Warrant is receiving events from the agent.

Configure your Object Types

Warrant Sync uses the events sent by a Sync Agent to automatically create, update, and delete warrants. In order to do this, Warrant Sync must understand how each of your object types relates to the data in your database. To specify this, you must add a source to each object type you want to sync via Warrant Sync. You can add a source to your object types via API or in the Dashboard.

Source Reference

Parameter	Description
dbType	Database type (Ex: mysql)
dbName	Name of the database where the table exists
table	Name of the table to associate object type with
primaryKey	Array of comma-separated strings that make up the primary id of the table
foreignKeys	Array of foreign key objects that describe the foreign key object relations for this table

Foreign Key Reference

Parameter	Description
type	The object type that this foreign key points to
column	Foreign key column name in the table
subject	Subject of the warrant that should be created between the two object types (This should be self or the name of the object type where this is defined)
relation	The relation between the two object types for the warrant to be created

Example Object Type Config

{
    "type": "user",
    "source": {
        "dbType": "mysql",
        "dbName": "ninecap",
        "table": "user",
        "primaryKey": ["id"],
        "foreignKeys": [
            {
                "type": "tenant",
                "column": "accountId",
                "subject": "self"
                "relation": "member",
            }
        ]
    },
    "relations": {
        "parent": {}
    }
}

This object type defines a source for the user object type. The source is a table named user in a mysql database named ninecap. The primary key for the user table is the id column. The user table also has a foreign key column accountId that references the tenant object type. The subject of a foreignKey specifies what the subject should be for warrants created by Warrant Sync for this object type. We specify the subject as self indicating that the tenant should be the subject of the created warrant with a member relation.

If we had a user id 7 created with an accountId of 10, then this is the following warrant that would be created by the Sync Service:

Object Type	Object Id	Relation	User
tenant	10	member	user:7

Properties Reference

Property	Required?	Description
API_KEY	Yes	Warrant API key used to link your agent to your Warrant account
AGENT_NAME	Yes	Sync agent name used to identify your agent
DATABASE_HOSTNAME	Yes	IP address or host name of the MySQL database server
DATABASE_DBNAME	Yes	Name of the MySQL database
DATABASE_PORT	Yes	Integer port number of the MySQL database server
DATABASE_USER	Yes	Name of the MySQL user to use when connecting to the MySQL database server
DATABASE_PASSWORD	Yes	Password to use when connecting to the MySQL database server
DATABASE_CONNECTION_TIMEZONE	Yes	Timezone used for the timestamps of database events.
DATABASE_WATCHED_TABLES	Yes	An optional, comma-separated list of regular expressions that match fully-qualified table identifiers of tables whose changes you want to capture. The connector does not capture changes in any table not included in DATABASE_WATCHED_TABLES. Each identifier is of the form databaseName.tableName. By default, the connector captures changes in every non-system table in each database whose changes are being captured.