Configuration

The system is configured via a single json document, privateer.json which contains information about all the moving parts: servers, clients, volumes and the vault configuration.

We imagine that your configuration will exist in some repo, and that that repo will be checked out on all involved machines. Please add .privateer_identity to your .gitignore for this repo.

A simple, complete, configuration could be:

{
    "servers": [
        {
            "name": "alice",
            "hostname": "alice.example.com",
            "port": 10022,
            "key_volume": "privateer_keys",
            "data_volume": "privateer_data",
            "container": "privateer_server"
        }
    ],
    "clients": [
        {
            "name": "bob",
            "backup": ["data"]
        }
    ],
    "volumes": [
        {
            "name": "data"
        }
    ],
    "vault": {
        "url": "http://localhost:8200",
        "prefix": "/secret/privateer"
    }
}

The sections here are all required:

• servers contains an array of server locations. Typically, you would have at least one as there's not a great deal of useful things that can be done with none. Each server will correspond to a single machine (here, alice.example.com) and have a friendly name (here, alice).
• clients contains an array of clients that might push to or pull from the servers. You will always have at least one client entry or you really cannot do anything useful. Each client has a friendly name (here, bob) and a list of targets to back up.
• volumes: contains an array of volumes that we will back up and restore over the system.
• vault: contains information about connecting to the HashiCorp Vault which contains the secrets that we will generate and use to connect from one machine to another.

There are some constraints that are enforced across sections of the configuration:

• The name fields must be unique across all clients and servers; no name can appear twice and no client can be a server.
• Every volume referenced by clients must appear in the volumes field, though you can have volumes listed in volumes that no client references.

We require the servers to explicitly set key_volume but not clients, because we expect that a single server machine might run multiple unrelated privateer servers, and that these will use different keys. If you do this, each server also requires a different port.

Practical considerations

We assume that privateer.json is committed to git, and that this is the mechanism for distributing and synchronising the configuration between machines.

Details

privateer.config

Client

Bases: BaseModel

Client configuration

Clients are only ever referred to from the client machine itself so we do not need to know their hostname.

Only the name attribute is required.

Attributes:

• name (str) –

  A friendly name for the client. This is the name used in all calls to the cli tool, or via the programmatic API.

• backup (list[str]) –

  An optional array of volumes to back up. This would be empty (or missing, equivalently) on a machine that is not the source of truth for any data, such as a staging machine. You can still restore from any volume that privateer knows about.

• key_volume (str) –

  The volume to store keys in. The default is privateer_keys which may be reasonable but is only what you want if the client only acts as a client for a single privateer configuration.

• schedule (Schedule | None) –

  Optionally a schedule for regular backups

Config

Bases: BaseModel

The privateer configuration.

Attributes:

• servers (list[Server]) –

  A list of Server descriptions. At least one here will be required to do anything useful.

• clients (list[Client]) –

  A list of Client descriptions, including the data sources that they will push into the system. At least one here will be required to do anything useful.

• volumes (list[Volume]) –

  A list of Volume descriptions. At least one here will be required to do anything useful.

• tag (str) –

  Optionally, the docker tag to use for privateer images. The default latest will be appropriate unless you are testing some new feature.

list_clients()

List known clients.

Return

A list of names of configured clients.

list_servers()

List known servers.

Return

A list of names of configured servers.

list_volumes()

List known volumes.

Return

A list of names of configured volumes.

machine_config(name)

Fetch the configuration for a given machine.

Return

Configuration for a machine; this has a different format for clients and servers, with few overlapping fields.

Schedule

Bases: BaseModel

Configure schedule for regular backups.

Attributes:

• jobs (list[ScheduleJob]) –

  An array of ScheduleJobs, describing a backup task

• port (int | None) –

  Optional port, if you want to run the yacron API. If not given, then the API will not be exposed.

• container (str) –

  Optional name of the container. If not given, we default to privateer_scheduler

ScheduleJob

Bases: BaseModel

Configure a regular backup job.

Attributes:

• server (str) –

  The name of the server to back up to.

• volume (str) –

  The name of the volume to back up.

• schedule (str) –

  The backup schedule, in "cron" format. Extension formats, such as @daily are supported, otherwise use a 5-element cron specifier. See https://crontab.guru/ for help generating and interpreting these.

Server

Bases: BaseModel

Configuration for a server.

There are no defaults for any field, so all must be provided.

Attributes:

• name (str) –

  A friendly name for the server. This is the name used in all calls to the cli tool, or via the programmatic API.

• hostname (str) –

  The full hostname for the server

• port (int) –

  The port to use for ssh. We do not require any ssh server already running at the host, but run our own, so if you are already running ssh on port 22 you should use a different port.

• key_volume (str) –

  The volume to use to persist keys. We suggest privateer_<application>_keys, where <application> is some short reference to the application being backed up.

• data_volume (str) –

  The volume to use to persist data. We suggest privateer_<application>_data, where <application> is some short reference to the application being backed up.

• container (str) –

  The name of the long-running container that will run the privateer server. We suggest privateer_<application>_server, where <application> is some short reference to the application being backed up.

Vault

Bases: BaseModel

Configure the vault.

Attributes:

• url (str) –

  The url of the vault server

• prefix (str) –

  The path prefix for secrets within a v1 key-value store. This is typically mounted in vault at /secret/, so something like /secret/privateer/<application> is a reasonable choice

• token (str | None) –

  Optional token (or name of an environment variable to find this token) to fetch secrets. If not present and a token is required then we look at the environment variables VAULT_TOKEN and VAULT_AUTH_GITHUB_TOKEN (in that order) and then prompt interactively for a token.

Volume

Bases: BaseModel

Describe a volume.

Attributes:

• name (str) –

  The name of the volume; this is the same as the name of the volume on disk (e.g., listed by docker volume list).

• local (bool) –

  An optional boolean indicating if the volume is local to the server. This is designed to support a workflow where content arrives on the server through some other process (in our case it's a barman process that is doing continual backup of a Postgres server).

read_config(path)

Read configuration from disk.

Parameters:

• path (str | Path) –

  Path to privateer.json

Return

The privateer configuration.