WarehousePG Backup

gpbackup and gprestore are Go utilities for performing WarehousePG Database backups. They are currently under active development.

Pre-Requisites

The project requires the Go Programming language version 1.23 or higher. Follow the directions here for installation, usage and configuration instructions. The project also has a dependency on sqlite3. This is installed by default on many platforms, but you must install it on your system if it is not present.

Downloading

Clone the repository:

git clone https://github.com/warehouse-pg/whpg-backup.git
cd whpg-backup

Building binaries

Make the whpg-backup directory your current working directory and run:

make depend
make build

The build target will compile the gpbackup, gprestore, and gpbackup_helper binaries and places them in $HOME/go/bin.

make build_linux is for cross compiling on macOS, and the target is Linux.

Validation and code quality

Test setup

Required for WarehousePG Database 6 or higher, several tests require the dummy_seclabel WarehousePG contrib module. This module exists only to support regression testing of the SECURITY LABEL statement. It is not intended to be used in production. Use the following commands to install the module (assumed WarehousePG Database is cloned in ~/workspace/whpg)

pushd $(find ~/workspace/whpg -name dummy_seclabel)
    make install
    gpconfig -c shared_preload_libraries -v dummy_seclabel
    gpstop -ra
    gpconfig -s shared_preload_libraries | grep dummy_seclabel
popd

Test execution

NOTE: The integration and end_to_end tests require a running WarehousePG Database instance.

To run all tests except end-to-end (linters, unit, and integration), use

make test

To run only unit tests, use

make unit

To run only integration tests

make integration

Integration test requirements

• Running WHPG instance
• WHPG's gpcloud extension

make -C gpcontrib/gpcloud/ install

• WHPG configured with --with-perl

To run end to end tests (requires a running WHPG instance), use

make end_to_end

We provide the following targets to help developers ensure their code fits Go standard formatting guidelines.

To run a linting tool that checks for basic coding errors, use

make lint

This target runs gometalinter.

Note: The lint target will fail if code is not formatted properly.

To automatically format your code and add/remove imports, use

make format

This target runs goimports and gofmt. We will only accept code that has been formatted using this target or an equivalent gofmt call.

Running the utilities

The basic command for gpbackup is

gpbackup --dbname <your_db_name>

The basic command for gprestore is

gprestore --timestamp <YYYYMMDDHHMMSS>

Run --help with either command for a complete list of options.

Cleaning up

To remove the compiled binaries and other generated files, run

make clean

More Information

The WarehousePG Backup wiki is currently under construction and will soon include detailed articles that provide in-depth explanations of various aspects of gpbackup and gprestore.

How to Contribute

See CONTRIBUTING.md file.

Code Formatting

We use goimports to format go code. See https://godoc.org/golang.org/x/tools/cmd/goimports The following command formats the gpbackup codebase excluding the vendor directory and also lists the files updated.

goimports -w -l $(find . -type f -name '*.go' -not -path "./vendor/*")

Troubleshooting

Dummy Security Label module is not installed or configured

If you see errors in many integration tests (below), review the Validation and code quality [Test setup](#Test setup) section above:

SECURITY LABEL FOR dummy ON TYPE public.testtype IS 'unclassified';
      Expected
          <pgx.PgError>: {
              Severity: "ERROR",
              Code: "22023",
              Message: "security label provider \"dummy\" is not loaded",

Tablespace already exists

If you see errors indicating the test_tablespace tablespace already exists (below), execute psql postgres -c 'DROP TABLESPACE test_tablespace' to cleanup the environment and rerun the tests.

    CREATE TABLESPACE test_tablespace LOCATION '/tmp/test_dir'
    Expected
        <pgx.PgError>: {
            Severity: "ERROR",
            Code: "42710",
            Message: "tablespace \"test_tablespace\" already exists",