Publish changes from main branch to the live website on GitHub pages

docs/.vitepress/config.mts

@@ -34,7 +34,8 @@ export default defineConfig(
         items: [
           { text: '7.x', link: '/docs/7x' },
           { text: '6.x', link: '/docs/6x' },
-          { text: 'Backup & restore', link: '/whpg-backup/' }
+          { text: 'Backup & restore', link: '/whpg-backup/' },
+          { text: 'PXF 6.x', link: '/pxf/6x/' }
         ]
       },
       { text: 'GitHub', link: 'https://github.com/warehouse-pg/warehouse-pg' },
@@ -101,6 +102,49 @@ export default defineConfig(
           { text: "Reference",
             link: "/whpg-backup/reference/" }
         ],
+      },
+      {
+        text: "PXF 6.x",
+        link: "/pxf/6x/",
+        collapsed: false,
+        items: [
+          { text: "Release notes", link: "/pxf/6x/release_notes/" },
+          { text: "Overview", link: "/pxf/6x/overview/" },
+          { text: "Installing", link: "/pxf/6x/installing" },
+          { text: "Configuring and starting", link: "/pxf/6x/configuring" },
+          { text: "Administering", link: "/pxf/6x/administering" },
+          {
+            text: "Connecting to external data",
+            link: "/pxf/6x/connecting/",
+            collapsed: true,
+            items: [
+              {
+                text: "Object stores",
+                link: "/pxf/6x/connecting/object-stores/",
+                collapsed: true,
+                items: [
+                  { text: "S3-compatible stores", link: "/pxf/6x/connecting/object-stores/s3" },
+                  { text: "Azure", link: "/pxf/6x/connecting/object-stores/azure" },
+                  { text: "Google Cloud Storage", link: "/pxf/6x/connecting/object-stores/gcs" },
+                ],
+              },
+              {
+                text: "Hadoop",
+                link: "/pxf/6x/connecting/hadoop/",
+                collapsed: true,
+                items: [
+                  { text: "HDFS", link: "/pxf/6x/connecting/hadoop/hdfs" },
+                  { text: "Hive", link: "/pxf/6x/connecting/hadoop/hive" },
+                  { text: "HBase", link: "/pxf/6x/connecting/hadoop/hbase" },
+                  { text: "Authenticating with Kerberos", link: "/pxf/6x/connecting/hadoop/kerberos" },
+                ],
+              },
+              { text: "SQL databases over JDBC", link: "/pxf/6x/connecting/jdbc" },
+              { text: "Network file system", link: "/pxf/6x/connecting/network-file-system" },
+            ],
+          },
+          { text: "Reference", link: "/pxf/6x/reference/" }
+        ],
       }
     ]
 

docs/pxf/6x/administering.md

@@ -0,0 +1,103 @@
+---
+title: Managing the PXF cluster
+navTitle: Managing the PXF cluster
+description: Learn how to start, stop, and check the status of PXF, and how to apply a configuration change.
+---
+
+Once PXF is running, you manage it from the coordinator host using the `pxf cluster` commands.
+
+## Understanding PXF directories and configuration files
+
+PXF uses two main directories:
+
+- `$PXF_HOME` is the read-only directory where PXF is installed, for example `/usr/local/pxf`. It holds the PXF service itself, along with the connector site templates under `templates/` that you copy into a server configuration.
+- `$PXF_BASE` is the writable runtime directory you create when you [initialize your cluster](configuring.md#initializing-and-starting-pxf), for example `$HOME/pxf-base`. It holds your configuration, including your server configurations under `servers/`. Keeping `$PXF_BASE` separate from `$PXF_HOME` means a PXF upgrade doesn't overwrite your configuration.
+
+See [Connecting to external data](connecting/index.md) for how PXF servers and profiles work.
+
+### Relocating $PXF_BASE
+
+Move `$PXF_BASE` to a new location if you're running low on disk space or reorganizing storage across hosts:
+
+1. Stop PXF on every host:
+
+    ```bash
+    pxf cluster stop
+    ```
+
+1. On the coordinator, move `$PXF_BASE` to the new location and update the environment variable, including in `~/.bashrc`:
+
+    ```bash
+    mv $PXF_BASE /new/path/to/pxf-base
+    export PXF_BASE=/new/path/to/pxf-base
+    ```
+
+1. Sync the relocated directory out to every segment host, then start PXF:
+
+    ```bash
+    pxf cluster sync
+    pxf cluster start
+    ```
+
+1. Once PXF is confirmed running from the new location, clean up the old `$PXF_BASE` directory on each segment host.
+
+## Starting, stopping, and restarting PXF
+
+Start, stop, or restart PXF across the whole cluster with a single command, rather than repeating it on every host. Only the `gpadmin` user can run these commands:
+
+```bash
+pxf cluster start
+pxf cluster stop
+pxf cluster restart
+```
+
+## Applying a configuration change
+
+Sync any change to a file under `$PXF_BASE/conf`, or to a server configuration, out to every segment host, then restart PXF. For example, to increase the PXF Java heap size:
+
+1. Edit `PXF_JVM_OPTS` in `$PXF_BASE/conf/pxf-env.sh` on the coordinator:
+
+    ```bash
+    export PXF_JVM_OPTS="-Xmx4g -Xms2g"
+    ```
+
+1. Sync the change to every segment host:
+
+    ```bash
+    pxf cluster sync
+    ```
+
+1. Restart PXF to apply it. Changes made only on the coordinator have no effect until you sync and restart:
+
+    ```bash
+    pxf cluster restart
+    ```
+
+### Configuring advanced settings
+
+Adjust settings under `$PXF_BASE/conf` to fit your environment, the same way as any other configuration change:
+
+- Change the listen address, overall log level, or thread limit by editing `pxf-application.properties`.
+- Change the JVM heap size by editing `pxf-env.sh`.
+- Set the hostname or port WHPG uses to reach PXF by setting the `PXF_HOST` and `PXF_PORT` environment variables in `~/.bashrc` on the relevant host. This change also requires restarting WHPG itself, not just PXF.
+- Fine-tune logging for individual loggers by editing `pxf-log4j2.xml`.
+- Define a custom PXF profile in `pxf-profiles.xml`, pairing a profile name with Java classes that implement PXF's plugin interfaces to read or write it.
+
+See [PXF configuration files](reference/configuration-files.md) for the full list of properties in each file.
+
+## Monitoring PXF
+
+`pxf cluster status` confirms PXF is both running and responsive on every host:
+
+```bash
+pxf cluster status
+```
+
+PXF also exposes HTTP endpoints on each host for monitoring tools, at `http://<host>:5888/<endpoint>`:
+
+| Endpoint | Description |
+|---|---|
+| `actuator/health` | The status of the PXF service. |
+| `actuator/info` | Build information for the PXF service. |
+| `actuator/metrics` | JVM, Tomcat, system, and PXF-specific metrics. |
+| `actuator/prometheus` | The same metrics, in a format a Prometheus server can scrape. |

docs/pxf/6x/configuring.md

@@ -0,0 +1,93 @@
+---
+title: Configuring and starting PXF for WarehousePG
+navTitle: Configuring and starting
+description: Learn how to configure PXF's runtime environment, initialize it across your cluster, and start the PXF service.
+---
+
+After you install PXF on every host in your WarehousePG (WHPG) cluster, set up its environment, initialize and start the service, and create the `pxf` extension.
+
+## Setting environment variables
+
+Set `$PXF_HOME` to the directory created during installation, so PXF and your shell can find the PXF service and its default configuration templates. PXF also needs a separate, writable runtime directory, [`$PXF_BASE`](administering.md#understanding-pxf-directories-and-configuration-files), where your per-cluster configuration, server definitions, and logs live. Keeping `$PXF_BASE` separate from `$PXF_HOME` means a PXF upgrade doesn't overwrite your configuration.
+
+On the coordinator, set `PXF_HOME`, `PXF_BASE`, and add the PXF `bin` directory to your `PATH`:
+
+```bash
+export PXF_HOME=/usr/local/pxf
+export PXF_BASE=$HOME/pxf-base
+export PATH="$PXF_HOME/bin:$PATH"
+```
+
+Add these lines to `~/.bashrc` on the coordinator so they persist across sessions.
+
+## Initializing and starting PXF
+
+Set up PXF's runtime configuration across the cluster and start the service, so PXF is ready to handle queries. Run the following commands from the coordinator host. Each `pxf cluster` command applies the action to every host in the cluster.
+
+1. Create the runtime configuration directory on every host:
+
+    ```bash
+    pxf cluster prepare
+    ```
+
+    This command creates `$PXF_BASE` on every host and copies the default configuration templates from `$PXF_HOME/conf` into it. If `$PXF_BASE` doesn't already exist, `pxf cluster prepare` creates it for you.
+
+1. Set `JAVA_HOME` in `$PXF_BASE/conf/pxf-env.sh` on the coordinator:
+
+    ```bash
+    echo "export JAVA_HOME=$(readlink -f $(which java) | sed 's:/bin/java$::')" >> $PXF_BASE/conf/pxf-env.sh
+    ```
+
+1. Sync the change to every host:
+
+    ```bash
+    pxf cluster sync
+    ```
+
+1. Start the PXF Java service on every host, listening on port 5888 by default:
+
+    ```bash
+    pxf cluster start
+    ```
+
+1. Confirm PXF is running on every host:
+
+    ```bash
+    pxf cluster status
+    __OUTPUT__
+    Checking status of PXF servers on coordinator host and 2 segment hosts...
+    PXF is running on 3 out of 3 hosts
+    ```
+
+## Creating the PXF extension
+
+Create the `pxf` extension in each database that needs external table access. `pxf cluster register`, run as the last step of [installing PXF](installing.md#installing-on-the-cluster), already placed the extension's control, SQL, and shared library files under `$GPHOME` on every host, so you don't need to repeat it here.
+
+1. Connect to the target database and create the extension:
+
+    ```sql
+    CREATE EXTENSION IF NOT EXISTS pxf;
+    ```
+
+    If the extension already exists from a previous PXF version, update it instead:
+
+    ```sql
+    ALTER EXTENSION pxf UPDATE;
+    ```
+
+    Repeat this step in every database where you want to query external tables.
+
+1. Grant roles access. Only WHPG superusers can use the `pxf` protocol by default, so grant `SELECT` to let a role read external tables through PXF, and `INSERT` to let it write them:
+
+    ```sql
+    GRANT SELECT ON PROTOCOL pxf TO <role_name>;
+    GRANT INSERT ON PROTOCOL pxf TO <role_name>;
+    ```
+
+::: info Note
+If you need to drop the `pxf` extension, use `DROP EXTENSION pxf CASCADE` to drop it along with any external table still using the `pxf` protocol.
+:::
+
+## Next steps
+
+See [Managing the PXF cluster](administering.md) for the PXF configuration files and ongoing cluster management commands, or go straight to configuring a connector, such as [Object stores](connecting/object-stores/index.md) or [Hadoop](connecting/hadoop/index.md), to reach an external data source.

docs/pxf/6x/connecting/hadoop/hbase.md

@@ -0,0 +1,81 @@
+---
+title: Connecting to HBase
+navTitle: HBase
+description: Configure PXF to read HBase table data and query it from WarehousePG.
+---
+
+Configure a server for HBase, then query its table data through external tables. PXF only supports reading from HBase, not writing to it.
+
+## Configuring the server
+
+Create a server that connects to an HBase cluster.
+
+::: info Note
+If you plan to filter on HBase external tables, copy `$PXF_HOME/share/pxf-hbase-*.jar` to every node in the HBase cluster first, and add its location to `$HBASE_CLASSPATH`. HBase's region servers need this JAR to execute a pushed-down filter.
+:::
+
+1. Create a server directory under `$PXF_BASE/servers`, and copy the `hbase-site.xml` template into it. For example, to configure a server named `hbase_server`:
+
+    ```bash
+    mkdir $PXF_BASE/servers/hbase_server
+    cp $PXF_HOME/templates/hbase-site.xml $PXF_BASE/servers/hbase_server
+    ```
+
+1. Edit `hbase-site.xml` with your connection details:
+
+    ```xml
+    <?xml version="1.0" encoding="UTF-8"?>
+    <configuration>
+        <property>
+            <name>hbase.rootdir</name>
+            <value>hdfs://<namenode_host>:<namenode_port>/hbase</value>
+        </property>
+        <property>
+            <name>hbase.zookeeper.quorum</name>
+            <value><zookeeper_host></value>
+        </property>
+    </configuration>
+    ```
+
+    Where:
+    - `hbase.rootdir` is the location on HDFS where HBase stores its data.
+    - `hbase.zookeeper.quorum` is the ZooKeeper ensemble HBase uses for coordination.
+
+    See [Configuration templates](../../reference/configuration-templates.md#hbase-sitexml) for the full list of `hbase-site.xml` properties.
+
+1. Sync the change to every segment host, then restart PXF to apply it:
+
+    ```bash
+    pxf cluster sync
+    pxf cluster restart
+    ```
+
+## Reading data
+
+PXF maps each HBase column to a `<column_family>:<qualifier>` column name. The following example reads an HBase table named `employees`, with `personal` and `job` column families, using the `hbase_server` server:
+
+```sql
+CREATE EXTERNAL TABLE employee_hbase ("personal:name" text, "job:department" text, "job:salary" int)
+    LOCATION ('pxf://employees?PROFILE=hbase&SERVER=hbase_server')
+    FORMAT 'CUSTOM' (FORMATTER='pxfwritable_import');
+
+SELECT * FROM employee_hbase;
+```
+
+WarehousePG (WHPG) caps column names at 63 characters and requires them to be character-based, while an HBase qualifier name can be longer or contain binary data. When a qualifier doesn't fit those rules, map it indirectly through a lookup table instead. Create a table named `pxflookup` in HBase with a single column family named `mapping`, then add a row per mapped qualifier, using the target HBase table's name as the row key and the WHPG column alias as the qualifier under `mapping`:
+
+```
+create 'pxflookup', 'mapping'
+put 'pxflookup', 'employees', 'mapping:name', 'personal:name'
+put 'pxflookup', 'employees', 'mapping:dept', 'job:department'
+```
+
+Reference the aliases directly as column names when you create the external table:
+
+```sql
+CREATE EXTERNAL TABLE employee_hbase_mapped (name text, dept text)
+    LOCATION ('pxf://employees?PROFILE=hbase&SERVER=hbase_server')
+    FORMAT 'CUSTOM' (FORMATTER='pxfwritable_import');
+```
+
+Add a `recordkey bytea` column to also read the row's key, HBase's unique per-row identifier that isn't stored in any column family. Type it `text` instead of `bytea` if you want to filter on it in a `WHERE` clause and have that filter pushed down to HBase.