Keystore Password Exposure in KVM Agent Security Setup

[YLChen-007]

Summary

A critical security vulnerability has been identified in the Apache CloudStack KVM agent security setup process where randomly generated keystore passwords are logged in plaintext. During the LibvirtServerDiscoverer.setupAgentSecurity() method execution, sensitive keystore passwords are exposed through SSH command logging in SSHCmdHelper.sshExecuteCmdOneShot(), compromising the security of agent-to-management-server communications.

Vulnerability Details

Component: com.cloud.hypervisor.kvm.discoverer.LibvirtServerDiscoverer

• setupAgentSecurity()
• SSHCmdHelper.sshExecuteCmdWithResult()
• SSHCmdHelper.sshExecuteCmdOneShot()

Vulnerability Type: Sensitive Information Disclosure / Password Exposure in Debug Logs (CWE-532, CWE-532)

Technical Description

Vulnerability Flow

1. Password Generation and Password Exposure

The same password is used again in the certificate import command:

Expose source 1

// void com.cloud.hypervisor.kvm.discoverer.LibvirtServerDiscoverer.setupAgentSecurity(Connection sshConnection, String agentIp, String agentHostname)
final SSHCmdHelper.SSHCmdResult setupCertResult = SSHCmdHelper.sshExecuteCmdWithResult(sshConnection,
        String.format("sudo /usr/share/cloudstack-common/scripts/util/%s " +
                        "/etc/cloudstack/agent/agent.properties %s " +  // ← keystorePassword exposed again
                        "/etc/cloudstack/agent/%s %s " +
                        "/etc/cloudstack/agent/%s \"%s\" " +
                        "/etc/cloudstack/agent/%s \"%s\" " +
                        "/etc/cloudstack/agent/%s \"%s\"",
                KeyStoreUtils.KS_IMPORT_SCRIPT,
                keystorePassword,                                        //  Password in command
                KeyStoreUtils.KS_FILENAME,
                KeyStoreUtils.SSH_MODE,
                KeyStoreUtils.CERT_FILENAME,
                certificateCommand.getEncodedCertificate(),
                KeyStoreUtils.CACERT_FILENAME,
                certificateCommand.getEncodedCaCertificates(),
                KeyStoreUtils.PKEY_FILENAME,
                certificateCommand.getEncodedPrivateKey()));

Expose source 2

// Answer com.cloud.baremetal.networkservice.BaremetalPingPxeResource.execute(PrepareCreateTemplateCommand cmd)
protected Answer execute(PrepareCreateTemplateCommand cmd) {
        ...

            String script =
                String.format("python /usr/bin/prepare_tftp_bootfile.py backup %1$s %2$s %3$s %4$s %5$s %6$s %7$s %8$s %9$s %10$s %11$s", _tftpDir, cmd.getMac(),
                    _storageServer, _share, _dir, cmd.getTemplate(), _cifsUserName, _cifsPassword, cmd.getIp(), cmd.getNetMask(), cmd.getGateWay());   //  Password in command
            if (!SSHCmdHelper.sshExecuteCmd(sshConnection, script)) {
                return new Answer(cmd, false, "prepare for creating template failed, command:" + script);
            }
         ...
    }

Expose source 3

// PreparePxeServerAnswer com.cloud.baremetal.networkservice.BaremetalPingPxeResource.execute(PreparePxeServerCommand cmd)
protected PreparePxeServerAnswer execute(PreparePxeServerCommand cmd) {

            String script =
                String.format("python /usr/bin/prepare_tftp_bootfile.py restore %1$s %2$s %3$s %4$s %5$s %6$s %7$s %8$s %9$s %10$s %11$s", _tftpDir, cmd.getMac(),
                    _storageServer, _share, _dir, cmd.getTemplate(), _cifsUserName, _cifsPassword, cmd.getIp(), cmd.getNetMask(), cmd.getGateWay());   //  Password in command
            if (!SSHCmdHelper.sshExecuteCmd(sshConnection, script)) {
                return new PreparePxeServerAnswer(cmd, "prepare PING at " + _ip + " failed, command:" + script);
            }

2. Debug Logging Exposure

Both commands are logged in SSHCmdHelper.sshExecuteCmdOneShot():

public static SSHCmdResult sshExecuteCmdOneShot(com.trilead.ssh2.Connection sshConnection, String cmd) throws SshException {
    LOGGER.debug("Executing cmd: " + cmd.split(KeyStoreUtils.KS_FILENAME)[0]);  // ← Password logged here
    Session sshSession = null;
    try {
        sshSession = sshConnection.openSession();
        Thread.sleep(1000);
        
        if (sshSession == null) {
            throw new SshException("Cannot open ssh session");
        }
        // ... execution continues
    }
    ...
	if (!StringUtils.isAllEmpty(result.getStdOut(), result.getStdErr())) {
           LOGGER.debug("SSH command: " + cmd.split(KeyStoreUtils.KS_FILENAME)[0] + "\nSSH command output:" + result.getStdOut().split("-----BEGIN")[0] + "\n" + result.getStdErr());  // ← Password logged here
    }
}

Incomplete Sanitization Attempt

The code attempts to sanitize by splitting on KS_FILENAME (likely "agent.jks"):

cmd.split(KeyStoreUtils.KS_FILENAME)[0]

However, this approach is fundamentally flawed:

The password appears before agent.jks in the command string, so it's included in the first split segment and fully exposed in logs.

For example: Expose source 1 Command:

# Original command:
sudo .../keystore-import.sh /etc/.../agent.properties Xy9$mK2pL#4nQ7vR /etc/.../agent.jks ssh ...

# After split on "agent.jks" and taking [0]:
sudo .../keystore-import.sh /etc/.../agent.properties Xy9$mK2pL#4nQ7vR /etc/.../
# ❌ Password is BEFORE the split point, so it's INCLUDED in the logged output

Again, the password appears before the split point and is fully exposed.

Root Cause Analysis

1. Flawed Sanitization Logic: The split(KeyStoreUtils.KS_FILENAME)[0] approach assumes the password appears after the keystore filename, but the actual command structure places it before

2. Position-Dependent Vulnerability: The sanitization logic is position-dependent but uses the wrong position marker

3. No Pattern-Based Redaction: No regex or pattern matching to identify and mask password parameters

4. Debug Logging in Production: Debug-level logging may be enabled in production environments, exposing sensitive data