Skip to content

Remove ansible.nodejs.org #4487

Description

@ryanaslett

Summary

ansible.nodejs.org (IBM host infra-ibm-ubuntu2004-x64-1, 169.60.150.91) is an unmanaged, unwatched AWX (open-source Ansible Tower) instance that has been effectively dead for years and should be decommissioned rather than repaired.

What's running there

  • AWX 17.1.0 (ansible/awx:17.1.0), deployed via docker-compose, containers created ~5 years ago.
  • awx_web, awx_task, awx_redis containers, plus awx_postgres.
  • No nginx/reverse proxy — the AWX web container binds host ports 80/443 directly.

Key detail: it's been idle for years, then crashed

The last time this AWX instance did anything at all was around 2023-09-29 (a project sync attempt), and the last time there's evidence of actually running a job against current code was back when the checkout was at the Feb 2022 commit. Either way, it's been sitting completely idle for roughly 2.5+ years before postgres even died in Dec 2025 — the disk-full crash-loop just finished off something that was already effectively unused.

Supporting evidence:

  • awx_postgres has been crash-looping since 2025-12-29 (no space left on device, 605+ restart attempts logged), because the host's root disk is 100% full (99G/99G).
  • The AWX-synced checkout of nodejs/build under /var/lib/awx/projects is pinned at commit 472b2953 ("Update README.md", doc: s/Node.js Foundation/Node.js Project/g #2874, merged 2022-02-21).
  • The project's .git/FETCH_HEAD (updated on each pre-job sync) has an mtime of 2023-09-29 11:56 UTC, and picked up no newer commits — the last sync attempt, whatever triggered it, did nothing useful.
  • The host's origin TLS cert (served directly by the AWX container, not through Cloudflare) is self-signed and expired 2024-04-09. The only reason https://ansible.nodejs.org still shows a valid cert to browsers is that Cloudflare is terminating TLS at the edge with the shared *.nodejs.org wildcard cert and not validating the origin connection.

Why this should be removed, not fixed

  • It's a privileged automation platform (AWX manages Ansible credentials/inventory) that nobody has been watching for 2.5+ years.
  • Disk exhaustion took down its database in Dec 2025 and nobody noticed for ~9 months — reinforcing that it has no active owner.
  • The AWX release (17.1.0) is multiple years out of date.
  • No current build/infra workflow appears to depend on it (last real activity predates most current infra automation, which now runs through this ansible/ repo directly).

Suggested next steps

  • Confirm nothing currently depends on this AWX instance (credentials stored only here, scheduled jobs, webhooks pointing at it, etc.)
  • Decommission the host (infra-ibm-ubuntu2004-x64-1 / ansible.nodejs.org, IBM, 169.60.150.91)
  • Remove its DNS record and Cloudflare configuration
  • Remove/rotate any credentials that were only ever exposed to this box (e.g. awx_password in the secrets repo)
  • Remove references to it from ansible/inventory.yml and ~/.ssh/config conventions once decommissioned

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions