Function(s) and scripts for PurlDB to subscribe to Package updates over AP #47

Activity

1. keshav-space

   added this to 02-FederatedCode - Software metadataon Dec 9, 2024
2. keshav-space

   moved this to In Progress in 02-FederatedCode - Software metadataon Dec 9, 2024
3. keshav-space

   self-assigned this

   on Dec 9, 2024
4. keshav-space

   closed this as completedin #50on Dec 12, 2024
5. github-project-automation

   moved this from In Progress to Done in 02-FederatedCode - Software metadataon Dec 12, 2024
6. keshav-space commented on Dec 17, 2024

   

   keshav-space

   on Dec 17, 2024

   Last edited by keshav-space

   MemberAuthor

   More actions

   Completed in: #49, #50 and aboutcode-org/purldb#557

   To test Package subscription over ActivityPub in PurlDB follow the below steps

   Inital PurlDB setup

   1. Clone PurlDB git clone https://github.com/aboutcode-org/purldb.git && cd purldb
   2. Checkout federatedcode integration branch git checkout federatedcode-integration
   3. Run make dev && make envfile && make postgres
   4. Add federartedcode host in .env file FEDERATEDCODE_HOST_URL="http://127.0.0.1:8000"
   5. Run make run
   6. Go to http://127.0.0.1:8001/api/package_activity/ (Package update received from FederatedCode AP will be available here)

   Initial FederatedCode setup

   7. Install FederatedCode (https://github.com/aboutcode-org/federatedcode?tab=readme-ov-file#quick-installation)
   8. Create superuser python manage.py createsuperuser
   9. Login to federatedcode admin panel http://127.0.0.1:8000/admin/
   10. Go to http://127.0.0.1:8000/admin/fedcode/service/add/ and create service slect newly created superuser as user, and leave remote-actor empty.
   11. Go to http://127.0.0.1:8000/admin/fedcode/remoteactor/add/ and add purldb remote actor. Populate url with http://127.0.0.1:8001/api/api/users/@purldb and put username as purldb
   12. Go to http://127.0.0.1:8000/admin/fedcode/person/add/ to add purldb. For now put any thing in summary and public key, leave user empty and in Remote actor select the one created in step 11.
   13. Clone this repo https://github.com/aboutcode-org/aboutcode-packages-npm-385
   14. Go to go to http://127.0.0.1:8000/create-repo and add the url of your clone created in step 13 https://github.com/<YOUR-GITHUB-USERNAME>/aboutcode-packages-npm-385
   15. Run python3 manage.py sync sync_scancode_scans to sync scan

   Demo

   Once the inital setup is done follow these steps

   1. From Purldb subscribe to package update using this command python manage_purldb.py subscribe_package "pkg:npm/atlasboard"
   2. Go to your fork (created in step 14) https://github.com/<YOUR-GITHUB-USERNAME>/aboutcode-packages-npm-385/blob/main/npm/atlasboard/1.1.10/scancodeio.json and make any change in scan and commit the change.
   3. In FederatedCode run python3 manage.py sync sync_scancode_scans and python3 manage.py federate
   4. Visit http://127.0.0.1:8001/api/package_activity/ and you will find the update for pkg:npm/atlasboard in package_activity.

   Important

   Replace YOUR-GITHUB-USERNAME with your GitHub user/org name.
7. pombredanne

   moved this from Done to Validated in 02-FederatedCode - Software metadataon Dec 18, 2024
8. pombredanne

   changed the title [-]Function(s) and scripts for VulnerableCode and/or PurlDB to subscribe to Package updates over AP[/-] [+]Function(s) and scripts for PurlDB to subscribe to Package updates over AP[/+] on Dec 18, 2024
9. keshav-space commented on Jan 6, 2025

   

   keshav-space

   on Jan 6, 2025

   MemberAuthor

   More actions

   @pombredanne Here is the instruction to test the package update in purldb using the docker build.

   Note

   This is very specific to running both purdb and federatedcode on same machine other wise we already have docs here https://federatedcode.readthedocs.io/en/latest/tutorial_getting_started.html and https://federatedcode.readthedocs.io/en/latest/tutorial_remote_subscribe_package_update.html

   Important

   This is designed to be used on linux 64-bits amd with python 3.11 or 3.12

   Prerequisites

   1. Obtain the IPv4 address of your machine (you can use ip addr show).

   Both federatedcode and purldb need to communicate with each other. They cannot reach each other using the host's loopback address because the host's loopback is not accessible from inside the Docker containers.

   Initial PurlDB setup

   1. Clone PurlDB git clone https://github.com/aboutcode-org/purldb.git and cd into it.
   2. Now edit the docker-compose.yml and add "Host(Your-IPv4-address)" in nginx route rule.
      nginx label should look like.

   services:
     nginx:
       image: nginx
       labels:
         - "traefik.enable=true"
         - "traefik.http.routers.web.rule=
           Host(`127.0.0.1`)
           || Host(`localhost`) || Host(`Your-IPv4-address`)"
         - "traefik.http.routers.web.entrypoints=web"

   3. Edit the docker_purldb.env file and add these lines.

   FEDERATEDCODE_HOST_URL="http://Your-IPv4-address:81/"
   ALLOWED_HOSTS="Your-IPv4-address"
   

   4. Create purldb network docker network create purldb
   5. Build image docker compose build
   6. docker compose up
   7. Go to http://Your-IPv4-address/api/package_activity/ (Package update received from FederatedCode AP will be available here)

   Initial FederatedCode setup

   8. Clone FederatedCode https://github.com/aboutcode-org/federatedcode and cd into it.
   9. Edit docker-compose.yml and change port exposed by nginx to 81 and 442 since these ports are already in use by purldb.

      nginx:
        image: nginx
        ports:
         - 81:80
         - 442:443

   10. Now edit the docker.env file and add these lines

   ALLOWED_HOSTS="Your-IPv4-address"
   CSRF_TRUSTED_ORIGINS="http://Your-IPv4-address:81"
   

   11. Build image docker compose build
   12. docker compose up
   13. Create superuser docker compose -f docker-compose.yml run --rm web python manage.py createsuperuser
   14. Login to federatedcode admin panel http://Your-IPv4-address:81/admin/
   15. Go to http://Your-IPv4-address:81/admin/fedcode/service/add/ and create service select newly created superuser as user, and leave remote-actor empty.
   16. Go to http://Your-IPv4-address:81/admin/fedcode/remoteactor/add/ and add purldb remote actor. Populate url with http://Your-IPv4-address/api/users/@purldb/ and put username as purldb
   17. Go to http://Your-IPv4-address:81/admin/fedcode/person/add/ to add purldb. For now put any thing in summary and public key, leave user empty and in Remote actor select the one created in step 16.
   18. Fork this repo https://github.com/aboutcode-org/aboutcode-packages-npm-385
   19. Go to go to http://Your-IPv4-address:81/create-repo and add the url of your clone created in step 18 https://github.com//aboutcode-packages-npm-385
   20. Run docker compose -f docker-compose.yml run --rm web python3 manage.py sync sync_scancode_scans to sync scan

   Demo

   Once the initial setup is done follow these steps

   1. cd into purldb clone and subscribe to package update using this command docker compose -f docker-compose.yml run --rm web python manage_purldb.py subscribe_package "pkg:npm/atlasboard" . On successful subscribe you will get this response {"status": "success", "message": "Successfully subscribed to package pkg:npm/atlasboard"}
   2. Go to your fork (created in step 18) https://github.com/<YOUR-GITHUB-USERNAME>/aboutcode-packages-npm-385/blob/main/npm/atlasboard/1.1.10/scancodeio.json and make any change in scan and commit the change.
   3. In FederatedCode run docker compose -f docker-compose.yml run --rm web python3 manage.py sync sync_scancode_scans and docker compose -f docker-compose.yml run --rm web python3 manage.py federate
   4. Visit PurlDB http://Your-IPv4-address/api/package_activity/ and you will find the update for pkg:npm/atlasboard in package_activity.

   Important

   Replace YOUR-GITHUB-USERNAME with your GitHub user/org name.