Competence4Energy: API(Backend)

The Competence4Energy API serves as the backbone for Competence4Energy application. This platform is designed to bridge the gap between researchers and professionals by allowing them to map, manage, and discover expertise within specific research fields.This repository is based on the original ZLE-API framework. However, we have implemented several extra features to improve functionality and user experience,

The backend is based on Django. For testing locally, a lightweight SQLite3 database is used. In production, Django runs inside a Docker container and the database is a PostgreSQL database running in a separate container.

Table of Contents

• Testing Locally with Django Runserver
• Access Admin Panels
• Local Installation via Docker (Deprecated)
• Backup and Restore PostgreSQL Database
• Navigation Through PostgreSQL Database
• OAuth Authentication Flow

Testing Locally with Django Runserver

1. Create .env file in root directory

Create a .env file in the root directory with the following content:

# Database credentials
COMPETENCE_DB_ADMIN=          # PostgreSQL database admin username
COMPETENCE_DB_PASSWORD=       # PostgreSQL database admin password
COMPETENCE_SECRET_KEY=        # Django secret key for cryptographic signing

# OAuth authentication credentials
GITLAB_CLIENT_ID=             # GitLab OAuth application client ID
GITLAB_CLIENT_SECRET=         # GitLab OAuth application client secret
REGAPP_CLIENT_SECRET=         # NFDI RegApp OAuth client secret

# Application URLs
FRONTEND_BASE_URL=http://localhost:8080  # Frontend application base URL
BACKEND_BASE_URL=http://localhost:8000   # Backend API base URL

2. Create a virtual environment (optional but recommended)

python3 -m venv venv
source venv/bin/activate  # On Windows, use "venv\Scripts\activate"

3. Install dependencies

Install all required packages from requirements.txt:

pip3 install -r requirements.txt

Key Dependencies:

• Django (5.2.3): The main web framework
• django-allauth (65.9.0): Authentication and OAuth integration
• djangorestframework (3.16.0): REST API framework
• django-cors-headers (4.7.0): CORS handling for frontend communication
• psycopg2-binary (2.9.10): PostgreSQL database adapter
• pandas (2.3.0): Data manipulation for Excel file processing
• openpyxl (3.1.5): Excel file reading/writing
• bibtexparser (1.4.3): BibTeX file parsing
• python-dotenv (1.1.0): Environment variable management

4. Setup database

python3 manage.py makemigrations
python3 manage.py migrate
python3 read_init_data.py
# Deletes and initializes database with test data, also resets ID counter for PostgreSQL

5. Create superuser

Create a Django superuser account to access the admin panel:

python3 manage.py createsuperuser

Follow the prompts to enter a username, email (optional), and password for the superuser account.

6. Run development server

source venv/bin/activate  # If not already activated
python3 -u manage.py runserver 0.0.0.0:8000

Access Admin Panels

Links

• Django Admin Panel:
  • Local: http://localhost:8000/competencebackend/admin/

Local Installation via Docker

1. Install Docker

Download and install Docker Desktop from: https://www.docker.com/products/docker-desktop/

2. Create .env file

Create a .env file in the zle-api root directory with the same content as described in Testing Locally with Django Runserver - Step 1.

3. Build API image and compose

docker build -t competence_api_image .
docker compose up -d

4. Go into zle-api container and migrate database

Using Docker Desktop app or command line:

docker exec -it competence_api /bin/sh

Inside the container:

python manage.py collectstatic --no-input
python3 manage.py makemigrations api  # If migrations have not already been run in the codebase
python manage.py migrate
python3 read_init_data.py

If migration fails:

python manage.py migrate api zero
python manage.py migrate

5. Create superuser

Create a Django superuser account to access the admin panel:

python manage.py createsuperuser

Follow the prompts to enter a username, email (optional), and password for the superuser account.

6. Access backend

You should be able to access the backend at:

http://localhost:8000/competencebackend/admin/

Backup and Restore PostgreSQL Database

Backup

cd /home/dev/competence/postgres_backup
docker exec -t competence_postgres_db pg_dumpall -c -U postgres > dump_$(date "+%d-%m-%Y"_"%H_%M_%S").sql

Restore

cd /home/dev/competence/postgres_backup
cat your_dump.sql | docker exec -i competence_postgres_db psql -U competence_admin

Navigation Through PostgreSQL Database

Access PostgreSQL container

docker exec -it competence_postgres_db /bin/sh

Connect to database

psql -U competence_admin -W competence

Password is found in /home/dev/competence/competence_api/.env

Useful PostgreSQL commands

• List tables: \dt
• List table schema: \d api_researchproject
• List table data: SELECT * FROM api_user;

OAuth Authentication Flow

The application uses OAuth 2.0 for authentication with RegApp NFDI-AAI service. The authentication is handled through django-allauth with OpenID Connect provider configuration.

Authentication Flow

1. Initial Login Request

   • Frontend redirects user to the authorization URL provided by django-allauth
   • User is redirected to RegApp NFDI-AAI authentication page

2. Authorization

   • User authenticates with RegApp
   • RegApp redirects back to the callback URL with an authorization code
   • Backend receives the callback through django-allauth

3. Token Exchange

   • django-allauth exchanges the authorization code for access and refresh tokens
   • User account is created or linked if it doesn't exist
   • User is redirected to the frontend

4. Session Management

   • Django session is established for authenticated users
   • Frontend can check authentication status via /competencebackend/auth_status/
   • User profile can be retrieved via /competencebackend/user_profile/

5. Logout

   • Frontend makes a request to logout endpoint
   • Django session is invalidated
   • User is redirected to the frontend

Security Measures

• OAuth 2.0 with PKCE (Proof Key for Code Exchange) enabled
• State parameter is used to prevent CSRF attacks
• Secure token storage and handling through django-allauth
• All sensitive routes require valid authentication
• Session-based authentication for API requests

Configuration

OAuth is configured in api/settings.py with the following providers:

• GitLab: For GitLab authentication
• NFDI RegApp Community: OpenID Connect provider for NFDI-AAI authentication

Environment variables required:

• GITLAB_CLIENT_ID
• GITLAB_CLIENT_SECRET
• REGAPP_CLIENT_SECRET