Many things

CLAUDE.md

@@ -134,6 +134,15 @@ Two scripts live in `scripts/` for diagnosing and backfilling team rosters on `/
 - `cleanup_bogus_imported_users.py [--event-id <id>] [--apply]` — finds and removes the user docs left behind by the older off-by-one `parse_projects` bug. Fingerprint: `imported=True` AND `propel_id=""` AND `email_address` present but not a valid email AND `import_source` starts with `projects-`. For each matched user it prunes the doc-ref from every team's `users[]` that references it, then deletes the user doc. Dry-run by default. After running, re-run `import_hackathon_users_from_csv.py --csv-type projects` against the affected events to import the real members.
 - `backfill_devpost_winners.py --event-id <id> --devpost-url <url> [--projects-csv <path>] [--apply]` — scrapes the Devpost project gallery for EVERY project tile, flagging winners (`aside.entry-badge img.winner`). For each project it matches to a Firestore team via a layered strategy: `teams.devpost_link` exact-URL → team name (case-insensitive) → email-overlap via Devpost projects CSV (auto-discovered from `/tmp/devpost_files/<event_id>/projects-*.csv`). Two backfills happen in one pass: (1) any matched team with an empty `devpost_link` gets the gallery URL written; (2) matched WINNERS additionally get `/software/<slug>` fetched for prize text + member names, with prize strings mapped to status — "1st place" → `FOUNDING_ENGINEERS`, "Completion" or "2nd place" → `COMPLETION_SUPPORT`, anything else marked Winner → `CATEGORY_WINNER` (rank-based; multi-prize teams get the best status, all prize text retained in `awards: []`). Conflicts (team already has a different `devpost_link`) are logged but never overwritten. Unmatched winners exit with code 2 so a human notices; unmatched non-winners are listed for visibility but don't fail the run (typical for teams that registered only on Devpost). Only sets `status`, `awards`, `winners_backfilled_at/source`, and `devpost_link`; never touches `users[]`. Re-runnable. Adds `beautifulsoup4` to requirements.
 
+## Resend API constraints
+`GET /emails` (list) supports ONLY `limit` (max 100)/`after`/`before` — no recipient or date filter; results newest-first. Per-recipient status = `Emails.get(id)` using the `resend_id` stored in volunteer `sent_emails`, or webhooks. Rate limit is low single-digit req/s per team and shared with email sending.
+
+### Resend status architecture (implemented 2026-06-10)
+- **Primary path**: `GET /api/admin/emails/resend-status` — per-ID `Emails.get` with Redis cache (`resend:status:{id}`). Terminal events (`delivered`, `bounced`, `complained`, etc.) TTL 7 days; transient (`sent`, `queued`, etc.) TTL 120 s. Cap 100 IDs, 0.3 s throttle between calls.
+- **List crawl**: `POST /api/admin/emails/resend-list` — never blocks on page load. Always returns from cache immediately. Pass `force: true` to kick a background crawl. Date-bounded to 90 days; max 30 pages; 0.5 s between pages. Cache TTL 15 min fresh / 1 hour stale.
+- **Admin UI**: on volunteer load, bulk-fetches IDs from `sent_emails` via the status endpoint. "Sync from Resend" button sends `force: true` and shows a 30 s snackbar if the sync is still running.
+- **Confirmation email tracking**: `send_volunteer_confirmation_email` now accepts `volunteer_id` and appends a `sent_emails` record with `recipient_type: 'application_confirmation'` after sending.
+
 ## Resend audience sync
 `scripts/sync_resend_audience.py --source {all|profiles|volunteers|mentors|judges|sponsors|helpers|leads} --audience "<name>" [--event-id <id>] [--selected-only] [--apply]` — pulls emails from Firestore (`users.email_address`, `volunteers.email` filtered by `volunteer_type`, `leads.email`) and upserts contacts into a Resend audience (creates if missing). Dry-run by default. Re-runnable: lists existing audience contacts first and only POSTs new emails. Needs `RESEND_API_KEY` with audiences scope — the existing `RESEND_WELCOME_EMAIL_KEY` is send-only and will 401. Uses the deprecated `resend.Audiences` SDK class (now an alias for Segments) — fine for now, but if it breaks switch to `resend.Segments`.
 

api/messages/messages_service.py

@@ -668,6 +668,7 @@ def upload_image_to_cdn(request):
 
             logger.debug(f"Saving file to temporary location: {temp_filepath}")
             file.save(temp_filepath)
+            _optimize_image_for_web(temp_filepath)
 
             # Get just the file name without the path as the destination
             destination_filename = os.path.basename(temp_filepath)
@@ -724,6 +725,7 @@ def upload_image_to_cdn(request):
                 logger.debug(f"Saving base64 image to temporary file: {temp_filepath}")
                 with open(temp_filepath, 'wb') as temp_file:
                     temp_file.write(image_data)
+                _optimize_image_for_web(temp_filepath)
 
                 # Get just the file name without the path as the destination
                 destination_filename = os.path.basename(temp_filepath)
@@ -769,6 +771,7 @@ def upload_image_to_cdn(request):
                 logger.debug(f"Saving binary image to temporary file: {temp_filepath}")
                 with open(temp_filepath, 'wb') as temp_file:
                     temp_file.write(binary_data)
+                _optimize_image_for_web(temp_filepath)
 
                 # Get just the file name without the path as the destination
                 destination_filename = os.path.basename(temp_filepath)
@@ -807,6 +810,7 @@ def upload_image_to_cdn(request):
             logger.debug(f"Saving raw image to temporary file: {temp_filepath}")
             with open(temp_filepath, 'wb') as temp_file:
                 temp_file.write(image_data)
+            _optimize_image_for_web(temp_filepath)
 
             # Get just the file name without the path as the destination
             destination_filename = os.path.basename(temp_filepath)
@@ -845,3 +849,41 @@ def _is_image_file(filename):
     logger.debug(f"File extension check for {filename}: {'valid' if is_image else 'invalid'} image file")
     return is_image
 
+
+def _optimize_image_for_web(filepath: str, max_dimension: int = 2048, jpeg_quality: int = 85) -> None:
+    """Resize and compress an image in-place when it exceeds web-friendly limits.
+
+    Uses Pillow (already in requirements.txt). Skips files that are already
+    small (< 500 KB) and within max_dimension on both axes. On failure the
+    original file is left untouched so the upload can still proceed.
+    """
+    from PIL import Image, ImageOps
+    try:
+        original_size = os.path.getsize(filepath)
+        with Image.open(filepath) as _raw:
+            # Apply EXIF orientation before anything else so portrait photos
+            # from phones (which store pixels sideways + an EXIF rotation tag)
+            # are not saved rotated after the tag is stripped on re-save.
+            img = ImageOps.exif_transpose(_raw)
+            needs_resize = img.width > max_dimension or img.height > max_dimension
+            if not needs_resize and original_size < 500_000:
+                return  # already web-friendly
+
+            if needs_resize:
+                img.thumbnail((max_dimension, max_dimension), Image.LANCZOS)
+
+            ext = os.path.splitext(filepath)[1].lower()
+            if ext in ('.jpg', '.jpeg'):
+                if img.mode in ('RGBA', 'P', 'LA'):
+                    img = img.convert('RGB')
+                img.save(filepath, format='JPEG', quality=jpeg_quality, optimize=True)
+            else:
+                img.save(filepath, optimize=True)
+
+        new_size = os.path.getsize(filepath)
+        logger.info(
+            f"_optimize_image_for_web: {original_size // 1024}KB → {new_size // 1024}KB"
+        )
+    except Exception as exc:
+        logger.warning(f"_optimize_image_for_web failed for {filepath}, uploading original: {exc}")
+

api/volunteers/volunteers_views.py

@@ -817,9 +817,11 @@ def admin_list_resend_emails():
         else:
             filter_emails = None
 
-        logger.info("Listing Resend emails, filter_count=%d", len(filter_emails) if filter_emails else 0)
+        force = bool(body.get('force', False))
+        logger.info("Listing Resend emails, filter_count=%d, force=%s",
+                    len(filter_emails) if filter_emails else 0, force)
 
-        result = list_all_resend_emails(filter_emails=filter_emails)
+        result = list_all_resend_emails(filter_emails=filter_emails, force=force)
 
         if result['success']:
             return _success_response(result, "Resend email list fetched successfully")

common/utils/redis_cache.py

@@ -1,34 +1,102 @@
 import os
-from typing import Any, Optional, Callable, TypeVar
 import json
 from functools import wraps
+from typing import Any, Callable, TypeVar
+from urllib.parse import urlparse, urlunparse
+
 from cachetools import TTLCache
 
+from common.log import get_logger, info, warning
+
 T = TypeVar('T')
 
+logger = get_logger("redis_cache")
+
+LOCAL_CACHE_MAXSIZE = 1000
+LOCAL_CACHE_TTL_SECONDS = 600
+
+
+def _redact_redis_url(redis_url: str) -> str:
+    """Remove credentials from Redis URLs before logging them."""
+    parsed = urlparse(redis_url)
+    if not parsed.netloc or "@" not in parsed.netloc:
+        return redis_url
+
+    user_info, host_info = parsed.netloc.rsplit("@", 1)
+    username = user_info.split(":", 1)[0] if ":" in user_info else user_info
+    redacted_user_info = f"{username}:***" if username else "***"
+    return urlunparse(parsed._replace(netloc=f"{redacted_user_info}@{host_info}"))
+
+
+def _disable_redis(operation: str, exc: Exception) -> None:
+    """Disable Redis after a runtime failure and fall back to local cache."""
+    global REDIS_ENABLED, REDIS_CLIENT
+
+    if REDIS_ENABLED:
+        warning(
+            logger,
+            "Redis cache operation failed; falling back to local TTL cache",
+            operation=operation,
+            error=str(exc),
+        )
+
+    REDIS_ENABLED = False
+    REDIS_CLIENT = None
+
 # Check if Redis is available and import if it is
 REDIS_ENABLED = False
 REDIS_CLIENT = None
+redis_url = os.environ.get('REDIS_URL')
+
+if redis_url:
+    try:
+        import redis
+        from redis.exceptions import RedisError
 
-try:
-    import redis
-    from redis.exceptions import RedisError
-
-    # Get Redis URL from environment (Fly.io provides this for Redis instances)
-    redis_url = os.environ.get('REDIS_URL')
-
-    if redis_url:
         REDIS_CLIENT = redis.from_url(redis_url)
         # Test the connection
         REDIS_CLIENT.ping()
         REDIS_ENABLED = True
-except (ImportError, RedisError, Exception):
-    # Either redis is not installed or connection failed
-    # We'll fall back to local caching
-    pass
+        info(
+            logger,
+            "Redis cache enabled",
+            cache_backend="redis",
+            redis_url=_redact_redis_url(redis_url),
+        )
+    except ImportError as exc:
+        warning(
+            logger,
+            "Redis client import failed; using local TTL cache",
+            cache_backend="local_ttl",
+            redis_url=_redact_redis_url(redis_url),
+            error=str(exc),
+        )
+    except RedisError as exc:
+        warning(
+            logger,
+            "Redis connection failed during startup; using local TTL cache",
+            cache_backend="local_ttl",
+            redis_url=_redact_redis_url(redis_url),
+            error=str(exc),
+        )
+    except Exception as exc:
+        warning(
+            logger,
+            "Unexpected Redis startup failure; using local TTL cache",
+            cache_backend="local_ttl",
+            redis_url=_redact_redis_url(redis_url),
+            error=str(exc),
+        )
+else:
+    info(
+        logger,
+        "Redis cache disabled; REDIS_URL not configured, using local TTL cache",
+        cache_backend="local_ttl",
+        local_ttl_seconds=LOCAL_CACHE_TTL_SECONDS,
+    )
 
 # Fallback local cache with 10 minute TTL
-local_cache = TTLCache(maxsize=1000, ttl=600)
+local_cache = TTLCache(maxsize=LOCAL_CACHE_MAXSIZE, ttl=LOCAL_CACHE_TTL_SECONDS)
 
 def cache_key(*args, **kwargs) -> str:
     """Generate a consistent cache key from arguments."""
@@ -54,9 +122,8 @@ def get_cached(key: str, default: Any = None) -> Any:
             if value:
                 return json.loads(value)
             return default
-        except Exception:
-            # Fall back to local cache if Redis fails
-            pass
+        except Exception as exc:
+            _disable_redis("get", exc)
 
     return local_cache.get(key, default)
 
@@ -80,9 +147,8 @@ def set_cached(key: str, value: Any, ttl: int = 600) -> bool:
             try:
                 REDIS_CLIENT.setex(key, ttl, json_value)
                 return True
-            except Exception:
-                # Fall back to local cache if Redis fails
-                pass
+            except Exception as exc:
+                _disable_redis("set", exc)
 
         # Use local cache
         local_cache[key] = value
@@ -104,8 +170,8 @@ def delete_cached(key: str) -> bool:
         if REDIS_ENABLED:
             try:
                 REDIS_CLIENT.delete(key)
-            except Exception:
-                pass
+            except Exception as exc:
+                _disable_redis("delete", exc)
 
         # Also remove from local cache
         if key in local_cache:
@@ -131,8 +197,8 @@ def clear_pattern(pattern: str) -> bool:
                 keys = REDIS_CLIENT.keys(pattern)
                 if keys:
                     REDIS_CLIENT.delete(*keys)
-            except Exception:
-                pass
+            except Exception as exc:
+                _disable_redis("clear_pattern", exc)
 
         # For local cache, scan and remove matching keys
         keys_to_delete = [k for k in local_cache if k.startswith(pattern)]

common/utils/validators.py

@@ -376,11 +376,11 @@ def validate_meals(meals):
 
 MAX_EVENT_PHOTOS = 100
 MAX_SOCIAL_POSTS = 25
-ALLOWED_SOCIAL_PLATFORMS = {"linkedin", "instagram", "threads"}
+ALLOWED_SOCIAL_PLATFORMS = {"linkedin", "instagram", "threads", "article"}  # article = freeform URL with no platform-host restrictions,
 SOCIAL_PLATFORM_HOSTS = {
     "linkedin": ("linkedin.com",),
     "instagram": ("instagram.com",),
-    "threads": ("threads.net", "threads.com"),
+    "threads": ("threads.net", "threads.com"),    
 }
 
 
@@ -424,11 +424,13 @@ def validate_social_posts(posts):
             raise ValueError(f"social_posts[{i}].url must be a valid URL")
         host = (urlparse(url).netloc or "").lower()
         host = host[4:] if host.startswith("www.") else host
-        allowed_hosts = SOCIAL_PLATFORM_HOSTS[platform]
-        if not any(host == h or host.endswith("." + h) for h in allowed_hosts):
-            raise ValueError(
-                f"social_posts[{i}].url host must match platform '{platform}' ({allowed_hosts})"
-            )
+        # For non-article platforms, enforce that the URL's host matches the expected platform hosts.
+        if platform != "article":
+            allowed_hosts = SOCIAL_PLATFORM_HOSTS.get(platform, ())
+            if not any(host == h or host.endswith("." + h) for h in allowed_hosts):
+                raise ValueError(
+                    f"social_posts[{i}].url host must match platform '{platform}' ({allowed_hosts})"
+                )
         caption = post.get("caption")
         if caption is not None and not isinstance(caption, str):
             raise ValueError(f"social_posts[{i}].caption must be a string")