549: renable refetch questionservice and add back to job queue

[alfardil]

549

Description of changes

In this PR, I re-enabled the RefetchIncompleteQuestionsService to check every 24 hours for any questions with incomplete fields + a job with a status of INCOMPLETE or PROCESSING.

Checklist before review

• I have done a thorough self-review of the PR
• Copilot has reviewed my latest changes, and all comments have been fixed and/or closed.
• If I have made database changes, I have made sure I followed all the db repo rules listed in the wiki here. (check if no db changes)
• All tests have passed
• I have successfully deployed this PR to staging
• I have done manual QA in both dev (and staging if possible) and attached screenshots below.

Screenshots

Dev

The images show the incomplete question (with id starting with 4f) being found by our scheduled task and then being added to the job queue where it is then processed.

Difficult to test in staging.

Staging

[alfardil]

/deploy

[github-actions]

The command to deploy to staging for the commit d5c33e3 has been triggered. View action run

[greptile-apps]

Greptile Summary

Refactored the incomplete question refetch mechanism to use a job queue pattern instead of directly fetching from LeetCode API

Key Changes:

• Replaced RefetchIncompleteQuestions class with RefetchIncompleteQuestionsService
• Changed from direct API calls to creating INCOMPLETE jobs that get processed by existing LeetcodeQuestionProcessService
• Added new repository method getAllIncompleteQuestionsWithNoJob() to prevent duplicate job creation
• Scheduled task now runs every 24 hours (using @Scheduled annotation) instead of being manually triggered
• Jobs are created with nextAttemptAt set to 1 hour ago to ensure immediate processing

Architecture Improvement:
This change properly separates concerns by having the scheduled task only identify incomplete questions and create jobs, while the existing job processor (LeetcodeQuestionProcessService) handles the actual LeetCode API calls with proper rate limiting and error handling.

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• The refactoring follows good architectural patterns by leveraging the existing job queue infrastructure. The SQL query properly prevents duplicate job creation by filtering out questions that already have INCOMPLETE or PROCESSING jobs. The change is well-contained, removes code duplication, and reuses the existing rate-limited LeetcodeQuestionProcessService.
• No files require special attention

Important Files Changed

Filename	Overview
src/main/java/com/patina/codebloom/scheduled/leetcode/RefetchIncompleteQuestionsService.java	Refactored scheduled service to create jobs instead of directly fetching question data, runs every 24 hours
src/main/java/com/patina/codebloom/common/db/repos/question/QuestionRepository.java	Added repository method to find incomplete questions without active jobs
src/main/java/com/patina/codebloom/common/db/repos/question/QuestionSqlRepository.java	Implemented SQL query to filter incomplete questions excluding those with INCOMPLETE or PROCESSING jobs

Sequence Diagram

sequenceDiagram
    participant Scheduler as Spring Scheduler
    participant RIQS as RefetchIncompleteQuestionsService
    participant QR as QuestionRepository
    participant JR as JobRepository
    participant DB as Database
    participant LQPS as LeetcodeQuestionProcessService
    participant LC as LeetcodeClient

    Note over Scheduler,RIQS: Every 24 hours
    Scheduler->>RIQS: @Scheduled trigger reFetchIncompleteQuestions()
    RIQS->>QR: getAllIncompleteQuestionsWithNoJob()
    QR->>DB: SELECT questions WHERE fields NULL AND no INCOMPLETE/PROCESSING job
    DB-->>QR: Return incomplete questions
    QR-->>RIQS: Return ArrayList<Question>
    
    alt Questions found
        loop For each incomplete question
            RIQS->>RIQS: Log warning about incomplete question
            RIQS->>JR: createJob(Job with INCOMPLETE status)
            JR->>DB: INSERT into Job table
            DB-->>JR: Return created job with timestamps
            JR-->>RIQS: Job created
            RIQS->>RIQS: Log job creation
        end
    else No questions
        RIQS->>RIQS: Log "All questions complete"
    end

    Note over Scheduler,LQPS: Every 30 minutes (separate process)
    Scheduler->>LQPS: @Scheduled trigger drainQueue()
    LQPS->>JR: findIncompleteJobs(maxSize)
    JR->>DB: SELECT incomplete jobs ordered by nextAttemptAt
    DB-->>JR: Return jobs
    JR-->>LQPS: Return List<Job>
    
    loop For each job
        LQPS->>JR: updateJob(status=PROCESSING)
        JR->>DB: UPDATE Job status
        LQPS->>QR: getQuestionById(questionId)
        QR->>DB: SELECT question
        DB-->>QR: Return question
        QR-->>LQPS: Return Question
        LQPS->>LC: findSubmissionDetailBySubmissionId()
        LC-->>LQPS: Return submission details
        LQPS->>QR: updateQuestion(with fetched data)
        QR->>DB: UPDATE Question
        LQPS->>JR: updateJob(status=COMPLETE)
        JR->>DB: UPDATE Job status
    end

Loading

[greptile-apps]

Greptile's behavior is changing!

From now on, if a review finishes with no comments, we will not post an additional "statistics" comment to confirm that our review found nothing to comment on. However, you can confirm that we reviewed your changes in the status check section.

_{This feature can be toggled off in your Code Review Settings by deselecting "Create a status check for each PR".}

[github-actions]

Overall Project	74.01% -0.41%	🍏
Files changed	0%	❌

File	Coverage
QuestionSqlRepository.java	86.95% -2.65%	❌
RefetchIncompleteQuestionsService.java	0%	❌

[github-actions]

Staging deployment failed for commit d5c33e3

**

View run

[github-actions]

Overall Project	74.6% -0.41%	🍏
Files changed	0%	❌

File	Coverage
QuestionSqlRepository.java	86.34% -2.72%	❌
RefetchIncompleteQuestionsService.java	0%	❌

[alfardil]

/deploy

[github-actions]

The command to deploy to staging for the commit 17726a5 has been triggered. View action run

[github-actions]

Staging deployment failed for commit 17726a5

**

View run

[github-actions]

Overall Project	76.79% -0.03%	🍏
Files changed	92.93%	🍏

File	Coverage
RefetchIncompleteQuestionsService.java	100%	🍏
QuestionSqlRepository.java	88.55% -0.51%	🍏

[github-actions]

Overall Project	76.76% -0.03%	🍏
Files changed	92.93%	🍏

File	Coverage
RefetchIncompleteQuestionsService.java	100%	🍏
QuestionSqlRepository.java	88.55% -0.51%	🍏

[github-actions]

Overall Project	77.12% -0.03%	🍏
Files changed	92.93%	🍏

File	Coverage
RefetchIncompleteQuestionsService.java	100%	🍏
QuestionSqlRepository.java	88.55% -0.51%	🍏