13332 components neetokb #19223

luancazarine · 2025-11-25T17:10:46Z

Resolves #13332

Summary by CodeRabbit

New Features
- Added a "New Published Article" source that monitors the API for newly published articles and emits events with article details
- Implemented pagination support for article fetching with configurable result limits
Chores
- Version bumped to 0.1.0
- Added platform dependency

_{✏️ Tip: You can customize this high-level summary in your review settings.}

…es and pagination; add new source for detecting published articles; update package version to 0.1.0 and include dependencies.

vercel · 2025-11-25T17:10:52Z

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments

Project	Deployment	Preview	Comments	Updated (UTC)
pipedream-docs	Ignored			Nov 25, 2025 5:17pm
pipedream-docs-redirect-do-not-edit	Ignored			Nov 25, 2025 5:17pm

coderabbitai · 2025-11-25T17:11:10Z

Warning

Rate limit exceeded

@luancazarine has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 10 minutes and 49 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between cb360bf and d7890a3.

⛔ Files ignored due to path filters (1)

pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml

📒 Files selected for processing (16)

components/apiframe/apiframe.app.mjs (1 hunks)
components/brand_dev/brand_dev.app.mjs (1 hunks)
components/contentdrips/contentdrips.app.mjs (1 hunks)
components/cursor/cursor.app.mjs (1 hunks)
components/emailchef/emailchef.app.mjs (1 hunks)
components/fynk/fynk.app.mjs (1 hunks)
components/helicone/helicone.app.mjs (1 hunks)
components/iplocate/iplocate.app.mjs (1 hunks)
components/magicalapi/magicalapi.app.mjs (1 hunks)
components/neetokb/sources/new-published-article/new-published-article.mjs (1 hunks)
components/relationcity/relationcity.app.mjs (1 hunks)
components/shortpen/shortpen.app.mjs (1 hunks)
components/socket/socket.app.mjs (1 hunks)
components/turbosmtp/turbosmtp.app.mjs (1 hunks)
components/validemail/validemail.app.mjs (1 hunks)
components/veryfi/veryfi.app.mjs (1 hunks)

Walkthrough

The changes introduce a new Neetokb component for Pipedream. A shared app module replaces deprecated authentication with private API utilities for requests, article listing, and pagination support. A new polling source fetches published articles at configurable intervals, filters for new items using persistent state, and emits events for each new article discovered.

Changes

Cohort / File(s)	Summary
API Constants `components/neetokb/common/constants.mjs`	New module exporting `LIMIT` constant set to 100 for pagination.
App Module `components/neetokb/neetokb.app.mjs`	Removed deprecated `authKeys()` method. Added private helpers: `_apiUrl()`, `_getHeaders()`, `_makeRequest()` for standardized API communication. Added `listArticles(args)` for fetching articles and `paginate()` async generator for page-by-page iteration using LIMIT and total/current page tracking.
Package Configuration `components/neetokb/package.json`	Version bumped from 0.0.2 to 0.1.0. Added dependency: "@pipedream/platform": "^3.1.1".
Polling Source `components/neetokb/sources/new-published-article/new-published-article.mjs`	New source module polling Neetokb API for published articles. Maintains persistent store of published article IDs, filters new articles, emits events in reverse chronological order, and bootstraps state with up to 25 articles on deploy. Includes state management methods `_getPublishedIds()` and `_setPublishedIds()`, event emission logic, and deploy hook.
Test Event `components/neetokb/sources/new-published-article/test-event.mjs`	Sample event object representing a published article with id, slug, title, state, view count, archive status, and category metadata.

Sequence Diagram(s)

sequenceDiagram
    participant Timer as Polling Timer
    participant Source as new-published-article
    participant App as neetokb App
    participant API as Neetokb API
    participant DB as Persistent Store
    
    Timer->>Source: trigger emitEvent()
    Source->>DB: _getPublishedIds()
    DB-->>Source: stored IDs
    Source->>App: paginate({fn: listArticles})
    loop For each page
        App->>API: _makeRequest(path: articles)
        API-->>App: articles + pagination data
        App-->>Source: yield article items
    end
    Source->>Source: filter articles not in stored IDs
    Source->>Source: emit event for each new article
    Source->>DB: _setPublishedIds(updated set)
    DB-->>Source: ack

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Pagination logic in neetokb.app.mjs: The paginate() async generator implements conditional continuation logic based on total_pages, current_page_number, and maxResults. Verify state updates (page_number, page_size) are correctly applied across iterations.
State management in source: Confirm _getPublishedIds() and _setPublishedIds() correctly serialize/deserialize the persistent store and that filtering logic (new Set) avoids duplicate emissions across subsequent polls.
Event emission order: The source emits in "chronological reverse order"—verify the sorting/ordering matches expected behavior and that the sampleEmit aligns with actual emitted structure.

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings, 1 inconclusive)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description is minimal and only contains an issue reference without explaining the changes or following the template structure.	Provide a detailed description following the template structure, including WHY section explaining the motivation and rationale for these changes.
Linked Issues check	⚠️ Warning	The PR implements most core requirements but has a significant gap: the new-published-article source lacks required and optional article/author filtering properties specified in issue #13332.	Add required 'articleID' and optional 'authorID' props to the new-published-article source to meet the specified requirements in issue #13332.
Title check	❓ Inconclusive	The title '13332 components neetokb' is vague and appears to be an issue reference format rather than a descriptive summary of changes.	Use a more descriptive title that summarizes the main change, such as 'Add neetokb component with polling source for published articles'.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Out of Scope Changes check	✅ Passed	All changes are within scope: constants module, updated app.mjs with API utilities, and new polling source align with creating a neetokb component set.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 3

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e26745a and cb360bf.

⛔ Files ignored due to path filters (1)

pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml

📒 Files selected for processing (5)

components/neetokb/common/constants.mjs (1 hunks)
components/neetokb/neetokb.app.mjs (1 hunks)
components/neetokb/package.json (2 hunks)
components/neetokb/sources/new-published-article/new-published-article.mjs (1 hunks)
components/neetokb/sources/new-published-article/test-event.mjs (1 hunks)

🧰 Additional context used

🧠 Learnings (3)

📚 Learning: 2024-12-12T19:23:09.039Z

Learnt from: jcortes
Repo: PipedreamHQ/pipedream PR: 14935
File: components/sailpoint/package.json:15-18
Timestamp: 2024-12-12T19:23:09.039Z
Learning: When developing Pipedream components, do not add built-in Node.js modules like `fs` to `package.json` dependencies, as they are native modules provided by the Node.js runtime.

Applied to files:

components/neetokb/package.json

📚 Learning: 2025-09-15T22:01:11.472Z

Learnt from: GTFalcao
Repo: PipedreamHQ/pipedream PR: 18362
File: components/leonardo_ai/actions/generate-image/generate-image.mjs:103-105
Timestamp: 2025-09-15T22:01:11.472Z
Learning: In Pipedream components, pipedream/platform's axios implementation automatically excludes undefined values from HTTP requests, so there's no need to manually check for truthiness before including properties in request payloads.

Applied to files:

components/neetokb/neetokb.app.mjs

📚 Learning: 2024-10-10T19:18:27.998Z

Learnt from: GTFalcao
Repo: PipedreamHQ/pipedream PR: 14265
File: components/the_magic_drip/sources/common.mjs:35-43
Timestamp: 2024-10-10T19:18:27.998Z
Learning: In `components/the_magic_drip/sources/common.mjs`, when processing items in `getAndProcessData`, `savedIds` is intentionally updated with IDs of both emitted and non-emitted items to avoid emitting retroactive events upon first deployment and ensure only new events are emitted as they occur.

Applied to files:

components/neetokb/sources/new-published-article/new-published-article.mjs

🧬 Code graph analysis (1)

components/neetokb/neetokb.app.mjs (1)

components/neetokb/common/constants.mjs (2)

LIMIT (1-1)

LIMIT (1-1)

🔇 Additional comments (4)

components/neetokb/package.json (1)

3-17: LGTM!

Version bump to 0.1.0 appropriately reflects the new functionality, and the @pipedream/platform dependency is correctly added to support the axios and timer utilities used in the new source.

components/neetokb/sources/new-published-article/test-event.mjs (1)

1-12: LGTM!

The sample event structure is appropriate for testing the new published article source.

components/neetokb/common/constants.mjs (1)

1-1: LGTM!

Clean extraction of the pagination limit constant.

components/neetokb/neetokb.app.mjs (1)

1-65: LGTM!

The API utility layer and pagination implementation are well-structured. The async generator correctly handles page iteration and early termination via maxResults.

components/neetokb/sources/new-published-article/new-published-article.mjs

…nsistency

lcaresia

LGTM!

vunguyenhung · 2025-11-27T11:37:25Z

For Integration QA:

Source Test Report: https://www.notion.so/Components-neetokb-Sources-2b8bf548bb5e81839a73c01585847530

vunguyenhung · 2025-11-28T07:39:46Z

Hi everyone, all test cases are passed! Ready for release!

Test reports

[Components] neetokb - Sources: https://www.notion.so/Components-neetokb-Sources-2b8bf548bb5e81839a73c01585847530

luancazarine added 2 commits November 25, 2025 12:07

Implement neetokb API integration with new methods for listing articl…

9388198

…es and pagination; add new source for detecting published articles; update package version to 0.1.0 and include dependencies.

pnpm update

cb360bf

luancazarine linked an issue Nov 25, 2025 that may be closed by this pull request

[Components] neetokb #13332

Closed

Merge branch 'master' into 13332-components-neetokb

fec39a5

coderabbitai bot reviewed Nov 25, 2025

View reviewed changes

Update key for new published article source to improve clarity and co…

d7890a3

…nsistency

pipedream-component-development requested a review from lcaresia November 25, 2025 17:40

lcaresia approved these changes Nov 27, 2025

View reviewed changes

luancazarine merged commit 3f5b8bc into master Nov 28, 2025
10 checks passed

luancazarine deleted the 13332-components-neetokb branch November 28, 2025 21:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

13332 components neetokb #19223

13332 components neetokb #19223

Uh oh!

luancazarine commented Nov 25, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

vercel bot commented Nov 25, 2025 •

edited

Loading

Uh oh!

coderabbitai bot commented Nov 25, 2025 •

edited

Loading

Rate limit exceeded

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lcaresia left a comment

Uh oh!

vunguyenhung commented Nov 27, 2025

Uh oh!

vunguyenhung commented Nov 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

13332 components neetokb #19223

13332 components neetokb #19223

Uh oh!

Conversation

luancazarine commented Nov 25, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

vercel bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rate limit exceeded

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Pre-merge checks and finishing touches

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lcaresia left a comment

Choose a reason for hiding this comment

Uh oh!

vunguyenhung commented Nov 27, 2025

For Integration QA:

Uh oh!

vunguyenhung commented Nov 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

luancazarine commented Nov 25, 2025 •

edited by coderabbitai bot

Loading

vercel bot commented Nov 25, 2025 •

edited

Loading

coderabbitai bot commented Nov 25, 2025 •

edited

Loading