8000 feat: [BE] Add pagination to published components endpoint · Issue #179 · TangleML/tangle · GitHub
[go: up one dir, main page]

Skip to content

feat: [BE] Add pagination to published components endpoint #179

Open

Description

@Mbeaulne

Description

The GET /api/published_components/ endpoint currently returns all components with no pagination. There are existing TODO comments in the code acknowledging this:

# TODO: Implement paging
# TODO: Implement filtering/search
# TODO: Implement visibility/access control

As the component library grows and a Components tab is added to the homepage, this needs pagination support.

Part of TangleML/tangle-ui#1963

Implementation

Follow the same pagination pattern used by GET /api/pipeline_runs/:

  • Add page_size query param (default 20)
  • Add page_token query param for cursor-based pagination (base64-encoded offset)
  • Add next_page_token to response
  • Add total_count to response (component count is much smaller than runs, so a full count is fine here)

Response change

@dataclasses.dataclass(kw_only=True)
class ListPublishedComponentsResponse:
    published_components: list[PublishedComponentResponse]
    next_page_token: str | None = None  # NEW
    total_count: int | None = None      # NEW

Relevant file

  • component_library_api_server.py — list() method

Activity

  1. changed the title [-]feat: Add pagination to published components endpoint[/-] [+]feat: [BE] Add pagination to published components endpoint[/+] on Mar 19, 2026
  2. yuechao-qin commented on Mar 19, 2026

    @yuechao-qin
    Collaborator

    Just trying to get a sense of priority / urgency:

    • how many components are being returned now without pagination?
    • do we expect the growth rate to be N times in the upcoming weeks?
    • can the UI mitigate this in other ways while waiting for this feature?
  3. Ark-kun commented on Mar 19, 2026

    @Ark-kun
    Contributor

    this needs pagination support

    Agreed.

    Add total_count to response (component count is much smaller than runs, so a full count is fine here)

    I'm a bit skeptical here.

  4. Mbeaulne commented on Mar 23, 2026

    @Mbeaulne
    CollaboratorAuthor

    Good questions — here's where things stand:

    • How many components are being returned now? Under 100 currently.
    • Do we expect N times growth in the upcoming weeks? Honestly not sure — it's possible. Pinging @Ark-kun for his take on expected growth.
    • Can the UI mitigate this in the meantime? Likely yes — we could implement front-end pagination over the full returned list as a stopgap. That said, if the component count does grow significantly it won't scale well, which is part of the motivation for doing this properly on the backend.
  5. Mbeaulne commented on Mar 23, 2026

    @Mbeaulne
    CollaboratorAuthor

    Curious what's driving the skepticism — is it the assumption that component counts will stay small enough to make a full total_count cheap? How many components are we realistically expecting over time?

  6. yuechao-qin commented on Mar 25, 2026

    @yuechao-qin
    Collaborator

    @Mbeaulne i think this is a good candidate for my goal of creating an experimental API in oasis-backend. I will need to 1) create a design doc and review it with the team and then 2) implement it. Curious how urgent are you wanting this feature. My thought is might have an release by mid April if everything goes smoothly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0