You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Running github-to-sqlite starred fetches the user's entire star list on every run, even when most or all of those stars are already in the database. This makes repeated syncs slow and uses unnecessary GitHub API requests for users with large star lists.
Describe the feature
Add an opt-in incremental mode, such as --incremental, that uses the database's existing stars for the specified user to avoid fetching the full history.
On an empty database, incremental mode should do a full initial import. On later runs, it should fetch recent stars and stop once it has reached stars already stored in the database. Existing full-fetch behavior should remain available.
Additional context
GitHub's starred-repositories endpoint supports sorting by when a repository was starred, but does not provide a since cursor. Incremental mode would therefore need to paginate through recent results and use the database to determine when it has caught up. It should account for multiple stars sharing a timestamp so that it doesn't skip any.
Activity