Skip to content

192 - Return errors from scrapers instead of panicking - #203

Open
Tyson-TVN wants to merge 5 commits into
developfrom
192-replace-panic-calls-with-returned-errors-in-scrapers
Open

Tyson-TVN wants to merge 5 commits into
developfrom
192-replace-panic-calls-with-returned-errors-in-scrapers

Conversation

@Tyson-TVN

Copy link
Copy Markdown

Fixes #192

Scrapers previously called panic()/log.Fatal()/log.Panic() on failure, which crashed the whole process and threw away progress on long-running scrapes (astra.go's full-year date sweep, coursebook.go's per-prefix walk). Each Scrape* function and the helpers they call now return an error instead, propagated up through main, which exits non-zero on failure without a panic.

Includes and builds on commits from @ThatRell (budgets.go, cometCalendar.go, coursebook.go) already pushed to this branch, rebased together with the rest of the scrapers/utils changes.

Test plan

  • go build ./... and go vet ./... pass
  • Added scrapers/errors_test.go verifying each Scrape* function returns a filesystem error instead of panicking when given a blocked output directory

ThatRell and others added 4 commits September 26, 2026 23:50
Scrapers previously called panic/log.Fatal/log.Panic on failure,
which crashed the whole process and threw away progress on
long-running scrapes (astra, coursebook). Scraper functions and the
helpers they call (RefreshToken, RefreshAstraToken,
GetCoursePrefixes, ExtractTextAndHref) now return errors instead,
propagated up through main, which exits non-zero on failure.

Fixes #192

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@Tyson-TVN
Tyson-TVN requested review from a team as code owners September 27, 2026 09:22
@Tyson-TVN Tyson-TVN linked an issue Sep 27, 2026 that may be closed by this pull request
CI's staticcheck step (ST1005) requires error strings to be
lowercase and not end with punctuation. Fixes the 6 flagged spots
in astra.go and coursebook.go from the panic-to-error refactor.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace panic calls with returned errors in scrapers

3 participants