chore: retire legacy analytics generators (#4910, #4911) - #4925
Conversation
There was a problem hiding this comment.
Pull request overview
Retires the legacy analytics report generators (Jupyter Book/PDF and Google Sheets notebooks) and standardizes the repo on the static-site analytics generator scripts, including some directory reshaping and updated deployment plumbing.
Changes:
- Deleted Jupyter Book report directories/files and legacy Sheets-report notebooks/readmes across analytics subfolders.
- Added/updated per-site static-site generator scripts and moved historic UA data JSONs into the new/non-
-sheetslocations. - Updated analytics docs and the deployment script to target the new static-site directory structure.
Reviewed changes
Copilot reviewed 34 out of 49 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| analytics/readme.md | Updates analytics docs to describe running static-site generators (and removes notebook kernel step). |
| analytics/lungmap/users_over_time_history.json | Adds LungMAP historic UA data in the new location. |
| analytics/lungmap/generate_static_site.py | Updates LungMAP static-site generator script (notably paths). |
| analytics/lungmap/constants.py | Adds LungMAP static-site configuration constants. |
| analytics/lungmap-analytics/sheets/readme.md | Removes legacy Sheets notebook instructions for LungMAP. |
| analytics/lungmap-analytics/sheets/generate_sheets_report.ipynb | Deletes LungMAP legacy Sheets report notebook. |
| analytics/lungmap-analytics/sheets/constants.py | Deletes LungMAP legacy Sheets-only constants. |
| analytics/lungmap-analytics/readme.md | Deletes LungMAP Jupyter Book report instructions. |
| analytics/lungmap-analytics/analytics_lungmap.py | Deletes LungMAP legacy Jupyter Book/python generator code. |
| analytics/lungmap-analytics/_toc.yml | Deletes LungMAP Jupyter Book TOC. |
| analytics/lungmap-analytics/_static/analytics-export.css | Deletes LungMAP Jupyter Book export CSS. |
| analytics/lungmap-analytics/_config.yml | Deletes LungMAP Jupyter Book config. |
| analytics/hca-explorer/generate_static_site.py | Adds HCA Explorer static-site generator script in the renamed folder. |
| analytics/hca-explorer/constants.py | Removes Sheets-only constants from HCA Explorer configuration. |
| analytics/hca-explorer-sheets/readme.md | Removes legacy Sheets notebook instructions for HCA Explorer. |
| analytics/hca-explorer-sheets/generate_sheets_report.ipynb | Deletes HCA Explorer legacy Sheets report notebook. |
| analytics/hca-analytics/users_over_time_history.json | Deletes HCA legacy Jupyter Book historic UA data file along with the retired directory. |
| analytics/hca-analytics/readme.md | Deletes HCA Jupyter Book report instructions. |
| analytics/hca-analytics/analytics_hca.py | Deletes HCA legacy Jupyter Book/python generator code. |
| analytics/hca-analytics/analytics_hca_bq.py | Deletes HCA legacy BigQuery/matplotlib reporting code. |
| analytics/hca-analytics/_toc.yml | Deletes HCA Jupyter Book TOC. |
| analytics/hca-analytics/_static/analytics-export.css | Deletes HCA Jupyter Book export CSS. |
| analytics/hca-analytics/_config.yml | Deletes HCA Jupyter Book config. |
| analytics/example/users_over_time_history.json | Deletes example legacy UA history JSON. |
| analytics/example/readme.md | Deletes example legacy Sheets/notebook documentation. |
| analytics/example/generate_sheets_report.ipynb | Deletes example legacy Sheets report notebook. |
| analytics/example/constants.py | Deletes example legacy Sheets constants. |
| analytics/deploy_sites.sh | Updates deploy script to copy built static sites from the renamed per-site folders. |
| analytics/anvil-explorer/utils.py | Adds AnVIL Explorer helper to fetch dataset titles for enrichment. |
| analytics/anvil-explorer/generate_static_site.py | Adds AnVIL Explorer static-site generator script in the renamed folder. |
| analytics/anvil-explorer/constants.py | Removes Sheets-only constants from AnVIL Explorer configuration. |
| analytics/anvil-explorer-sheets/utils.py | Deletes legacy Sheets utilities (superseded by renamed/static-site flow). |
| analytics/anvil-explorer-sheets/readme.md | Removes legacy Sheets notebook instructions for AnVIL Explorer. |
| analytics/anvil-explorer-sheets/generate_sheets_report.ipynb | Deletes AnVIL Explorer legacy Sheets report notebook. |
| analytics/anvil-catalog/users_over_time_history.json | Adds AnVIL Catalog historic UA data in the new location. |
| analytics/anvil-catalog/generate_static_site.py | Adds AnVIL Catalog static-site generator script in the renamed folder. |
| analytics/anvil-catalog/constants.py | Removes Sheets-only constants from AnVIL Catalog configuration. |
| analytics/anvil-catalog-sheets/readme.md | Removes legacy Sheets notebook instructions for AnVIL Catalog. |
| analytics/anvil-catalog-sheets/generate_sheets_report.ipynb | Deletes AnVIL Catalog legacy Sheets report notebook. |
| .gitignore | Drops ignore rule for analytics _build outputs (legacy Jupyter Book artifact). |
Suppressed comments (1)
analytics/lungmap/generate_static_site.py:61
deploy_sites.shexpects generated output underanalytics/lungmap/site/, butoutput_dir="../../gh-pages/lungmap"writes elsewhere, so the deploy script will deploy an empty/stale site.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
@hunterckx thx ! See the suppressed comment above by copilot. It looks real. |
|
@NoopDog See my earlier comment -- the deploy script is out-of-scope I think (but perhaps worth a ticket to address it) |
NoopDog
left a comment
There was a problem hiding this comment.
@hunterckx thx — here's the review. Two inline comments below, plus two notes on files outside the diff hunks:
analytics/analytics_package/analytics/static_site/README.md — still references the pre-rename folder names (anvil-explorer-sheets, anvil-catalog-sheets, hca-explorer-sheets, lungmap-analytics/sheets) in the cd instructions (line 25) and the per-app script table (lines 59–62); anyone following the doc gets "no such file or directory". Line 29's claim that the script "writes the site to ./site/" is also wrong for the three explorer scripts, which write to ../../gh-pages/<app>.
analytics/anvil-catalog/generate_static_site.py — anvil-catalog is the odd one out: it writes to a gitignored ./site/ (via /analytics/**/site/) and appears in neither deploy_sites.sh nor gh-pages/index.html, yet the new readme section documents all four folders identically — the generated report silently goes nowhere. Pre-existing asymmetry, but if it's intentional (internal-only report) it deserves a note in the readme.
…now-unused `utils.py` (#4911)
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 35 out of 49 changed files in this pull request and generated no new comments.
Suppressed comments (1)
analytics/readme.md:34
- The local viewing instructions only work for sites generated into
gh-pages/and are incorrect for theanvil-cataloggenerator (which writes toanvil-catalog/site). This can confuse someone who just generated the catalog site and then follows the../../gh-pagescommand.
NoopDog
left a comment
There was a problem hiding this comment.
Re-reviewed the updated branch — all review feedback addressed: deploy_sites.sh removed in favor of a committed gh-pages/index.html, the duplicate fetch logic replaced with the shared fetch_entity_title_map helper (with timeout), docs updated and the anvil-catalog exception noted. Remaining minor items (requirements.txt cleanup) are tracked in #4912. LGTM — thanks @hunterckx!
Closes #4910
Closes #4911
Notes:
requirements.txt; I think that's best saved for chore: slim analytics package to the static-site core #4912