Repository navigation
fix: preserve original GTFS source in reports - #2178
mackenziereading19 wants to merge 3 commits into
Conversation
|
Quick CI note: the current Web service CI failure occurs at google-github-actions/auth@v2, before the Gradle tasks run. The other workflows are passing, and the full :web:service:test suite passed locally. I don’t currently see a code-related failure in this PR, but I’m happy to rerun or make changes if maintainers spot anything implementation-specific. |
|
|
||
| /** @param {string=} url **/ | ||
| function createJob(url) { | ||
| /** @param {string=} url @param {string=} filename **/ |
There was a problem hiding this comment.
Nitpick: The empty line should not be removed.
| if (!Strings.isNullOrEmpty(body.getCountryCode())) { | ||
| storageHelper.saveJobMetadata(new JobMetadata(jobId, body.getCountryCode())); | ||
| String originalGtfsSource = | ||
| !Strings.isNullOrEmpty(body.getUrl()) ? body.getUrl() : body.getFilename(); |
There was a problem hiding this comment.
We should sanitize the URL that we're going to put in the report to remove any possible user and password.
|
Thanks @jcpitre, I've addressed both comments in
All 26 web-service tests and Spotless pass locally with Java 17. Fresh CI is running now. |
|
Thanks for the change, but there is still a problem that I should have pointed out initially. Another point is that the sanitizing should not throw an exception that would result in the whole download operation failing. If there's any problem with sanitizing we should just revert to the URL as it was displayed before (even if we agree it's not that useful) |
|
Thanks @jcpitre, I’ve addressed the follow-up in e3b7712.
|
|
Thanks for the change.
In both these cases if we return the unsanitized url the secret would be revealed. It should return null. The caller can handle a null originalGtfsSource. |
Summary
Fixes #1539 by preserving the human-readable GTFS source separately from the operational source used by the validator.
For web uploads, reports now show the original uploaded filename instead of the temporary server-side filename. For URL submissions, reports show the original URL, as requested in the issue discussion.
Implementation
originalGtfsSourceprovenance toValidationRunnerConfigwhile retaininggtfsSourcefor operational file/URL access.originalGtfsSourcewhen present, otherwisegtfsSource) used by JSON and HTML reports.CreateJobRequestand persistedJobMetadata.ValidationControllerandValidationHandlerwithout changing the operational temporary file URI.JobMetadatathat does not containoriginalGtfsSource.Validation
:main:testpassed.:web:service:testpassed.:main:spotlessCheckpassed.:web:service:spotlessCheckpassed.npm run checkpassed with 0 errors and 0 warnings (3 pre-existing hints)./create-job.gtfsSourceremains separate fromoriginalGtfsSource.JobMetadataJSON still deserializes withoriginalGtfsSource == null.git diff --checkpassed.Closes #1539