Skip to content

Update the sample database for OpenVDM 2.17, from the demo site (#22) - #23

Merged
webbpinner merged 3 commits into
devfrom
issue_22
Oct 28, 2026
Merged

webbpinner merged 3 commits into
devfrom
issue_22

Conversation

@webbpinner

@webbpinner webbpinner commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #22.

Brings openvdm_sample_data.sql up to date with the demo site (http://openvdm.oceandatatools.org), as it is on 2026-10-28. Values come from the demo's read-only API (getCollectionSystemTransfers, getExtraDirectories, getRequiredExtraDirectories, and the cruise data and ship-to-shore equivalents) and its exported cruise_config.json. The second commit re-syncs the first with changes made on the demo since.

  • description columns (Add a description field to collection system transfer and extra directory records openvdm#432): added to the OVDM_CollectionSystemTransfers and OVDM_ExtraDirectories tables and their INSERTs.
  • Collection system descriptions: all six, as the demo has them.
  • Extra directories: Dashboard_Data and From_PublicData have the current 2.17 default descriptions; Transfer_Logs and Tracklines have none. New: NAV_QA (id 5, Products/NAV_QA, enabled, not required, empty description).
  • Filters: CTD *{cruiseID}_CTD[0-9][0-9][0-9]_{mm}{DD}*, XBT *{cruiseID}_XBT[0-9][0-9][0-9]_{mm}{DD}*. Checked against the sample files with the placeholders expanded as the transfer does: the six ODT2601 XBT casts and two ODT2601 CTD casts are copied; XBT05 (Rename sample XBT cast 005 to XBT05, so it fails the transfer's filename filter #16), ODT2508_CTD001 and decktest aren't.
  • Source directories: XBT /XBT, Sealog /sealog, as the edit form saves them (its script adds the leading /). SMB transfers and their Test Setup strip it before joining it to the mount point (build_cst_filelist(), mount_path()), so the transfers read the same directories.
  • Cruise details: UPDATEs on OVDM_CoreVars set the cruise ID, name, description, PI, location, dates and ports to the demo's (ODT2601, Gulf of Mexico Exploration, 2012/04/28 00:00 – 2012/04/30 00:00, Pascagoula, MS → Galveston, TX). They update the rows openvdm_db.sql creates, so the other core variables are kept. The vessel and contact (Discoverer, in the exported config) aren't database values.
  • The cruise data transfers and ship-to-shore transfers already matched the demo.

Typos copied from the demo

The descriptions are as the demo has them, including EM302's "Mulitbeam". EM302's description also lists raw/*.kmall, which the sample data doesn't include.

Don't merge into master before OpenVDM 2.17 is released

The OpenVDM installer strips the CREATE TABLE blocks and only runs the rest of the file into its own schema, with errors hidden. On 2.16, which has no description column, the INSERTs fail: tested with the installer's exact sed rewrite, a 2.16.2 database gets ERROR 1054: Unknown column 'description' and no sample transfers. The 2.16 installer clones master by default, so this should go to master with the 2.17 release, as release 1.0.0 did for 2.16. Until then, testing 2.17 with the sample data means answering dev to the installer's "Sample data branch?".

install_sample_data.sh keeps the CREATE TABLE blocks, so it works on 2.16 too (the column is unused there).

Testing

In Docker, on a fresh 2.17 openvdm_db.sql (from the 2.17 branch):

  • mysql:8.0 and mariadb:10.11, this SQL as-is (the install_sample_data.sh path) and after OpenVDM 2.17's installer rewrite (utils/install-openvdm.sh): loads silently with 6 collection system transfers, 5 extra directories, 7 cruise data transfers and 6 ship-to-shore transfers. Every field matches the demo's API, apart from status and pid, and the cruise details match its cruise_config.json.
  • The installers' sed substitutions (/data/sample_data, survey, sample_smb_passwd) don't match anything in the descriptions or cruise details.

🤖 Generated with Claude Code

webbpinner and others added 3 commits October 11, 2026 14:10
- description columns on the collection system transfer and extra
  directory tables (OpenVDM #432), with the demo site's descriptions; the
  required Dashboard_Data and From_PublicData directories get OpenVDM's
  2.17 defaults
- CTD and XBT include filters check the cast date ({mm}{DD}, the sample
  files' MMDD)
- XBT and Sealog source directories start with "/", as the edit form
  saves them; SMB transfers strip it
- new CTD_Plots extra directory (CTD/Plots), for bin/plot_ctd_casts.py

Needs OpenVDM 2.17 when installed by the OpenVDM installer, which keeps
its own schema: 2.16 has no description column.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

The demo has changed since the first commit. Taken from its read-only API
and cruise_config.json, and checked against both:

- The six collection system transfer descriptions, as the demo has them
  now (ROV_OpenRVDAS's comes from the API; lowering_config.json doesn't
  export it).
- CTD includeFilter: *{cruiseID}_CTD[0-9][0-9][0-9]_{mm}{DD}* (no leading
  */). It still copies the two ODT2601 casts and skips ODT2508_CTD001 and
  decktest.
- Extra directories: Dashboard_Data and From_PublicData get the current
  2.17 default descriptions, Tracklines has none, and id 5 is now NAV_QA
  (Products/NAV_QA, enabled, not required) in place of CTD_Plots.
- Cruise details: UPDATEs on OVDM_CoreVars for the cruise ID, name,
  description, PI, location, dates and ports (Gulf of Mexico Exploration,
  2012/04/28 - 2012/04/30). OpenVDM's installer keeps UPDATEs (it only
  drops the DROP/CREATE TABLE blocks), so both install paths set them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…emo (#22)

The demo's CTD, OpenRVDAS, ROV_OpenRVDAS, Sealog and XBT descriptions now
open with "**Source:**", as EM302's already did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@webbpinner
webbpinner merged commit ceeb0ff into dev Oct 28, 2026
1 check passed
@webbpinner
webbpinner deleted the issue_22 branch October 28, 2026 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant