adds a vcsim based test suite and runs it in CI - #541
Merged
Merged
Conversation
The VMware source could only be exercised against a live vCenter, which is why bugs like #476 were found by users rather than by a test. vcsim, the simulator from the govmomi project, replays an inventory captured with `govc object.save`, so the real pyVmomi code path can run in CI. Adds: - tests/fixtures/vcsim: the two captures contributed in #474, a 6.7.0 and an 8.0.3 vCenter. The tests are parametrized over every archive in that directory, so adding a capture needs no test changes. - tests/conftest.py: starts one vcsim per capture on a free port, gives each test a clean in-memory NetBoxInventory, and feeds the source a generated settings.ini. A second connection through pyVmomi provides an independent view of the same simulator to compare against. - tests/test_vmware_source.py: every VM and host is synced, host serial, model and manufacturer match the hardware, VM cpu and memory match the hypervisor, interface MAC addresses match, guest IPs of VMs with running tools are synced and bound to an interface, two runs produce the same inventory, and a run against objects that already exist in NetBox creates nothing new. - .github/workflows/test.yml: installs vcsim and runs pytest on pushes and pull requests. The suite needs no vCenter and no NetBox. Without vcsim installed the vcsim backed tests skip rather than fail, so `pytest` still works on a machine that only has the Python dependencies. Verified on Python 3.13 (the CI version) and 3.14: 16 passed. To check the tests actually catch regressions, four faults were injected into the source one at a time: memory reported in GB, guest IPs never collected (the shape of #476), a VM silently dropped, and the MAC address dropped from interface data. Each one turned the suite red, and it went green again after reverting.
The CI job runs `pytest`, which does not put the working directory on sys.path the way `python -m pytest` does, so conftest failed to import the module package. Set pythonpath in the pytest config instead, which fixes it for a plain `pytest` invocation from a checkout as well.
This was referenced Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Following up on #474: the test setup, before the remaining PRs.
The VMware source could only be exercised against a live vCenter, which is why a bug like #476 reached users instead of a test run. vcsim, the simulator from the govmomi project, replays an inventory captured with
govc object.save, so the real pyVmomi code path runs in CI with no vCenter and no NetBox involved.What is in here
tests/fixtures/vcsim/holds the two captures @ionoci contributed in Maintainer wanted - sunsetting this repository by 30.11.2026 #474 (a 6.7.0 and an 8.0.3 vCenter). The vcsim backed tests are parametrized over every archive in that directory, so dropping in a new capture covers it without touching test code. The README there documents how to take one.tests/conftest.pystarts one vcsim per capture on a free port, gives each test a clean in-memoryNetBoxInventory, and writes the source a generatedsettings.ini. It also opens a second, independent pyVmomi connection to the same simulator, so the tests compare the sync against the SDK rather than against themselves.tests/test_vmware_source.pyasserts that every VM and host is synced; host serial, model and manufacturer match the hardware (allowing for manufacturer normalisation,Dell Inc.toDell); VM cpu and memory match the hypervisor; interface MAC addresses match; guest IPs of VMs with running tools are synced and bound to an interface; two runs produce the same inventory; and a run against objects that already exist in NetBox creates nothing new..github/workflows/test.ymlinstalls vcsim and runs pytest on pushes and pull requests tomainanddevelopment.Without vcsim installed the vcsim backed tests skip rather than fail, so
pyteststill works on a machine that only has the Python dependencies.Verification
16 tests pass on Python 3.13 (the version the workflow uses) and on 3.14.
To show the suite actually catches regressions rather than just passing, I injected four faults into the source, one at a time, and ran it against the 6.7.0 capture:
After reverting each one the suite was green again.
Notes
NBMACAddressobjects for its interfaces (82 objects for 64 distinct interface and address pairs). It does not survive a rerun against saved objects, so it looks like extra POSTs rather than lasting damage. The test compares address sets so it does not fail on it. I will open a separate issue with the details.