Repository navigation
bst workspace open dislikes git sources that have submodules #18
Description
See original issue on GitLab
In GitLab by [Gitlab user @coldtom] on Nov 8, 2018, 10:41
Summary
Attempting to run bst workspace open caused me a stack trace, I think due to the submodules in a git element, but this may not be the case.
Steps to reproduce
- Clone freedesktop-sdk
- Run
bst workspace open desktop/gst-libav gst-libav
What is the current bug behavior?
The following stack trace:
[--:--:--][][] START Loading elements
[00:00:01][][] SUCCESS Loading elements
[--:--:--][][] START Resolving elements
[00:00:01][][] SUCCESS Resolving elements
[--:--:--][][] START Resolving cached state
[00:00:00][][] SUCCESS Resolving cached state
[--:--:--][][] START Checking sources
[00:00:00][][] SUCCESS Checking sources
[--:--:--][d4ca660b][ main:desktop/gstreamer-libav.bst ] START Staging sources to here
[--:--:--][ ][ main:desktop/gstreamer-libav.bst-0 ] STATUS Running host command
/usr/bin/git show 1.14.0-0-g1830c75cd28fc68bd26f9530db18c2da474a4aa2:.gitmodules
[--:--:--][ ][ main:desktop/gstreamer-libav.bst-0 ] STATUS Running host command
/usr/bin/git ls-tree 1.14.0-0-g1830c75cd28fc68bd26f9530db18c2da474a4aa2 common
[--:--:--][ ][ main:desktop/gstreamer-libav.bst-0 ] STATUS Running host command
/usr/bin/git ls-tree 1.14.0-0-g1830c75cd28fc68bd26f9530db18c2da474a4aa2 gst-libs/ext/libav
[--:--:--][ ][ main:desktop/gstreamer-libav.bst-0 ] STATUS Running host command
/usr/bin/git ls-tree 1.14.0-0-g1830c75cd28fc68bd26f9530db18c2da474a4aa2 gst-libs/ext/gas-preprocessor
[--:--:--][ ][ main:desktop/gstreamer-libav.bst-0 ] START Setting up workspace "/home/tom/.cache/buildstream/build/workspace-desktop-gstreamer-libavyrmfg9lh"
[--:--:--][][] BUG Primary URL marked twice with different URLs
Traceback (most recent call last):
File "/home/tom/.local/bin/bst", line 8, in <module>
sys.exit(cli())
File "/home/tom/.local/lib/python3.7/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/home/tom/Documents/work/buildstream/buildstream/_frontend/cli.py", line 162, in override_main
standalone_mode=standalone_mode, **extra)
File "/home/tom/.local/lib/python3.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/tom/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/tom/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/tom/.local/lib/python3.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/tom/.local/lib/python3.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/tom/.local/lib/python3.7/site-packages/click/decorators.py", line 27, in new_func
return f(get_current_context().obj, *args, **kwargs)
File "/home/tom/Documents/work/buildstream/buildstream/_frontend/cli.py", line 705, in workspace_open
force=force)
File "/home/tom/Documents/work/buildstream/buildstream/_stream.py", line 513, in workspace_open
target._open_workspace()
File "/home/tom/Documents/work/buildstream/buildstream/element.py", line 1810, in _open_workspace
source._init_workspace(temp)
File "/home/tom/Documents/work/buildstream/buildstream/source.py", line 672, in _init_workspace
self.init_workspace(directory)
File "/home/tom/Documents/work/buildstream/buildstream/plugins/sources/git.py", line 408, in init_workspace
mirror.init_workspace(directory)
File "/home/tom/Documents/work/buildstream/buildstream/plugins/sources/git.py", line 220, in init_workspace
url = self.source.translate_url(self.url)
File "/home/tom/Documents/work/buildstream/buildstream/source.py", line 455, in translate_url
self.mark_download_url(url, primary=primary)
File "/home/tom/Documents/work/buildstream/buildstream/source.py", line 502, in mark_download_url
"Primary URL marked twice with different URLs"
AssertionError: Primary URL marked twice with different URLs
What is the expected correct behavior?
Workspace should open correctly.
- BuildStream version affected: /milestone %BuildStream_v1.2
Activity
BuildStream-Migration-Bot commentedon Feb 3, 2021
In GitLab by [Gitlab user @abderrahimk] on Nov 11, 2018, 16:00
This also happened with me with
core/totem.bstfrom gnome-build-meta. Trying to debug it, I found thatmark_download_urlis called twice for totem, once with a url ofgitlab_gnome_org:GNOME/totemand onec with a url ofhttps://gitlab.gnome.org/GNOME/libgd.gitwhich is a submodule.So the problem happens only when there is a submodule.
BuildStream-Migration-Bot commentedon Feb 3, 2021
In GitLab by [Gitlab user @abderrahimk] on Nov 11, 2018, 16:01
changed title from
bst workspace opendislikes git sources {-sometim-}es tobst workspace opendislikes git sources {+that have submodul+}esabderrahim commentedon Feb 21, 2022
IIUC, it seems we need to call
mark_download_urlonly once withprimary=True(the default). So the second call for the submodule URL should passprimary=False.Another option is to drop submodule support from the git plugin and use a different plugin for submodules. This is what we're currently doing in freedesktop-sdk with the
git_moduleplugin from bst-external/bst-plugins-experimental.abderrahim commentedon Jun 10, 2022
I guess this needs to be moved to the plugins repository now?
gtristan commentedon Jul 14, 2022
Currently this is a known limitation also with bst-1, not considering this a blocker for 2.0 but hopefully easy enough to fix.
nanonyme commentedon Oct 30, 2022
Needs testing anyway on bst2 whether this is happening at all anymore. I briefly looked at the code and it seems to use primary=False for all submodules.