Repository navigation
pytest2.8 invariantly writes to working directory + fails on readonly filesystem #1029
Description
Activity
Thanks for the report! 😄
Hmmm I see two possibilities:
- Every call to
config.cache.setwill internally catch read-only errors, and silently fail in this case. - The internal cache plugin will explicitly capture errors during write, and print a pytest warning.
- Add an option to disable
lf-plugin.
I think 1 is too error prone, and 3 will remove much of the usefulness of the
--lfoption, because I usually always want it active when I'm running tests (and will always forget to pass--enable-lf). I think 2 is perhaps more appropriate, as the internal cache is now always enabled it perhaps should be extra careful for cache writing failures.- Every call to
2 is how it should work, since we already made the warning recording a historic hook call, we can propperly issue a config.warn in such cases
The fix only addresses half of the problem. How do I get pytest to stop writing to my working directory and why did this change happen?
How do I get pytest to stop writing to my working directory and why did this change happen?
I think the only way to prevent that currently is by passing
-p no:cacheproviderin the command line, or by modifying yourpytest.ini:[pytest] addopts = -p no:cacheprovider
(In fact I will add this to the docs)
This change happened because in
2.8pytest-cache has been merged into the core. It brings with two important functionalities: re-run last failures or failures first and config.cache object, which lets plugins persist data between test sessions.One of the drawbacks is that now
pytestwill always create a.cachedirectory in therootdirof the test session, as the cache has to be active in order to--lfdo its work. We did not foreseen the problems it could cause like your issue demonstrated, unfortunately (which already has been addressed in #1048, btw).This will be fixed in
2.8.1when it comes out (soon), but until then you can disable the cache completely as I noted above.Reacted by Massood and Eloy Adonis ColellThe standard place to put this is ~/.cache. To easily disambiguate caches against various $PWD's, you can mkdir -p $HOME/.cache/pytest/$PWD. Even better would be to use $XDG_CACHE_HOME with a default value of $HOME/.cache
That's an excellent suggestion I think. Would you mind opening a new issue with it? Thanks! 😄
we could probably use a hash + use a symlink back - using full paths is a huge error source due to path length limits on various platforms
using full paths is a huge error source due to path length limits on various platforms
That's a good point! I know I've had my problems with this on Windows. 😅
But I fear adding another layer of complexity to the system might bring more trouble than it is worth, to be honest. 😬
i feel the need to make basetmp as well as cache locations configurable
we should brainstorm that in some way (since it also makes sense to put the cache + basetmp into something a CI system can easyly pick up for exampleplus XDG_CONFIG_DIRS is the absolutely wrong thing to use in many non-containered ci situations
@RonnyPfannschmidt That's wrongheaded on several counts.
a) A very convenient, and standard, way to configure tools' cache location is to set $XDG_CACHE_DIR. Many of my project's fixtures do this for other reasons already. Similarly, the most convenient, and standard, way to set "basetmp" would be $TMPDIR.
b) "something a CI system can easily pick up" is exactly these environment variables. Many systems will already support this, and those that don't will support injecting environment variables.
c) "non-containered ci" will either support ~/.cache correctly, or set $XDG_CACHE_DIR, because this is necessary for other tools that use the standard. Again, this is already handled in many of my projects because of other tools.
Reacted by Artem20 remaining items
pytestcreates the.cachedirectory even when no tests are actually executed and the cache functionality is not used at all. In a sense the user pays for what they are not using. Here is an example:rm .cache/ -rf rm empty.py touch empty.py py.test empty.py # warns that no tests were run ls -lad .cache # suprise !PS. Maybe I should file a new issue for this because although related to this issue it's not limited to read-only filesystems.
please a new issue
The other day I tried to run our test suite in a bubblewrap sandbox, to make sure our tests don't require network connectivity, and don't write to disk unnecessarily:
bwrap --ro-bind / / --dev-bind /dev /dev --tmpfs /tmp --unshare-net py.testTo my surprise py.test did not show me the tracebacks of the failed tests, but instead crashed trying to write the last-failed cache.
Anyway, just wanted to confirm that this issue is still present in py.test 3.2.3, and mention my use-case.
nobody actually went to fix it
@mgedmin thanks. Just want to mention a workaround:
-p no:cacheproviderwill disable the cache plugin and the problem should go away.Reacted by Alvin WanTaking the oneliners above (and adapting them for time changes), I get the following:
rm -rf foo && mkdir foo && echo 'def test(): pass' > foo/test.py && docker run -ti -v "$PWD/foo:/code:ro" ubuntu:xenial bash -exc 'apt-get update && apt-get install -y --no-install-recommends python-pip python-setuptools && pip install pytest && cd /code && py.test test.py'works fine!
rm -rf foo && mkdir -p foo/.cache/v/cache && echo 'def test(): pass' > foo/test.py && docker run -ti -v "$PWD/foo:/code:ro" ubuntu:xenial bash -exc 'apt-get update && apt-get install -y --no-install-recommends python-pip python-setuptools && pip install pytest && cd /code && py.test test.py'also works fine!
it appears this was fixed at some point, closing.
I believe these to be the same issue so I'm only making a single report.
And the error from our CI server (which is running a repo inside docker)