Repository navigation
configure .cache via $XDG_CACHE_DIR #1089
Description
Activity
is there a good package that helps finding those paths and helps clearing unused caches
after all if the cache folder is outside of the working directory,
its lifetime is unrelated to the working directoryimagine for example a unaware ci system that makes one folder per build could easily DOS the ci server due to filling up ~/.cache of the ci user
with the current mode of operation, cache and working directory are simply co-located and share a lifetime, so data management is easy, the cache is gone when one deletes the working directory
im not per se opposed to following XDG, but it makes lifetimes of file-system objects more tricky on our case and seems not really available on windows either
(its my personal impression that xdg works only well on posix boxes with a typical linux/bsd setup)This one at least helps find a good platform-specific cache dir: https://pypi.python.org/pypi/appdirs/1.4.0
This one helps manage expiry for a file-based cache.
Making one of the keys be the working directory would make it work.i shorty investigated appdirs and pyfscache, unfortunately both are unsuitable
appdirs seems bug-ridden
pyfscache is for something entirely different not fitting the use-casehow about introducing a ini value/env variable to select the cache backend,
which defaults to worktree, but one can choose xdg/disabled?@hpk42 oppinions?
Personally I'd rather default to xdg instead of needing to add .cache to the gitignore in every one of my projects
while i agree that it is a nice default, we cant do a change like that befor 3.0
also before we can do it propperly we need to come up with a solid way to clean things up
Maybe introducing the cache plugin defaulted to on should have been in 3.0 too :/
since its a normal feature addition, it doesnt warrant a release that large
i hate ".cache" folder for aesthetic reason. my projects folders are already populated by ide, git, python and others "support folders". sometimes i have more folders than actually python program files! please make ".cache" location configurable!
Reacted by Artemhow about introducing a ini value/env variable to select the cache backend
Anyone is against this? While it may not be the best option, it is simple to implement and backward compatible.
It should support variable expansion and default to
.cachefor backward compatibility. This would allow users to change it like this:[pytest] cache_dir=$XDG_CACHE_DIR
Or use an environment variable:
PYTEST_CACHE_DIR=/some/pathReacted by Artem, Alexander Rodionov, Felix Meyer-Wolters and Martín GaitánI'm against the most easy variant, I want a simple solution
@RonnyPfannschmidt I've found that the function you'd need exists in pip.utils:
https://github.com/pypa/pip/blob/develop/pip/utils/appdirs.py#L13
How about we add a configuration option that decides between the two systems?
cache_location, which can belocal(<rootdir>/.cache, the default) orsystem(using the heuristics suggested by @bukzor)? This could be added in2.9.If we make a poll and people wants to change
systemas the default, we can announce that the default will change in2.10, giving plenty of time for people to change to their desired setting while in2.9and not get caught off guard.BTW, I'm just throwing some ideas so we can reach some consensus, I don't mind having the
.cachedirectory in pytest's rootdir: I configured my git-global-ignore file to always ignore it.35 remaining items
@ssbarnea this is about pytest cache, which is different from byte code cache
You need at least one cache per test invocation root, but potentially more than one for virtualenvs/python versions
And you need to be able to cleanup potentially independently of the existing folders
I see two possible approaches here:
- use only user-level .cache and delay addressing the cleanup for a follow-up contribution as that is not critical (if we include project root folder in hash)
- adopt the project level
.cachemodel similar to https://dot-config.github.io/ - basically changing default to be.cache/pytest_cacheshould be enough for that. At least we avoid being another software the clutters our project root directory with temp stuff.
pytest is the only tool in my python stack which still pollutes my source directories.
python,poetry,pipenv, etc all use~/.cache/...by default or allow me to configure conveniently via env var.Reacted by Oleksandr Simonov, Tomáš Faikl, karlicoss and Sirz BenjieReacted by Curt J. SampsonWould also very, very much like to see this, please.
Using
XDG_CACHE_HOMEwould cause confusion and pain.XDG_*directories are specifically for users' personal environment configuration (XDG stands for X Desktop Group, and for historical reasons is the term used in the freedesktop.org specifications) and this should not be shared with development projects that may write conflicting information and may want to remove information when doing clean builds and the like.Consider what can happen if you use
XDG_CONFIG_HOMEfor configuration information for a particular project when testing it. Running whatever in your development checkout sets this would overwrite the user's configuration, which may well be different from the configuration used for testing. Further, if you have two different development checkouts, each would overwrite the other's information.Mixing different projects' build artifacts also makes clean builds more difficult since it's hard to know what to remove and removals may affect other projects. In the current situation, you simply
rm -rf .pytest_cacheor similar in your project dir and you know that a) you are fully clean on that, and b) you have not affected any other projects, or other copies of this project.I don't mind if you add this as some sort of option for people who really want it, but it should not be the default.
pytest is the only tool in my python stack which still pollutes my source directories. python, poetry, pipenv, etc all use ~/.cache/... by default or allow me to configure conveniently via env var.
We have
.gitignorefiles for a reason. And env vars are a terrible configuration mechanism because, unlike apyproject.tomlorpytest.inifile, they are unobvious, easily change from session to session and terminal to terminal, and hard to set reliably in the first place. Where build artifacts go should not be random based on users' configuration, but should be defined in a config file in the project itself so that both users and the build system always know where everything is and always use a consistent set of files.(FWIW, I just got nailed today by the default 'use $CWD'—effectively the same thing as using an env var—when my
--lf/--ffoptions broke because I had two cache directories; which one was used depended on the circumstances. I implemented the correct solution: add apytest.inifile that puts the cache dir in one place for the project, regardless of the my personal environment.)(Also FWIW, I configure my projects to place all build products, cache dirs, etc. under a
.build/directory at the top level of the project, at least where the tools don't directly block this technique. This greatly reduces clutter (which I also hate), and also gives you an easy way to do a fully clean build: just remove that one directory.)By now there are dozens of tools that map folders in there to worktrees including hatch pre-commit and others
Having a distinction is a solved problem
- added 3 commits that reference this issue
on Sep 15, 2026 - added 3 commits that reference this issue
on Sep 23, 2026 - added 3 commits that reference this issue
on Oct 4, 2026
Forked from #1029
a) A very convenient, and standard, way to configure tools' cache location is to set $XDG_CACHE_DIR. Many of my project's fixtures do this for other reasons already. Similarly, the most convenient, and standard, way to set "basetmp" would be $TMPDIR.
b) "something a CI system can easily pick up" is exactly these environment variables. Many systems will already support this, and those that don't will support injecting environment variables.
c) "non-containered ci" will either support ~/.cache correctly, or set $XDG_CACHE_DIR, because this is necessary for other tools that use the standard. Again, this is already handled in many of my projects because of other tools.