This repository was archived by the owner on Sep 8, 2025. It is now read-only.
Repository navigation
This repository was archived by the owner on Sep 8, 2025. It is now read-only.
Problems with http cache in multitheaded environment. #208
Description
From kxepal on November 26, 2011 17:36:08
What steps will reproduce the problem? 1. Create multitheaded script
2. Share Database instance between threads
3. Make a lot of requests until crush happens (see exception.log attachment)
There are two different exceptions, but I suppose that they have same nature.
I'll make test script and patch a little bit later, just have created issue to not forget about this problem.
Attachment: exception.log
Original issue: http://code.google.com/p/couchdb-python/issues/detail?id=208
Activity
djc commentedon Jul 12, 2014
From kxepal on November 26, 2011 16:11:08
Attached patch and test case
Attachment: 02-issue-208-http.patch
djc commentedon Jul 12, 2014
From kxepal on November 26, 2011 17:02:54
Forgot to add test case to suite, also named it well.
Attachment: 01-issue-208-test_http.patch
djc commentedon Jul 12, 2014
From matt.goo...@gmail.com on November 28, 2011 07:29:00
That looks like it should work, thanks!
I extracted the cache code from http.Session recently to make it more testable but haven't pushed it yet. My fault entirely, but do you mind if I rework your patch on top on my changes? Should I post the new diff back here for you to check before I commit with your name against it?
djc commentedon Jul 12, 2014
From kxepal on November 28, 2011 09:57:01
There is no problem if you make patch for my patch - finally, we all works for the same project and I always like to see what things I've missed(:
djc commentedon Jul 12, 2014
From djc.ochtman on February 16, 2012 02:31:58
Matt, a ping on this?
djc commentedon Jul 12, 2014
From matt.goo...@gmail.com on September 20, 2012 11:32:34
I've just pushed some code including r6ab0ed59cf1d to extract a Cache class from the middle of the http.Session code. It doesn't actually fix the issue but I wanted to do this to make testing easier/possible.
djc commentedon Jul 12, 2014
From matt.goo...@gmail.com on September 20, 2012 11:47:50
It looks like I may have a heavily modified version of kxepal's test that works against the new Cache class and fails consistently. I'll check it out more - it's a little insane ;) - and see if kxepal's fix works ok.
djc commentedon Jul 12, 2014
From kxepal on September 20, 2012 12:00:12
Nice refactoring! However, I have a little scepsis about tests, since now they would proof the fact that dict iterators are not threadsafe. It still requires to change
.iteritems()to.items()or wrap_cleanupmethod body with lock to let it work in multithreaded environment [1]. In perspective, if we'd like to add Python3 support, we still need to use lock or alien for <Python-2.7.itemsview()method since behavior of.items()is changed, so lock-variant might be more better and optimal.[1] https://code.google.com/p/couchdb-python/source/browse/couchdb/http.py?spec=svn6ab0ed59cf1d013ed7a2fcaf3eaf32ac8130965a&name=6ab0ed59cf1d&r=6ab0ed59cf1d013ed7a2fcaf3eaf32ac8130965a#431