Repository navigation
[BUG] --watch does not reload code changes, despite console output saying so "files changed, reloading arq worker..." #429
Description
I am not able to get --watch to work, despite the console output looking like it has restarted arq. Steps to reproduce below, what am I missing?
How to reproduce?
I created an isolated python project in a fresh conda environment (Python 3.11.7) and able to reproduce it:
demo.py file (similar to example in arq doc)
import asyncio
from arq import create_pool
from arq.connections import RedisSettings
async def my_task(ctx):
return "HELLO-1" # <--- try changing this string output when worker is running
async def main():
redis = await create_pool(RedisSettings())
await redis.enqueue_job('my_task')
class WorkerSettings:
functions = [my_task]
if __name__ == '__main__':
asyncio.run(main())Worker running using: arq_try arq demo.WorkerSettings --watch ./
(.conda) (base) ➜ arq_try arq demo.WorkerSettings --watch ./
09:08:06: Starting worker for 1 functions: my_task
09:08:06: redis_version=7.2.4 mem_usage=1.57M clients_connected=4 db_keys=35
09:08:11: 0.45s → 0c923d4152a0456693f3b11b0d08879b:my_task()
09:08:11: 0.00s ← 0c923d4152a0456693f3b11b0d08879b:my_task ● 'HELLO-1' # <--- CORRECT
### now I edit demo.py and change the output string to HELLO-2 ####
files changed, reloading arq worker...
09:08:17: shutdown on SIGUSR1 ◆ 1 jobs complete ◆ 0 failed ◆ 0 retries ◆ 0 ongoing to cancel
09:08:17: Starting worker for 1 functions: my_task
09:08:17: redis_version=7.2.4 mem_usage=1.57M clients_connected=4 db_keys=36
09:08:20: 0.31s → 725b3dc60cd942feb5023aa2c90a5b27:my_task()
09:08:20: 0.00s ← 725b3dc60cd942feb5023aa2c90a5b27:my_task ● 'HELLO-1' # <--- WRONG: THIS SHOULD BE HELLO-2 as the demo.py file was changed / modified
(.conda) (base) ➜ arq_try pip list
Package Version
----------------- --------
anyio 4.2.0
arq 0.25.0
certifi 2024.2.2
click 8.1.7
h11 0.14.0
hiredis 2.3.2
httpcore 1.0.2
idna 3.6
pip 24.0
redis 5.0.1
setuptools 69.0.3
sniffio 1.3.0
typing_extensions 4.9.0
watchfiles 0.21.0
wheel 0.42.0
Project directory tree:
Any help or pointers appreciated!
Reacted by Guro Bokum
Activity
epicwhale commentedon Mar 2, 2024
Anyone else facing this issue?
For now, I have configured my development with a VS code launch json config, that does a
"autoReload": {"enable": true}as a temporary workaroundvidjuheffex commentedon Mar 11, 2024
Lucky,
I don't even get:
files changed, reloading arq worker... 09:08:17: shutdown on SIGUSR1 ◆ 1 jobs complete ◆ 0 failed ◆ 0 retries ◆ 0 ongoing to cancelowalerys commentedon Mar 31, 2024
I also don't get the changes reflected in the worker even though it states it's been reloaded.
owalerys commentedon Mar 31, 2024
The problem is that the worker's source is never reloaded on a detected change in files watched.
See this line: https://github.com/samuelcolvin/arq/blob/ec1532b29d490e66b5c22424b51781865ea4cf44/arq/cli.py#L77
My solution is to use watchfiles directly
watchfiles "arq worker.WorkerSettings" ./