RuntimeError: memory access out of bounds (Pyodide 128MB limit) #42

Activity

1. dom96 commented on Nov 3, 2025

   

   dom96

   on Nov 3, 2025

   Contributor

   More actions

   Currently 128mb is the maximum a Worker can use. What are you doing in your Worker that is exceeding this? Can you share a minimal repro?
2. deanflaviano commented on Nov 4, 2025

   

   deanflaviano

   on Nov 4, 2025

   Author

   More actions

   tldr, I tried to create a proxy to route requests to my AI API backend with images 1-5MB in size. I tried most of the different ways of full loading, buffering, streaming, forwarding the requests and also testing request limits and different max connections keep alive connections etc, but I could never get the server to handle more than 5-10 request/second before running into memory limits. When I tried it on a JS worker it could handle 20-40/requests per second before running into issues.

   In production, the workers will scale automatically, but imo 5-10 requests per seconds was too low to be reliable. My conclusion was that the Cloudflare python workers add too much bloat in the request handling.

   I asked Claude to minimum viable example from my latest try, which gives a rough idea of what I was trying to achieve.

   """
   Minimal Cloudflare Worker demonstrating memory issues with image streaming
   """
   
   from fastapi import FastAPI, Request, HTTPException
   from starlette.responses import StreamingResponse
   from workers import WorkerEntrypoint
   import httpx
   
   app = FastAPI(title="CF Worker Memory Test")
   
   
   async def stream_proxy(request: Request, backend_url: str) -> StreamingResponse:
       """Stream image data from client -> backend -> client without buffering"""
       
       # Prepare headers for backend request
       headers = {"Content-Type": request.headers.get("content-type", "application/json")}
       
       # Conservative connection limits
       limits = httpx.Limits(max_connections=30, max_keepalive_connections=30)
       
       client = httpx.AsyncClient(timeout=None, limits=limits, follow_redirects=False)
       
       # Stream request to backend
       cm = client.stream(
           "POST",
           backend_url,
           content=request.stream(),  # Stream upload without buffering
           headers=headers,
       )
       resp = await cm.__aenter__()
       
       # Handle errors
       if resp.status_code >= 300:
           err_bytes = await resp.aread()
           await cm.__aexit__(None, None, None)
           await client.aclose()
           raise HTTPException(
               status_code=resp.status_code,
               detail=err_bytes.decode("utf-8", errors="ignore") or "Backend error"
           )
       
       # Stream response back to client
       async def iter_response():
           try:
               async for chunk in resp.aiter_bytes():
                   if chunk:
                       yield chunk
           finally:
               await cm.__aexit__(None, None, None)
               await client.aclose()
       
       return StreamingResponse(
           iter_response(),
           media_type=resp.headers.get("content-type", "application/octet-stream"),
           status_code=resp.status_code,
       )
   
   
   @app.get("/")
   async def root():
       """Health check"""
       return {"status": "ok", "message": "Memory test worker"}
   
   
   @app.post("/process_image")
   async def process_image(request: Request):
       """
       Accept image and proxy to backend
       
       Expected request body (multipart/form-data or JSON with base64):
       {
           "image": "base64_encoded_image_data" or binary file
       }
       """
       # Get backend URL from environment
       backend_url = request.scope.get('env', {}).get(
           'BACKEND_URL',
           'https://your-test-backend.example.com/process'
       )
       
       return await stream_proxy(request, backend_url)
   
   
   class Default(WorkerEntrypoint):
       async def fetch(self, request):
           """Cloudflare Worker entry point"""
           import asgi
           return await asgi.fetch(app, request, self.env)
   
3. dom96 commented on Nov 5, 2025

   

   dom96

   on Nov 5, 2025

   Contributor

   More actions

   What issues did you run into with the JS worker? Was it memory problems too? If you're running into limitations with JS Workers then Python Workers are going to suffer from the same limits.
4. deanflaviano commented on Nov 5, 2025

   

   deanflaviano

   on Nov 5, 2025

   Author

   More actions

   Yes, both ran out of memory, but the JS workers requires a lot less memory per request, as mentioned the python workers can only handle 5-10 request/second and the JS workers could handle 20-40/requests. I believe the Python Workers is a good use-cases to use as proxy AI APIs, however, they memory handling makes the scaling of the workers in Prod a risk factor, while JS workers have more room to scale and avoid the memory limits.