support for asyncio

Hi guys.

Do you consider providing this client library with support for asyncio ?

Regards,
Tomasz Prus

(copy of kubernetes-client/python-base#27)

I've prepared the Pull request where I show how callbacks can be used with asyncio. Let me know what you think about it. Thanks.

@tomplus

I just experimented with Swagger v2.3.1 which supports asyncio+aiohttp based client library generation, so that we could use a "native" async interface instead of interpolating callbacks to futures.

What I have done to make it working, starting from the kubernetes-client/gen repository:

Set SWAGGER_CODEGEN_COMMIT in openapi/python.sh to "v2.3.1"
Add <library>asyncio</library> under openapi/python.xml's <configuration> element.

Run the generator script as instructed.

my config was:

export KUBERNETES_BRANCH="release-1.8"
export CLIENT_VERSION="1.0"  # just a random value
export PACKAGE_NAME="kubernetes"

After generation, restructure the generated client library to match the structure guide
— UPDATE: these are the "small changes" in @tomplus comment in feat: add examples for asynchronous usage (callback, asycio) #324.
- Relocate the generated kubernetes python package to kubernetes/client.
- Add python-base as kubernetes/base git submodule.
- Create symbolic links kubernetes/stream, kubernetes/config, kubernetes/watch like this repo
- Create kubernetes/__init__.py like this repo
- Replace all absolute imports to use relative imports (e.g., from kubernetes.api_client.apps_v1beta1_deployment import AppsV1beta1Deployment in the generated code to from .apps_v1beta1_deployment import AppsV1beta1Deployment)
- Merge requirements of python-base and the generated library
- pip install -e . inside a virtualenv
- Fix up circular import errors related to V1beta1JSONSchemaProps
- NOTE: I think find and sed commands in the generator's python.sh script should do the above changes automatically, but it didn't work out of the box. So I did manually.
Comment out ssl_verify argument in kubernetes/client/api_client.py since aiohttp forbids passing both ssl_context and ssl_verify option (the later is included in the former)

Then I could run the following example:

import asyncio
from kubernetes import client, config

loop = asyncio.get_event_loop()

async def do():
    config.load_kube_config()
    v1 = client.CoreV1Api()
    ret = await v1.list_pod_for_all_namespaces(watch=False)
    for i in ret.items:
        print("%s\t%s\t%s" % (i.status.pod_ip, i.metadata.namespace, i.metadata.name))

loop.run_until_complete(do())
loop.close()

So the majority of APIs can be converted to use asyncio + aiohttp without much efforts, thanks to Swagger. I hope my "manual" fix-ups could be automated somehow!
Still, the python-base streaming wrappers should be rewritten from scratch.

asyncio+aiohttp support would be really appreciated !

I use aiohttp as a server which accesses the python-client so it will be very benefitial to me as well.

Here's how you can do an async watch with the using client (minus error handling):

class Connector(object):
    def __init__(self, q: asyncio.Queue, loop: asyncio.AbstractEventLoop, timeout: int = 30):
        client_config = type.__call__(Configuration)
        config.load_kube_config(client_configuration=client_config, persist_config=False)
        client = ApiClient(configuration=client_config)
        self.__configuration = client_config
        self.__v1 = CoreV1Api(client)
        self.__extv1beta1 = ExtensionsV1beta1Api(client)
        self.__queue = q
        self.__loop = loop

    async def start(self):
        async with aiohttp.ClientSession() as session:
            await asyncio.gather(
                self.__watch(session=session, api=self.__v1, url='/api/v1/endpoints', kind='V1Endpoints'),
                self.__watch(session=session, api=self.__v1, url='/api/v1/services', kind='V1Service'),
                self.__watch(session=session, api=self.__extv1beta1, url='/apis/extensions/v1beta1/ingresses', kind='V1beta1Ingress'),
                loop=self.__loop)

    async def __watch(self, session: aiohttp.ClientSession, api, url: str, kind: str):
        params = {'watch': 'true'}
        async with session.get(self.__configuration.host + url, params=params,
                               headers=self.__configuration.api_key) as response:
            while True:
                chunk = await response.content.readline()
                if not chunk:
                    break
                j = json.loads(chunk)
                j['object'] = api.api_client._ApiClient__deserialize(data=j['object'], klass=kind)
                await self.add_event(j)

At last I’ve prepared the new library kubernetes_asyncio which is based on this Python client, but uses the asyncio generator from swagger-codegen. I've added a script to create this client to kubernetes-client/gen repo in this PR kubernetes-client/gen#60. I've also decided to incorporate functionality from sub-repository kubernetes-client/python-base because a lot of changes according to asyncio were needed.

Please take a look, thanks.

@tomplus

Inspired by @tomplus I have followed in his footsteps and created aiokubernetes.

Unlike kubernetes_asyncio, it is not backwards compatible. The user visible changes are actually minor but allowed me to remove unused Python 2.x code paths and drop Python 3.5 in favor of Python 3.7 (perhaps relevant for #558).

I added documentation and usage examples and would be grateful for any feedback - thank you.

@olitheolix

Please see #324 (comment) for discussion about experimenting asyncio library in this repo.

@olitheolix we need to be backwards compatible for this package. I haven't seen reasons for dropping Python 3.5 (and even strong needs for fully supporting 3.7) yet. What has been changed?

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

/remove-lifecycle stale

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

/remove-lifecycle stale

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

/remove-lifecycle stale

/lifecycle frozen

@tomplus

So what's the state of this? Is there a goal to merge @tomplus changes or implement a new solution?

I would also love to see this become a possibility, what needs to be done for this change? Would it be possible for anyone to do a wrap-up of what is needed for native asyncio support?

kubernetes_asyncio works great

The attached PR seems to have gone stale and closed. People are gonna really struggle to develop WebSocket-based K8s integrations in Python without this, obviously among other use-cases.

@des1redState

Why not use the dropin replacement kebernetes-asyncio? @des1redState

@realabja

@realabja, LOL ironically @tomplus who opened this ticket is the author of kubernetes-asyncio.

It would be much easier for everyone if such support was builtin, I'm quite sure it is a lot of work maintaining a separate version that constantly needs to catch up with the official client.

watching for this feature because modern python web frameworks are trending to adopt asyncio due to the GIL.

@zeddit

watching for this feature because modern python web frameworks are trending to adopt asyncio due to the GIL.

@zeddit We've just dropped the modern async Kubernetes client - kubesdk. Check it out, hope it will help.

@yliaog

I think this done now?

This series of patches brought us to the latest OpenAPI-generator shape, which has an asyncio client:

cc @yliaog

Would be great for folks here to test if this works for them.

Navigation Menu

Repository navigation

support for asyncio #323

38 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

support for asyncio #323

Description

Activity

tomplus commented on Aug 19, 2017

achimnol commented on Jan 21, 2018

seeker89 commented on Feb 1, 2018

EvgenyKhaliper commented on Mar 7, 2018

hanikesn commented on May 18, 2018

tomplus commented on May 20, 2018

olitheolix commented on Jul 7, 2018

roycaihw commented on Jul 27, 2018

fejta-bot commented on Apr 24, 2019

mitar commented on Apr 24, 2019

fejta-bot commented on Jul 23, 2019

mitar commented on Jul 23, 2019

38 remaining items

k8s-triage-robot commented on Mar 30, 2022

mindw commented on Mar 30, 2022

roycaihw commented on Mar 30, 2022

takeda commented on Dec 4, 2022

abi-jey commented on Mar 30, 2023

remram44 commented on Mar 30, 2023

des1redState commented on Sep 13, 2023

abi-jey commented on Sep 13, 2023

takeda commented on Sep 14, 2023

zeddit commented on May 28, 2025

nik-paushkin commented on Nov 30, 2025

tamird commented on Jul 31, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions