Skip to content

Usage of udatetime causing CI failures for Exchange packs #5346

Closed

Description

@blag

SUMMARY

The weekend CI tests for all 100+ StackStorm Exchange packs failed again this weekend:

#!/bin/bash
~/ci/.circle/test ; ~/ci/.circle/exit_on_py3_checks $?

Traceback (most recent call last):
  File "/home/circleci/ci/.circle/validate.py", line 21, in <module>
    from st2common.models.api.pack import PackAPI
  File "/tmp/st2/st2common/st2common/models/api/pack.py", line 24, in <module>
    from st2common import log as logging
  File "/tmp/st2/st2common/st2common/log.py", line 32, in <module>
    from st2common.logging.handlers import FormatNamedFileHandler
  File "/tmp/st2/st2common/st2common/logging/handlers.py", line 24, in <module>
    from st2common.util import date as date_utils
  File "/tmp/st2/st2common/st2common/util/date.py", line 24, in <module>
    import udatetime
  File "/home/circleci/virtualenv/lib/python3.6/site-packages/udatetime/__init__.py", line 20, in <module>
    from udatetime.rfc3339 import (
ImportError: /home/circleci/virtualenv/lib/python3.6/site-packages/udatetime/rfc3339.cpython-36m-x86_64-linux-gnu.so: undefined symbol: __exp_finite
Unable to retrieve pack name.
Original script exit code: 1

Exited with code exit status 1

CircleCI received exit code 1

I believe that this time it is related to this Debian issue where the udatetime library calls out to a C library, which does not properly link to the libm library, as detailed in freach/udatetime#30 and fixed in freach/udatetime#31, which is a miniscule, easy fix yet has been unmerged for over a year.

The udatetime package, introduced to StackStorm back in March in an attempt to optimize storage performance, has not been touched since 2018 and I believe that it's been abandoned. The author's blog has also not seen an update since 2018, and he hasn't posted anything to Twitter since 2018 either. His Xing profile does indicate that he has changed jobs multiple times since 2018 though.

I believe that the udatetime pack has been abandoned. We should consider offering to help maintain it, or we should move to a different library like ciso8601, which is actively maintained (as of this writing, the last release was on August 6th, 2021), and is also faster than udatetime, since performance was obviously a reason to select udatetime to begin with. @Kami Can we get your guidance here please?

STACKSTORM VERSION

StackStorm installed in CircleCI build container, cloned from the master branch:

git clone --depth 1 --single-branch --branch master https://github.com/StackStorm/st2.git /tmp/st2
OS, environment, install method

Ubuntu 18.04
Installed via git master branch, see log 1, and log 2.

Steps to reproduce the problem

Run pack tests in CircleCI.

Expected Results

I expected that udatetime would be importable and not cause issues, especially when the fix is literally a one-line fix that has been deployed in Debian for over a year.

Actual Results

All StackStorm Exchange weekend builds failed in CircleCI.

Activity

  1. added this to the 3.7.0 milestone on Aug 28, 2021
  2. Kami commented on Aug 28, 2021

    @Kami
    Member

    I'm fine with switching to ciso8601 or a similar alternative which is (actively) maintained and support.

    In fact, as you said cisto8601, may even be better choice for multiple reasons.

    1. Based on their benchmarks, it's faster
    2. It supports ISO 8601 or RFC 3339 date strings. IIRC, udatetime only supports RFC 3339 so we have a catch-all fallback in case we try to parse date string udatetime doesn't directly support. Although that may not be 100% the case and we may still need a fallback, which is fine

    Note: ciso8601 doesn't support the entirety of the ISO 8601 spec, only a popular subset

  3. modified the milestones: 3.7.0, 3.8.0 on Jul 28, 2022
  4. amanda11 commented on Jul 28, 2022

    @amanda11
    Contributor

    We also need to switch as udatetime won't build for python 3.9 - freach/udatetime#32.

  5. moved this from To do to Done in StackStorm v3.8.0on Sep 19, 2022
  6. nzlosh commented on Sep 19, 2022

    @nzlosh
    Contributor

    The switch to ciso8601 has been made #5692

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions