[cleanup] Avoid importing pandas at ccflow import time - #272
Merged
Merged
Conversation
Contributor
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #272 +/- ##
==========================================
- Coverage 93.58% 93.58% -0.01%
==========================================
Files 176 176
Lines 20590 20586 -4
Branches 1358 1359 +1
==========================================
- Hits 19269 19265 -4
Misses 1048 1048
Partials 273 273 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ptomecek
force-pushed
the
cleanup/lazy-numpy-pandas-tokenize
branch
from
September 21, 2026 18:43
b2e0f2d to
c5f3949
Compare
ptomecek
marked this pull request as draft
September 21, 2026 18:52
ptomecek
force-pushed
the
cleanup/lazy-numpy-pandas-tokenize
branch
from
September 21, 2026 19:35
c5f3949 to
0e55267
Compare
ptomecek
changed the title
ptomecek
marked this pull request as ready for review
September 21, 2026 19:47
normalize_token registered its only pandas handler by dispatching on pd.Timestamp, which required `import pandas` at module load and so made every `import ccflow` pay for pandas whether or not the caller ever tokenizes one. That handler's body is identical to the datetime one apart from its tag, and pandas was only ever needed as a dispatch key, so identify Timestamp by name inside the existing datetime handler instead. Subclasses are still matched by walking the MRO, and plain datetimes cost one extra identity check. numpy is deliberately left alone: ccflow.exttypes.pydantic_numpy already imports it eagerly, so deferring its handlers would have no effect today. Token values are unchanged, so existing cache keys remain valid. Signed-off-by: Pascal Tomecek <pascal.tomecek@cubistsystematic.com>
ptomecek
force-pushed
the
cleanup/lazy-numpy-pandas-tokenize
branch
from
September 21, 2026 19:48
0e55267 to
ad447ed
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
import ccflowpulled in pandas unconditionally. The only pandas-specific handler in the tokenizer is forpd.Timestamp, and registering it meant importing pandas at module load, so every user paid that cost whether or not they ever tokenized a pandas object.pd.Timestampsubclassesdatetime, so it already reaches thedatetimehandler. The only thing the dedicated registration added was a distinct tag, so that aTimestampand an equal plaindatetimedon't produce the same token. Matching on the type's name gives the same result without the import.Tokens are unchanged, so existing cache keys stay valid.