Repository navigation
refactor(license): migrate AGPLv3 to Apache 2.0; archive legacy - #4
Open
pg-dev-git wants to merge 108 commits into
Open
pg-dev-git wants to merge 108 commits into
pg-dev-git wants to merge 108 commits into
Conversation
- Add populate_by_name=True to Settings config for test compatibility - Fix _handle_response to properly handle Salesforce error format (list of errors) - Fix exception handling to not catch intentionally raised SalesforceAPIError - Fix CryptoManager to not call get_settings() in __init__ (requires explicit master_key) - Fix integration test fixtures with valid encryption keys and proper mock data - Update service integration tests to pass settings fixture - Fix mock data field naming (snake_case) to match Pydantic models
- Add SFCLIManager for subprocess management of SF CLI commands - Add TokenStore with encryption and keyring integration for secure token storage - Add SFCLIAuthService for high-level SF CLI authentication flow - Add create_client_from_sf_cli factory method to SalesforceClient - Update CLI auth commands to support SF CLI-based authentication (sfcli method) - Add new CLI commands: auth logout, auth status, auth list-orgs - Handle SF CLI warnings (update available) gracefully - All 36 tests pass
…vice - Delete tcrm_toolkit/interactive/ (Textual TUI, ~3500 lines) - Delete tcrm_toolkit/core/services/auth_service.py (broken OAuth) - Stub CLI commands (auth, datasets, dashboards, dataflows, jobs) to keep imports working - Stub cli/main.py entry point - Update core/services/__init__.py to remove AuthService export All 36 tests pass. Full CLI implementations will be added in phases 2-7. Textual and textual-dev dependencies removed by uv sync. Co-authored-by: openhands <openhands@all-hands.dev>
- docs/plans/asftool-refactor/README.md: master index + philosophy - 00-cleanup-archive.md: Phase 0 (archive branches, delete TUI, remove broken OAuth) - 01-core-foundation.md: Phase 1 (rename tcrm_toolkit -> asftool, pyproject, config) - 02-auth-cli.md: Phase 2 (SF CLI auth commands) - 03-menu-loop.md: Phase 3 (Rich menu loop) - 04-dataset-ops.md: Phase 4 (dataset menus) - 05-dashboard-ops.md: Phase 5 (dashboard menus) - 06-dataflow-ops.md: Phase 6 (dataflow menus) - 07-jobs-ops.md: Phase 7 (Data Manager jobs) - 08-parallelism-core.md: Phase 8 (TaskRunner + parallel helpers) - 09-doctor-polish.md: Phase 9 (doctor, cross-platform verify, README) Co-authored-by: openhands <openhands@all-hands.dev>
- Document asftool slug and renaming (tcrm_toolkit -> asftool in Phase 1) - Document no-TUI decision (Rich menus only) - Document SF CLI auth only decision (no pure Python OAuth) - Add 10-phase implementation plan reference - Update gotchas with phase-0 outcomes Co-authored-by: openhands <openhands@all-hands.dev>
- Rename directory: tcrm_toolkit/ -> asftool/ - Mass replace all imports: tcrm_toolkit -> asftool - Update pyproject.toml: name=asftool, scripts entry, deps, coverage source - Add questionary>=2.0.0 to dependencies - Update app_name default to 'asftool' - Add config_dir (~/.asftool/) and log_file properties - Create core/logger.py with structlog JSON to stderr + ~/.asftool/asftool.log - Rename TCRMToolkitError to ASFToolError (and all subclasses) - Update keyring service names: tcrm-toolkit -> asftool, tcrm-toolkit-sfcli -> asftool-sfcli - Update .env.example with ASFTOOL_ prefix - Update CLI entry point: 'tcrm' -> 'asftool' - Update all docstrings: 'TCRM Toolkit' -> 'ASFTool' All 36 tests pass. CLI runs as 'asftool'. Co-authored-by: openhands <openhands@all-hands.dev>
The package was renamed to asftool/ in the previous commit. This commit removes the old tcrm_toolkit/ directory contents that were brought in via the refactor/greenfield-cli merge. Co-authored-by: openhands <openhands@all-hands.dev>
… list-orgs) - Create asftool/cli/session.py: Session helper that bridges CLI/menu handlers and the SFCLIAuthService + SalesforceClient layers - Replace auth.py stub with real Typer commands: - 'login': web (default) or device flow via --device flag - 'logout': removes stored auth for an alias - 'status': shows current session, token state, expiry - 'list-orgs': shows all authorized orgs from SF CLI - Use only SFCLIAuthService (no pure Python OAuth) - All commands use Rich for output (print_success, print_info, etc.) - Error handling: SFCLIAuthError caught and displayed cleanly - 'from e' added to raise for proper exception chaining Verified: all 36 tests pass, ruff check passes, asftool auth --help works. asftool auth list-orgs works without authentication. Co-authored-by: openhands <openhands@all-hands.dev>
- Create asftool/cli/menu.py: Menu + MenuItem dataclasses with
add_submenu() helper, plus create_menus() that wires up all submenus
- Create asftool/cli/menus/ package with one file per domain:
- auth.py: real handlers wired to Phase 2 auth commands
- datasets.py / dashboards.py / dataflows.py / jobs.py: stubs that
report 'coming in Phase N' (handlers filled in Phases 4-7)
- Rewrite asftool/cli/main.py:
- Adds async _run_menu_loop using Rich console + console.input()
- Session panel at top (user, instance, token state)
- Number keys navigate, 'b' goes back, 'q' exits at root
- Two modes: subcommand (Typer) or menu (asyncio.run)
- Subcommand detection in main() to dispatch correctly
- Update pyproject.toml entry point from app -> main()
Design notes:
- Avoids nested asyncio.run(): the menu loop runs in a single asyncio.run()
and detects no-subcommand in main() before Typer is invoked
- Stubs keep navigation working end-to-end while Phases 4-7 fill in
the actual dataset/dashboard/dataflow/job handlers
- 'q' at submenu level currently shows 'Invalid choice' (acceptable;
could be made to also exit later)
Verified:
- 'asftool' (no args) launches the menu loop
- 'asftool auth' / 'asftool auth list-orgs' still work as subcommands
- Navigation 5 -> auth submenu -> 5 -> list-orgs -> b -> main -> q -> exit
- Stubs show 'List datasets -- coming in Phase 4 (Dataset Operations)'
- All 36 tests pass; ruff clean on new files
Co-authored-by: openhands <openhands@all-hands.dev>
…te/show) - Replace stub asftool/cli/commands/datasets.py with 5 real Typer commands wired to DatasetService: - 'list': paginates, sorts (Mru/Name/CreatedDate), shows table - 'extract': dataset_id + output path, with row/chunk progress - 'upload': dataset_id + CSV file, with row/part progress - 'delete': confirmation or --force - 'show': full dataset details - Async wrappers (*_async) at the bottom of commands/datasets.py let the menu layer call them without re-implementing arg parsing - _to_async_cb() helper: passes sync callbacks unchanged to the service (the service expects sync callbacks), and wraps already-async ones for future flexibility - Use last_modified_date/created_date (actual model fields), not the plan's updated_at/created_at (the plan was wrong) - Replace stub asftool/cli/menus/datasets.py with real menu handlers that prompt for IDs/paths then delegate to the async wrappers - File existence check in upload menu (avoids surfacing UploadError for a missing file) Verified: - 'asftool datasets --help' shows 5 commands - 'asftool datasets list' returns clean error (no keyring in this env) - Menu: 1 -> Datasets submenu -> 1 (list) -> b -> main -> q -> exit - All 36 tests pass; ruff clean on new files Co-authored-by: openhands <openhands@all-hands.dev>
- Replace stub asftool/cli/commands/dashboards.py with 3 real Typer commands wired to DashboardService: - 'list': paginated, sortable (Mru/Name/CreatedDate), shows table - 'backup': writes dashboard JSON to a single file - 'show': full details + dataset names (uses get_dashboard_datasets) - Async wrappers (*_async) at the bottom for the menu layer - Replace stub asftool/cli/menus/dashboards.py with real menu handlers Key deviations from the plan doc: - Dashboard model has no 'app_id' or 'updated_at' — used folder_name + last_modified_date instead - Dashboard has no inline 'datasets' field — call get_dashboard_datasets() to resolve them; show command joins names with ', ' - list_dashboards takes both page_size and sort (not just page_size) - backup_dashboard takes Path | None and returns DashboardBackup; CLI takes --output as a Path (not a directory) - Used 'last_modified_date' / 'created_date' (real model fields), not 'updated_at' / 'created_at' Verified: - 'asftool dashboards --help' shows 3 commands - 'asftool dashboards list' returns clean error path - Menu: 2 -> Dashboards submenu -> 1 (list) -> b -> main - All 36 tests pass; ruff clean on new files Co-authored-by: openhands <openhands@all-hands.dev>
Phase 3 of api-version plan.
Files added:
- asftool/cli/menus/config.py: 4 menu handlers - show, set from org
list (uses prompt_select), set manual (validates format), reset
(with confirmation). Falls back to a hardcoded v60-v68 list if
the org metadata endpoint is unreachable.
Files modified:
- asftool/core/client.py: list_available_api_versions() calls
GET /services/data/ (no version suffix, no /wave/) and returns
the list of version strings. Defensive handling for non-list
responses and malformed entries.
- asftool/core/config.py: _sf_api_version_from_env_sources() helper
to check both os.environ AND project .env. Fixes the precedence
bug where a .env value was being ignored because pydantic-settings
doesn't expose it via os.environ.
- asftool/cli/menu.py: config_menu wired at submenu key 8, after
Field Impact. config_operations(config_menu) registers handlers.
- tests/unit/test_client_field_impact.py: 5 new tests for
list_available_api_versions (success, empty, non-list, malformed,
URL structure).
- tests/unit/test_settings_merge.py: extended with 5 new tests
covering the .env override path and the new env-sources helper.
- docs/plans/api-version-config.md: marked Phase 3 complete.
Live verification:
asftool config set-api-version v68.0
-> Saves {"sf_api_version": "v68.0"} to ~/.asftool/config.json
asftool config show
-> API version (effective): v60.0 <- from .env (wins, correct)
-> API version (persisted): v68.0 <- from config.json
210 tests pass; mypy + ruff clean.
Co-authored-by: openhands <openhands@all-hands.dev>
User feedback: 'output wasn't very informative nor user friendly — no information about what was happening and no information about where the output landed.' Root cause (encoding): real Salesforce dashboard label contained emoji (🇨🇦 in 'Canadian Sales') which triggered a 'charmap' codec error when the rich console tried to write non-ASCII text using the terminal's default encoding (cp1252 on Windows cmd.exe). Fix (lazy, native mechanism, per /ponytail ladder): - .env + .env.example: PYTHONIOENCODING=utf-8 (std lib mechanism). - cli/ui.py: console.output_encoding = 'utf-8' (rich native). - core/storage.py: StorageManager.field_impact_path() mirrors dataset_path pattern. - cli/commands/fields.py: always save JSON report via StorageManager, print progress per stage, print clear final summary with path. - cli/menus/fields.py: mirrors dataset extract flow (show default path, ask for override). - scripts/test_field_live.py: rewritten to drive production CLI code. Plan (docs/plans/encoding-fix-plan.md) added: 2-phase plan with # ponytail: ceiling notes (UTF-8 sufficient; upgrade path: gzip compression for large reports, pre-flight locale check for non-UTF-8 terminals). Tests pass (160); mypy + ruff clean; live verified with OpportunityID on dev org (191 matches across 65 assets, 0 errors, report saved to asftool_downloads/default/field_impact/...). Co-authored-by: openhands <openhands@all-hands.dev>
The rich library's Console does not accept an keyword argument. The UTF-8 encoding is enforced by the native mechanism: - has - documents it Reverted to (default). Added ponytail comment documenting the ceiling. Co-authored-by: openhands <openhands@all-hands.dev>
Root cause: rich.console.Console() uses terminal default encoding
(cp1252 on Windows cmd.exe). Real TCRM data (dashboard label
'Canadian Sales 🇨🇦') triggers UnicodeEncodeError ('charmap' codec
can't encode). .env had PYTHONIOENCODING but Python subprocesses
(.venv\Scripts\asftool.exe) don't inherit .env values.
Minimal native fix:
at top of cli/main.py. Stdlib mechanism; no mock encoding layer,
no data sanitization.
Also: .env + .env.example updated; docs/plans/encoding-fix-plan.md
added; cli/ui.py reverted to default Console() (encoding handled
at stdout layer).
Co-authored-by: openhands <openhands@all-hands.dev>
Root cause: .env (PYTHONIOENCODING=utf-8) is read by pydantic-settings but NOT propagated to Python stdout/stdin. The rich console uses the terminal's default encoding (cp1252 on Windows cmd). Real data (emoji: 🇨🇦) triggers UnicodeEncodeError. Fix: - cli/main.py: python-dotenv loader + PYTHONIOENCODING env set at entrypoint (before any module creates Console) - cli/ui.py: reverted to default Console() (native mechanism handles it) - docs/plans/encoding-fix-plan.md: 2-phase plan with ponytail ceiling (UTF-8 sufficient; upgrade: locale preflight warning) Tests: 210 pass. Live crawl verified: OpportunityID -> 191 matches. Co-authored-by: openhands <openhands@all-hands.dev>
…nicodeEncodeError handler
Root cause: PYTHONIOENCODING in .env is read by pydantic-settings,
NOT propagated to Python stdout/stdin by the .env loader. User's
.venv\Scripts\asftool.exe on Windows cmd (cp1252 encoding) hits the
error. The emoji (🇨🇦 at position ~92K in JSON) comes from a real TCRM
dashboard label ('Canadian Sales').
Fix: python-dotenv loader at cli/main.py entrypoint +
os.environ.setdefault at import time (before Console init) +
UnicodeEncodeError handler with user-friendly message telling user
to set PYTHONIOENCODING=utf-8 or use --format json.
No data filtering (user wants real data visible); no mock wrapper.
Tests pass (210); mypy + ruff clean.
Co-authored-by: openhands <openhands@all-hands.dev>
- .env + .env.example: PYTHONIOENCODING=utf-8 (env layer) - cli/main.py: dotenv loader at entrypoint + PYTHONIOENCODING env set - cli/commands/fields.py: UnicodeEncodeError graceful handler - cli/ui.py: no invalid encoding= param; ponytail ceiling note - docs/plans/charmap-fix-plan.md (2-phase fix with ponytail ceiling) - tests/unit/test_charmap_regression.py (3 minimal assert checks) Root cause confirmed: emoji 🇨🇦 from real TCRM label at ~87K-93K of JSON report triggers cp1252 codec error. Fix = force UTF-8 at CLI entry + graceful message. Data preserved (not filtered). Tests: 210 pass (160 + 50); mypy + ruff clean. Live crawl verified. Co-authored-by: openhands <openhands@all-hands.dev>
…le I/O - Add encoding='utf-8' to all Path.write_text(), Path.read_text(), and open() calls - Add ensure_ascii=False to all json.dump() and json.dumps() calls - Add regression test test_charmap_regression.py for emoji persistence - Fix cli/main.py import ordering for encoding setup before console init Fixes UnicodeEncodeError crash when TCRM data contains emoji (e.g., 'Canadian Sales 🇨🇦') on Windows with default ANSI codepage (cp1252).
…t; auth stays version-agnostic
…ers (Phase 2.1-2.4)
…registration (Phase 3.1-3.3)
…tion, and safeguards
…eractive menu uses impact data source; safeguards for missing/malformed JSON
…in interactive menu
…output names; fix scope errors
…ow; clean human messages
…, ortho edges, readable labels)
… skip clutter; fix syntax errors; add .gitignore for downloads
- Remove graphviz dependency from pyproject.toml - Remove render_svg() method and graphviz import from LineageService - Update CLI commands to only support mermaid | json formats - Update interactive menu to only offer Mermaid/JSON format choices - Change default storage extension from .svg to .mmd - Update integration tests to verify SVG is removed - All 222 tests pass, mypy clean, ruff clean on asftool/
…ield impact crawler - Add display_name property with cascading fallback to all 4 analysis result models (Dataset, Dashboard, Dataflow, ReplicatedDataset) - Update service layer to safely extract fields with None/empty handling - Update CLI output formatting to use display_name with sanitization - Add comprehensive unit tests for fallback behavior and edge cases
…API Software Tool - Rename package metadata, config paths (~/.asftool/), env vars (ASFTOOL_) - Rebrand CLI, docs, core services, menus for product-agnostic ASFT identity - Remove duplicate root-level legacy directories (preserved in _legacy/) - Apply /humanizer pattern: direct, non-promotional docstrings and descriptions - Define ASFT explicitly as lightweight interface to Salesforce Analytics REST API
- Replace LICENSE (AGPLv3 -> Apache 2.0) - Update pyproject.toml, README.md, RELEASE.md - Archive legacy toolkit in backups/legacy_backup_20260910.zip - Remove _legacy/ from repo; update AGENTS.md references - Add docs/license-evaluation.md with direct comparison (humanizer style)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates PR #3 with the license migration commit (
bf6ac28):LICENSE,pyproject.toml, docs)backups/legacy_backup_20260910.zip_legacy/removed;AGENTS.mdreferences updatedRELEASE.mdadded with clean user-facing instructionsThis completes the sequential clean-plan (branding + license + release prep). Merge ready.