From 53aa1fd530a2d53ec4d2f98eef9e8355a05087fe Mon Sep 17 00:00:00 2001 From: Rahul Krishna Date: Mon, 22 Jun 2026 20:59:08 -0400 Subject: [PATCH 1/4] feat(java): add a read-only Neo4j analysis backend (parity with Python/TypeScript) Adds cldk.analysis.java.neo4j.JNeo4jBackend, a read-only Cypher client that reconstructs the canonical JApplication from the graph codeanalyzer-java (>=2.4.0) emits with --emit neo4j, then answers all 36 JavaAnalysisBackend queries with the in-memory backend's logic. Selected by passing a Neo4jConnectionConfig as the backend= config to CLDK.java()/JavaAnalysis (JavaBackend union extended). Verified 1-to-1 against JCodeanalyzer on the daytrader8 sample: everything the graph contains reconstructs identically. Three producer-side gaps in the 2.4.0 emitter are tracked upstream (codeanalyzer-java#156/#157/#158), not query bugs. - backend_config: JavaBackend = Union[CodeAnalyzerConfig, Neo4jConnectionConfig] - java_analysis/core: select JNeo4jBackend by config type; allow project_path=None - bump codeanalyzer-java pin to 2.4.0 (adds the Neo4j emitter) - tests: selection + ABC contract (run anywhere) + skip-by-default daytrader parity - README: mermaid + backends table show Java's read-only Neo4j backend Closes #167 --- CHANGELOG.md | 13 +- README.md | 317 +++----- cldk/analysis/commons/backend_config.py | 5 +- cldk/analysis/java/java_analysis.py | 44 +- cldk/analysis/java/neo4j/__init__.py | 22 + cldk/analysis/java/neo4j/config.py | 27 + cldk/analysis/java/neo4j/neo4j_backend.py | 707 ++++++++++++++++++ cldk/analysis/java/neo4j/reconstruct.py | 285 +++++++ cldk/core.py | 5 +- pyproject.toml | 4 +- .../java/test_java_backend_contract.py | 14 +- .../analysis/java/test_java_neo4j_backend.py | 147 ++++ .../java/test_java_neo4j_selection.py | 75 ++ 13 files changed, 1445 insertions(+), 220 deletions(-) create mode 100644 cldk/analysis/java/neo4j/__init__.py create mode 100644 cldk/analysis/java/neo4j/config.py create mode 100644 cldk/analysis/java/neo4j/neo4j_backend.py create mode 100644 cldk/analysis/java/neo4j/reconstruct.py create mode 100644 tests/analysis/java/test_java_neo4j_backend.py create mode 100644 tests/analysis/java/test_java_neo4j_selection.py diff --git a/CHANGELOG.md b/CHANGELOG.md index f8370aca..1b75db8e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -63,7 +63,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 where calls to a bare module name that is also imported (e.g. `os`/`re`/`json`) are dropped from the emitted call graph. `PythonAnalysis` / `CLDK.analysis(language="python")` accept the same optional `neo4j_config`. -- Bumped `codeanalyzer-python` to `0.2.0` (adds the Neo4j graph emitter). +- Read-only Neo4j-backed **Java** analysis backend (`cldk.analysis.java.neo4j.JNeo4jBackend`), + completing Neo4j parity across all three languages. It reconstructs the canonical `JApplication` + from the graph `codeanalyzer-java` (>= 2.4.0) emits with `--emit neo4j` and answers all 36 + `JavaAnalysisBackend` queries with the in-memory backend's logic. Verified against the daytrader8 + sample (145 classes): everything the graph actually contains reconstructs identically to + `JCodeanalyzer`. Three producer-side gaps in the 2.4.0 emitter make the graph an incomplete + projection (tracked upstream, not query-layer bugs): all fields of a class collapse to one node + (codeanalyzer-java#156), imports lose the type name (codeanalyzer-java#157), and `J_CALLS` + materializes only a fraction of the call graph (codeanalyzer-java#158). `JavaAnalysis` / + `CLDK.java(...)` accept a `Neo4jConnectionConfig` as the `backend=` config to select it. +- Bumped `codeanalyzer-python` to `0.2.0` (adds the Neo4j graph emitter); bumped `codeanalyzer-java` + to `2.4.0` (adds the Neo4j graph emitter). - Optional `neo4j` extra (`pip install cldk[neo4j]`) for the Neo4j Python driver. ## [v1.0.7] - 2026-02-14 diff --git a/README.md b/README.md index 937a0e2a..c1a41e58 100644 --- a/README.md +++ b/README.md @@ -1,261 +1,192 @@ -Logo -

-

-

-

+Codellm-Devkit logo +

- - + + - - + + - + - - - - - -

-Codellm-Devkit (CLDK) is a multilingual program analysis framework that bridges the gap between traditional static analysis tools and Large Language Models (LLMs) specialized for code (CodeLLMs). Codellm-Devkit allows developers to streamline the process of transforming raw code into actionable insights by providing a unified interface for integrating outputs from various analysis tools and preparing them for effective use by CodeLLMs. - -Codellm-Devkit simplifies the complex process of analyzing codebases that span multiple programming languages, making it easier to extract meaningful insights and drive LLM-based code analysis. `CLDK` achieves this through an open-source Python library that abstracts the intricacies of program analysis and LLM interactions. With this library, developer can streamline the process of transforming raw code into actionable insights by providing a unified interface for integrating outputs from various analysis tools and preparing them for effective use by CodeLLMs. - -**The purpose of Codellm-Devkit is to enable the development and experimentation of robust analysis pipelines that harness the power of both traditional program analysis tools and CodeLLMs.** -By providing a consistent and extensible framework, Codellm-Devkit aims to reduce the friction associated with multi-language code analysis and ensure compatibility across different analysis tools and LLM platforms. - -Codellm-Devkit is designed to integrate seamlessly with a variety of popular analysis tools, such as WALA, Tree-sitter, LLVM, and CodeQL, each implemented in different languages. Codellm-Devkit acts as a crucial intermediary layer, enabling efficient and consistent communication between these tools and the CodeLLMs. +# Codellm-Devkit (CLDK) -Codellm-Devkit is constantly evolving to include new tools and frameworks, ensuring it remains a versatile solution for code analysis and LLM integration. +**A unified, multilingual program-analysis SDK for Code LLMs.** CLDK turns raw source code into structured, LLM-ready program facts — symbol tables, call graphs, type hierarchies, and more — behind a single Python API, so you can build analysis-augmented LLM pipelines without wrangling a different static-analysis tool for every language. -Codellm-Devkit is: +Under the hood, CLDK orchestrates mature analysis engines (WALA, Tree-sitter, Jedi, CodeQL, ts-morph) and normalizes their output into consistent, typed [Pydantic](https://docs.pydantic.dev/) models. You get the same ergonomic interface whether you are analyzing Java, Python, or TypeScript. -- **Unified**: Provides a single framework for integrating multiple analysis tools and CodeLLMs, regardless of the programming languages involved. -- **Extensible**: Designed to support new analysis tools and LLM platforms, making it adaptable to the evolving landscape of code analysis. -- **Streamlined**: Simplifies the process of transforming raw code into structured, LLM-ready inputs, reducing the overhead typically associated with multi-language analysis. +CLDK is: -Codellm-Devkit is an ongoing project, developed at IBM Research. +- **Unified** — one framework and one mental model across languages and analysis backends. +- **Extensible** — designed to take on new languages, engines, and graph backends (e.g. Neo4j). +- **Streamlined** — raw code in, structured LLM-ready facts out, with the tooling complexity hidden. -## Contact +> Developed at IBM Research. CLDK is an actively evolving project — issues and contributions are welcome. -For any questions, feedback, or suggestions, please contact the authors: - -| Name | Email | -| ------------- | --------------------------------------------------- | -| Rahul Krishna | [i.m.ralk@gmail.com](mailto:imralk+oss@gmail.com) | -| Rangeet Pan | [rangeet.pan@ibm.com](mailto:rangeet.pan@gmail.com) | -| Saurabh Sihna | [sinhas@us.ibm.com](mailto:sinhas@us.ibm.com) | ## Table of Contents -- [Contact](#contact) -- [Table of Contents](#table-of-contents) +- [Installation](#installation) - [Quick Start](#quick-start) -- [Architectural and Design Overview](#architectural-and-design-overview) - - [1. **Data Models**](#1-data-models) - - [2. **Analysis Backends**](#2-analysis-backends) - - [Java](#java) - - [Python](#python) - - [C](#c) - - [3. **Utilities and Extensions**](#3-utilities-and-extensions) +- [Supported Languages & Backends](#supported-languages--backends) +- [Architecture](#architecture) +- [Documentation](#documentation) - [Contributing](#contributing) - - [Publication (papers and blogs related to CLDK)](#publication-papers-and-blogs-related-to-cldk) - - -## Quick Start +- [Citation](#citation) +- [Maintainers](#maintainers) -In this section, we will walk through a simple example to demonstrate how to get started with CLDK. +## Installation -1. Install the CLDK package using pip: - - ```bash - pip install cldk - ``` - - -2. To use CLDK, just import the `CLDK` class from the `cldk` module: - - ```python - from cldk import CLDK - ``` - -3. Next, to select a language for analysis, create an instance of the `CLDK` class with the desired language: - - ```python - cldk = CLDK(language="java") # For Java analysis - ``` - -4. Create an analysis object over the Java application by providing the path to the project: +```bash +pip install cldk +``` - ```python - analysis = cldk.analysis(project_path="/path/to/your/java/project") - ``` - This will initialize the analysis pipeline for the specified project. The analysis engine, in the backend, will parse the java project and build a symbol table representing the program structure and return the artifact to CLDK which will map it to the CLDK data schema (`cldk/models/java/models.py`). +Optional extras: - Depending on the size of the project, this step may take some time as it involves parsing, building, and statically analyzing the codebase. +```bash +pip install "cldk[neo4j]" # read-only Neo4j graph backend (Java / Python / TypeScript) +``` -5. Once the analysis is complete, you can call the various methods provided by the `analysis` object to interact with the analyzed codebase. For example, you can retrieve method bodies, signatures, and call graphs. +## Quick Start - ```python - # Iterate over all the files in the project - from CLDK import cldk +Create a language-specific analysis facade with the per-language factory methods, then query it: - analysis: JavaAnalysis = CLDK(language="java").analysis(project_path="/path/to/your/java/project") - - all_files = [file_path for file_path, class_file in analysis.get_symbol_table().items()] +```python +from cldk import CLDK - # Process each file - for file_path in all_files: - # Additional processing can be done here - pass - ``` +# Pick a language — each returns a typed analysis facade. +analysis = CLDK.java(project_path="/path/to/java/project") +# analysis = CLDK.python(project_path="/path/to/python/project") +# analysis = CLDK.typescript(project_path="/path/to/ts/project") +``` - Likewise, you can also retrieve method bodies. +Walk the symbol table and pull method bodies: - ```python - from cldk import CLDK +```python +from cldk import CLDK - analysis: JavaAnalysis = CLDK(language="java").analysis(project_path="/path/to/your/java/project") - for class_file in analysis.get_symbol_table().values(): - for type_name, type_declaration in class_file.type_declarations.items(): - for method in type_declaration.callable_declarations.values(): - method_body = analysis.get_method_body(method.declaration) - print(f"Method: {method.declaration}\nBody: {method_body}\n") - ``` +analysis = CLDK.java(project_path="/path/to/java/project") -## Architectural and Design Overview +for file_path, class_file in analysis.get_symbol_table().items(): + for type_name, type_declaration in class_file.type_declarations.items(): + for method in type_declaration.callable_declarations.values(): + body = analysis.get_method(type_name, method).code + print(f"{type_name}.{method}\n{body}\n") +``` -Below is a very high-level overview of the architectural of CLDK: +Build a call graph by raising the analysis level: +```python +from cldk import CLDK +from cldk.analysis import AnalysisLevel + +analysis = CLDK.java( + project_path="/path/to/java/project", + analysis_level=AnalysisLevel.call_graph, +) +call_graph = analysis.get_call_graph() # a networkx.DiGraph +``` -```mermaid -graph TD -User <--> A[CLDK] - - A --> A1[cldk.analysis] - - A1 --> A2[cldk.analysis.java] - A2 --> A3[codeanalyzer → WALA] - A3 --> JA[Analysis] - - A1 --> A4[cldk.analysis.c] - A4 --> A5[clang] - A5 --> CA[Analysis] - - A1 --> A6[cldk.analysis.python] - A6 --> A7[treesitter_python] - A7 --> PA[Analysis] - - A1 --> A8[cldk.analysis.commons] - A8 --> LSP[LSP] - A8 --> TS[treesitter base] - A8 --> TU[treesitter utils] - - A --> M[cldk.models] - M --> MJ[Java models] - M --> MP[Python models] - M --> MC[C models] - M --> MT[treesitter models] - - A --> U[cldk.utils] - U --> UX[exceptions] - U --> UL[logging] - U --> US[sanitization] - US --> USJ[java sanitization] +Select a backend by passing a typed config. For example, query a pre-populated graph **read-only** +over Neo4j (no source or analyzer run needed) — available for all three languages: +```python +from cldk import CLDK +from cldk.analysis.commons.backend_config import Neo4jConnectionConfig + +analysis = CLDK.java( + backend=Neo4jConnectionConfig( + uri="bolt://localhost:7687", + application_name="my-app", # the graph is populated out of band + ), +) +classes = analysis.get_all_classes() ``` -The user interacts with the CLDK API via the top-level `CLDK` interface exposed in `core.py`. This interface is responsible for configuring the analysis session, initializing language-specific pipelines, and exposing a high-level, language-agnostic API for interacting with program structure and semantics. - -CLDK is currently implemented with full support for **Java**, **Python**, and **C**. Each language module is structured around two core components: **data models** and **analysis backends**. +> **Deprecation:** the old `CLDK(language="java").analysis(...)` entry point still works as a thin compatibility shim (it emits a `DeprecationWarning`). Prefer the `CLDK.java()` / `CLDK.python()` / `CLDK.typescript()` factory methods. +## Supported Languages & Backends -### 1. **Data Models** +Each language is analyzed by a dedicated `codeanalyzer-*` engine; CLDK normalizes the result into typed models exposed through the same API. All three also support an optional **read-only Neo4j backend** — pass a `Neo4jConnectionConfig` and the SDK answers the same queries with Cypher over a graph the analyzer populates out of band (`--emit neo4j`). -Each supported language has its own set of Pydantic-based data models, located in the `cldk.models` module (e.g., `cldk.models.java`, `cldk.models.python`, `cldk.models.c`). These models provide: +| Language | Analysis engine | What it provides | +| --- | --- | --- | +| **Java** | [`codeanalyzer-java`](https://github.com/codellm-devkit/codeanalyzer-java) | WALA + JavaParser. Bytecode-level call graphs, type hierarchies, symbol resolution, CRUD-operation and entry-point detection. Optional read-only **Neo4j** graph backend. | +| **Python** | [`codeanalyzer-python`](https://github.com/codellm-devkit/codeanalyzer-python) | Jedi with optional CodeQL augmentation. Symbol tables, call graphs, and class/method resolution. Optional read-only **Neo4j** graph backend. | +| **TypeScript / JavaScript** | [`codeanalyzer-typescript`](https://github.com/codellm-devkit/codeanalyzer-typescript) | ts-morph with Jelly-based call graphs. Symbols, call graph, types, decorators, and call sites. Optional read-only **Neo4j** graph backend. | -- **Structured representations** of language elements such as classes, methods, annotations, fields, and statements. -- **Typed access** using dot notation (e.g., `method.return_type` or `klass.methods`), promoting developer productivity. -- **Serialization support** to and from JSON and other formats, enabling easy storage, inspection, and exchange of analysis results. -- **Consistency** across languages via shared modeling conventions and base abstractions, including a common treesitter schema. +The backend is selected by the **type** of the `backend=` config you pass to a factory: the in-process analyzer (default) or a `Neo4jConnectionConfig` for the read-only graph backend. +## Architecture +The user interacts only with the top-level `CLDK` interface (`core.py`), which configures the session, initializes the language-specific pipeline, and exposes a high-level, language-agnostic API. Each language module is built from two pieces: **data models** and an **analysis backend**. -### 2. **Analysis Backends** +```mermaid +graph TD + User <--> CLDK + CLDK --> M[cldk.models
typed Pydantic schemas] + CLDK --> A[cldk.analysis] -Each language has a dedicated analysis backend implemented under `cldk.analysis.`, responsible for coordinating concrete analysis steps using language-specific tooling: + A --> J[cldk.analysis.java] + A --> P[cldk.analysis.python] + A --> T[cldk.analysis.typescript] -#### Java -- **Backend:** `cldk.analysis.java` -- **Tools:** JavaParser, WALA (via CodeAnalyzer JAR) -- **Capabilities:** Bytecode-level call graphs, symbol resolution, method declarations, type hierarchies + J --> EJ[codeanalyzer-java
WALA · JavaParser] + P --> EP[codeanalyzer-python
Jedi · CodeQL] + T --> ET[codeanalyzer-typescript
ts-morph · Jelly] -#### Python -- **Backend:** `cldk.analysis.python` -- **Tools:** `codeanalyzer-python` (Jedi + CodeQL, default on), Tree-sitter for source-level parsing -- **Capabilities:** Symbol table, call graph, class/method resolution, comments/docstrings + J -. read-only .-> N[(Neo4j)] + P -. read-only .-> N + T -. read-only .-> N +``` -> **Note — analysis cache:** Caching is owned entirely by -> `codeanalyzer-python`; CLDK keeps no cache of its own. Artifacts (the -> backend virtualenv, CodeQL database, and `analysis_cache.json`) live under -> the backend's `cache_dir`, which defaults to `/.codeanalyzer` and -> can be redirected with the `cache_dir` argument. **CodeQL is enabled by -> default** (`use_codeql=True`), so the first analysis of a project builds a -> CodeQL database and provisions the CodeQL CLI — expect a slow cold run; -> subsequent runs reuse the backend's checksum-validated cache. Pass -> `use_codeql=False` for Jedi-only analysis. Add the `cache_dir` location -> (e.g. `.codeanalyzer/`) to your `.gitignore` — it is large and -> environment-specific. +**Data models** — each language has its own set of Pydantic models under `cldk.models` (`cldk.models.java`, `cldk.models.python`, `cldk.models.typescript`). They give you structured, typed, dot-accessible representations of classes, methods, fields, and statements, with JSON serialization and shared conventions across languages. -#### C -- **Backend:** `cldk.analysis.c` -- **Tools:** Clang frontend -- **Capabilities:** Structural symbol resolution and method/function layout using Clang AST +**Analysis backends** — each language has a backend under `cldk.analysis.` that coordinates its engine (see the table above) and maps the result onto the data models. The read-only Neo4j backends (`cldk.analysis..neo4j`) reconstruct the *same* models from a Cypher graph, so they are drop-in interchangeable with the in-process analyzers. Backends are orchestrated internally; you only call high-level methods such as `get_symbol_table()`, `get_method(...)`, and `get_call_graph(...)`, and CLDK handles tool coordination, parsing, and marshalling under the hood. -All analysis backends share common infrastructure defined in `cldk.analysis.commons`, including: -- **Tree-sitter utilities** (`treesitter_java`, `treesitter_python`) -- **LSP integration hooks** -- **Generic model builders and transformation utilities** +## Documentation -Backends are internally orchestrated such that the user does not interact with them directly. Instead, they simply call high-level SDK methods such as: +Full documentation lives at **[codellm-devkit.info](https://codellm-devkit.info)**. -```python -get_method_body(...) -get_method_signature(...) -get_call_graph(...) -``` +## Contributing -CLDK handles tool coordination, language resolution, parsing, transformation, and data marshalling under the hood. +We welcome contributors of all experience levels — see the [CONTRIBUTING](./CONTRIBUTING.md) guide to get started. ---- +## Citation -### 3. **Utilities and Extensions** +If you use CLDK in your research, please cite: -The `cldk.utils` module provides additional support functionality: -- **Exception handling utilities** -- **Logging configuration** -- **Sanitization logic** (especially for Java, via `sanitization.java.treesitter_sanitizer`) +```bibtex +@article{krishna2024codellm, + title = {Codellm-Devkit: A Framework for Contextualizing Code LLMs with Program Analysis Insights}, + author = {Krishna, Rahul and Pan, Rangeet and Pavuluri, Raju and Tamilselvam, Srikanth and Vukovic, Maja and Sinha, Saurabh}, + journal = {arXiv preprint arXiv:2410.13007}, + year = {2024} +} +``` -These modules ensure robustness and clean error management across backend interactions and user API layers. +Related publications: -## Contributing +1. Pan, Rangeet, Myeongsoo Kim, Rahul Krishna, Raju Pavuluri, and Saurabh Sinha. "[Multi-language Unit Test Generation using LLMs.](https://arxiv.org/abs/2409.03093)" arXiv preprint arXiv:2409.03093 (2024). +2. Pan, Rangeet, Rahul Krishna, Raju Pavuluri, Saurabh Sinha, and Maja Vukovic. "[Simplify your Code LLM solutions using CodeLLM Dev Kit (CLDK).](https://www.linkedin.com/pulse/simplify-your-code-llm-solutions-using-codellm-dev-kit-rangeet-pan-vnnpe/)" Blog. -We invite contributors of all levels of experience! We would love to see you get involved in the project. See the [CONTRIBUTING](./CONTRIBUTING.md) guide to get started. +## Maintainers +| Name | Email | +| --- | --- | +| Rahul Krishna | [i.m.ralk@gmail.com](mailto:imralk+oss@gmail.com) | +| Rangeet Pan | [rangeet.pan@ibm.com](mailto:rangeet.pan@gmail.com) | +| Saurabh Sinha | [sinhas@us.ibm.com](mailto:sinhas@us.ibm.com) | -### Publication (papers and blogs related to CLDK) -1. Krishna, Rahul, Rangeet Pan, Raju Pavuluri, Srikanth Tamilselvam, Maja Vukovic, and Saurabh Sinha. "[Codellm-Devkit: A Framework for Contextualizing Code LLMs with Program Analysis Insights.](https://arxiv.org/pdf/2410.13007)" arXiv preprint arXiv:2410.13007 (2024). -2. Pan, Rangeet, Myeongsoo Kim, Rahul Krishna, Raju Pavuluri, and Saurabh Sinha. "[Multi-language Unit Test Generation using LLMs.](https://arxiv.org/abs/2409.03093)" arXiv preprint arXiv:2409.03093 (2024). -3. Pan, Rangeet, Rahul Krishna, Raju Pavuluri, Saurabh Sinha, and Maja Vukovic., "[Simplify your Code LLM solutions using CodeLLM Dev Kit (CLDK).](https://www.linkedin.com/pulse/simplify-your-code-llm-solutions-using-codellm-dev-kit-rangeet-pan-vnnpe/?trackingId=kZ3U6d8GSDCs8S1oApXZgg%3D%3D)", Blog. +Licensed under the [Apache License 2.0](./LICENSE). diff --git a/cldk/analysis/commons/backend_config.py b/cldk/analysis/commons/backend_config.py index 3806aeaa..a84050ff 100644 --- a/cldk/analysis/commons/backend_config.py +++ b/cldk/analysis/commons/backend_config.py @@ -99,9 +99,8 @@ class Neo4jConnectionConfig: application_name: str | None = None -# Per-language discriminated unions the facades match on. Java has no Neo4j backend yet, so its -# only admissible config is the codeanalyzer one. -JavaBackend = CodeAnalyzerConfig +# Per-language discriminated unions the facades match on. +JavaBackend = Union[CodeAnalyzerConfig, Neo4jConnectionConfig] PyBackend = Union[PyCodeAnalyzerConfig, Neo4jConnectionConfig] TSBackend = Union[CodeAnalyzerConfig, Neo4jConnectionConfig] diff --git a/cldk/analysis/java/java_analysis.py b/cldk/analysis/java/java_analysis.py index 1f6f946f..60707b14 100644 --- a/cldk/analysis/java/java_analysis.py +++ b/cldk/analysis/java/java_analysis.py @@ -52,12 +52,13 @@ from tree_sitter import Tree -from cldk.analysis.commons.backend_config import CodeAnalyzerConfig, JavaBackend, cache_subdir +from cldk.analysis.commons.backend_config import CodeAnalyzerConfig, JavaBackend, Neo4jConnectionConfig, cache_subdir from cldk.analysis.commons.treesitter import TreesitterJava from cldk.models.java import JCallable from cldk.models.java import JApplication from cldk.models.java.models import JCRUDOperation, JComment, JCompilationUnit, JMethodDetail, JType, JField from cldk.analysis.java.codeanalyzer import JCodeanalyzer +from cldk.analysis.java.neo4j import JNeo4jBackend from cldk.analysis.java.backend import JavaAnalysisBackend @@ -149,22 +150,33 @@ def __init__( self.eager_analysis = eager_analysis self.target_files = target_files self.backend_config: JavaBackend = backend if backend is not None else CodeAnalyzerConfig() - # Java has a single backend family; the config only carries the cache root. analysis.json - # is cached under /java (None in source_code mode, where the analyzer streams - # results over a pipe). - cache_path = cache_subdir(self.backend_config.cache_dir, project_dir, "java") - if cache_path is not None: - cache_path.mkdir(parents=True, exist_ok=True) self.treesitter_java: TreesitterJava = TreesitterJava() - # Initialize the analysis backend - self.backend: JavaAnalysisBackend = JCodeanalyzer( - project_dir=self.project_dir, - source_code=self.source_code, - eager_analysis=self.eager_analysis, - analysis_level=self.analysis_level, - analysis_json_path=cache_path, - target_files=self.target_files, - ) + self.backend: JavaAnalysisBackend + if isinstance(self.backend_config, Neo4jConnectionConfig): + # Read-only: the graph is populated out of band; the SDK only polls it. + cfg = self.backend_config + application_name = cfg.application_name or (Path(project_dir).name if project_dir else None) + self.backend = JNeo4jBackend( + neo4j_uri=cfg.uri, + neo4j_username=cfg.username, + neo4j_password=cfg.password, + neo4j_database=cfg.database, + application_name=application_name, + ) + else: + # The config only carries the cache root. analysis.json is cached under /java + # (None in source_code mode, where the analyzer streams results over a pipe). + cache_path = cache_subdir(self.backend_config.cache_dir, project_dir, "java") + if cache_path is not None: + cache_path.mkdir(parents=True, exist_ok=True) + self.backend = JCodeanalyzer( + project_dir=self.project_dir, + source_code=self.source_code, + eager_analysis=self.eager_analysis, + analysis_level=self.analysis_level, + analysis_json_path=cache_path, + target_files=self.target_files, + ) def get_imports(self) -> List[str]: """Return all import statements in the source code. diff --git a/cldk/analysis/java/neo4j/__init__.py b/cldk/analysis/java/neo4j/__init__.py new file mode 100644 index 00000000..fc0a3cbe --- /dev/null +++ b/cldk/analysis/java/neo4j/__init__.py @@ -0,0 +1,22 @@ +################################################################################ +# Copyright IBM Corporation 2026 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +"""Read-only Neo4j-backed Java analysis backend (Cypher queries over the codeanalyzer-java graph).""" + +from cldk.analysis.java.neo4j.config import Neo4jConnectionConfig +from cldk.analysis.java.neo4j.neo4j_backend import JNeo4jBackend + +__all__ = ["JNeo4jBackend", "Neo4jConnectionConfig"] diff --git a/cldk/analysis/java/neo4j/config.py b/cldk/analysis/java/neo4j/config.py new file mode 100644 index 00000000..3ed20d43 --- /dev/null +++ b/cldk/analysis/java/neo4j/config.py @@ -0,0 +1,27 @@ +################################################################################ +# Copyright IBM Corporation 2026 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +"""Connection settings for the read-only Neo4j-backed Java analysis backend. + +The definition has been hoisted to :mod:`cldk.analysis.commons.backend_config`; it is re-exported +here for symmetry with the Python and TypeScript backends. +""" + +from __future__ import annotations + +from cldk.analysis.commons.backend_config import Neo4jConnectionConfig + +__all__ = ["Neo4jConnectionConfig"] diff --git a/cldk/analysis/java/neo4j/neo4j_backend.py b/cldk/analysis/java/neo4j/neo4j_backend.py new file mode 100644 index 00000000..e992f7fa --- /dev/null +++ b/cldk/analysis/java/neo4j/neo4j_backend.py @@ -0,0 +1,707 @@ +################################################################################ +# Copyright IBM Corporation 2026 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +"""Neo4j-backed Java analysis backend (read-only Cypher client). + +A drop-in alternative to :class:`~cldk.analysis.java.codeanalyzer.JCodeanalyzer`: it exposes the +**same query method surface** (the 36 methods of :class:`JavaAnalysisBackend`) so the +:class:`~cldk.analysis.java.JavaAnalysis` facade can delegate to either one, but instead of running +the analyzer JAR it **reconstructs the canonical ``JApplication`` from a Neo4j graph** (the one +``codeanalyzer-java`` >= 2.4.0 emits with ``--emit neo4j``) and then answers every query with the +*identical* logic the in-memory backend uses. Mirrors the Python / TypeScript Neo4j backends. + +It is purely a **query client**: it never builds the graph and has no dependency on the analyzer JAR, +a JDK, or the project sources. The graph is populated out of band — e.g. a job running +``codeanalyzer-java --emit neo4j`` — and the SDK only polls it. + +Reconstruction strategy (see :mod:`reconstruct`): the backend bulk-fetches every node + relationship +for the application in a handful of Cypher queries, groups children by parent, builds an +``analysis.json``-shaped dict, and hands it to ``JApplication(**payload)`` — the same constructor +path as ``JCodeanalyzer._init_japplication``. With ``self.application`` and ``self.call_graph`` +populated, the 36 query methods are the same code the in-memory backend runs. + +Identity / scoping model (must match the emitter; see ``codeanalyzer-java/schema.neo4j.json``): +``:JType`` (id = fqn) and ``:JCallable`` (id = ``#``) share a ``:JSymbol`` label; +compilation units are ``:JCompilationUnit`` keyed by ``file_key`` (== file path == symbol-table key); +call edges are ``(:JCallable)-[:J_CALLS {type, weight, source_kind, destination_kind}]->(:JCallable)``; +every project-owned node carries a ``_module`` provenance prop, so one DB can host several apps, all +scoped under ``(:JApplication {name})-[:J_HAS_UNIT]->(:JCompilationUnit)``. + +Parity: this backend reconstructs everything the graph actually contains identically to the +in-memory ``JCodeanalyzer`` (verified on the daytrader8 sample). It cannot recover what the +``codeanalyzer-java`` (2.4.0) emitter drops, however — three known producer-side gaps make the graph +an incomplete projection (tracked upstream, NOT query-layer bugs): + +* every ``:JField`` is emitted with the id ``#field#null``, so all fields of a class collapse to + one node (codeanalyzer-java#156); +* imports are projected to ``:JPackage`` only, losing the imported type name (codeanalyzer-java#157); +* ``J_CALLS`` materializes only a small fraction of the call graph — edges are absent even when both + endpoint callables are present as nodes (codeanalyzer-java#158). + +Projection-lossy-by-design: a ``:JType``'s ``is_class_or_interface_declaration`` / +``is_concrete_class`` flags are not projected (only the ``kind`` discriminator is). +""" + +from __future__ import annotations + +import json +import logging +from itertools import chain, groupby +from typing import Any, Dict, List, Tuple, Union + +import networkx as nx + +from cldk.analysis.commons.treesitter import TreesitterJava +from cldk.analysis.java.backend import JavaAnalysisBackend +from cldk.analysis.java.neo4j import reconstruct as R +from cldk.models.java import JGraphEdges +from cldk.models.java.enums import CRUDOperationType +from cldk.models.java.models import JApplication, JCRUDOperation, JCallable, JCallableParameter, JComment, JField, JMethodDetail, JType, JCompilationUnit, JGraphEdgesST +from cldk.utils.exceptions.exceptions import CodeanalyzerExecutionException + +logger = logging.getLogger(__name__) + + +class JNeo4jBackend(JavaAnalysisBackend): + """Query the application view of a Java project over Neo4j (Cypher), read-only. + + Args: + neo4j_uri: Bolt URI of the Neo4j server (e.g. ``bolt://localhost:7687``). + neo4j_username / neo4j_password: Credentials (read-only is sufficient). + neo4j_database: Database name (None ⇒ server default). + application_name: The ``:JApplication`` anchor name to scope every query to. Matches the + ``--app-name`` the graph was loaded with (defaults to the project directory name). + """ + + def __init__( + self, + neo4j_uri: str, + neo4j_username: str, + neo4j_password: str, + neo4j_database: str | None = None, + application_name: str | None = None, + ) -> None: + try: + from neo4j import GraphDatabase + except ModuleNotFoundError as e: # pragma: no cover - import guard + raise CodeanalyzerExecutionException( + "The Neo4j backend requires the 'neo4j' driver. Install it with " + "`pip install neo4j` (or `pip install cldk[neo4j]`)." + ) from e + + if not application_name: + raise CodeanalyzerExecutionException("application_name is required to scope queries to an application.") + self.application_name = application_name + self._database = neo4j_database + self._driver = GraphDatabase.driver(neo4j_uri, auth=(neo4j_username, neo4j_password)) + + self._units: List[str] = self._load_unit_keys() + self.application: JApplication = self._reconstruct_application() + self.analysis_level = "call_graph" if self.application.call_graph else "symbol_table" + self.call_graph: nx.DiGraph | None = self._generate_call_graph(using_symbol_table=False) if self.application.call_graph else None + + # -----[ lifecycle ]----- + def close(self) -> None: + """Close the underlying Neo4j driver.""" + self._driver.close() + + def __enter__(self) -> "JNeo4jBackend": + return self + + def __exit__(self, *exc: Any) -> None: + self.close() + + def _run(self, query: str, **params: Any) -> List[Dict[str, Any]]: + with self._driver.session(database=self._database) as session: + return [record.data() for record in session.run(query, **params)] + + def _load_unit_keys(self) -> List[str]: + rows = self._run( + "MATCH (:JApplication {name: $app})-[:J_HAS_UNIT]->(u:JCompilationUnit) RETURN u.file_key AS k", + app=self.application_name, + ) + return [r["k"] for r in rows] + + # ===================================================================================== + # Reconstruction: bulk-fetch the graph and rebuild the canonical JApplication. + # ===================================================================================== + def _nodes(self, label: str) -> Dict[str, Dict[str, Any]]: + """All nodes of a label owned by this app, keyed by id/file_key/name.""" + rows = self._run( + f"MATCH (n:{label}) WHERE n._module IN $u RETURN coalesce(n.id, n.file_key, n.name) AS k, properties(n) AS p", + u=self._units, + ) + return {r["k"]: r["p"] for r in rows} + + def _adj(self, rtype: str, scope_child: bool = True) -> Dict[str, List[str]]: + """Adjacency parent_key → [child_keys] for a relationship, scoped to this app.""" + where = "b._module IN $u" if scope_child else "a._module IN $u" + rows = self._run( + f"MATCH (a)-[:{rtype}]->(b) WHERE {where} " + "RETURN coalesce(a.id, a.file_key, a.name) AS a, coalesce(b.id, b.file_key, b.name) AS b", + u=self._units, + ) + out: Dict[str, List[str]] = {} + for r in rows: + out.setdefault(r["a"], []).append(r["b"]) + return out + + def _reconstruct_application(self) -> JApplication: + units = self._units + # ---- node prop maps ---- + cu_nodes = { + r["k"]: r["p"] + for r in self._run( + "MATCH (:JApplication {name: $app})-[:J_HAS_UNIT]->(u:JCompilationUnit) RETURN u.file_key AS k, properties(u) AS p", + app=self.application_name, + ) + } + types = self._nodes("JType") + callables = self._nodes("JCallable") + fields = self._nodes("JField") + params = self._nodes("JParameter") + callsites = self._nodes("JCallSite") + variables = self._nodes("JVariable") + enums = self._nodes("JEnumConstant") + records = self._nodes("JRecordComponent") + initblocks = self._nodes("JInitializationBlock") + crudops = self._nodes("JCrudOperation") + crudqs = self._nodes("JCrudQuery") + comments = self._nodes("JComment") + + # ---- adjacencies ---- + a_callable = self._adj("J_HAS_CALLABLE") + a_field = self._adj("J_HAS_FIELD") + a_enum = self._adj("J_HAS_ENUM_CONSTANT") + a_record = self._adj("J_HAS_RECORD_COMPONENT") + a_init = self._adj("J_HAS_INIT_BLOCK") + a_param = self._adj("J_HAS_PARAMETER") + a_callsite = self._adj("J_HAS_CALLSITE") + a_var = self._adj("J_DECLARES_VAR") + a_crudop = self._adj("J_HAS_CRUD_OPERATION") + a_crudq = self._adj("J_HAS_CRUD_QUERY") + a_comment = self._adj("J_HAS_COMMENT") + a_import = self._run( + "MATCH (u:JCompilationUnit)-[r:J_IMPORTS]->(t) WHERE u._module IN $u " + "RETURN u.file_key AS cu, coalesce(t.fqn, t.name) AS path, properties(r) AS p", + u=units, + ) + + # ---- ordered helpers ---- + def _comments_of(owner_id: str) -> List[dict]: + ids = a_comment.get(owner_id, []) + built = [R.comment(comments[i]) for i in ids if i in comments] + return sorted(built, key=lambda c: (c["start_line"], c["start_column"])) + + def _first_comment(owner_id: str) -> dict | None: + cs = _comments_of(owner_id) + return cs[0] if cs else None + + def _param_index(pid: str) -> int: + try: + return int(pid.rsplit("#param#", 1)[1]) + except (IndexError, ValueError): + return 0 + + def _build_callsite(cs_id: str) -> dict: + p = callsites[cs_id] + op_ids = a_crudop.get(cs_id, []) + q_ids = a_crudq.get(cs_id, []) + crud_op = R.crud_operation(crudops[op_ids[0]]) if op_ids and op_ids[0] in crudops else None + crud_q = R.crud_query(crudqs[q_ids[0]]) if q_ids and q_ids[0] in crudqs else None + return R.callsite(p, comment_node=_first_comment(cs_id), crud_op=crud_op, crud_q=crud_q) + + def _callsites_of(owner_id: str) -> List[dict]: + ids = a_callsite.get(owner_id, []) + built = [(callsites[i], _build_callsite(i)) for i in ids if i in callsites] + return [cs for _, cs in sorted(built, key=lambda t: (t[0].get("start_line", -1), t[0].get("start_column", -1)))] + + def _vars_of(owner_id: str) -> List[dict]: + ids = a_var.get(owner_id, []) + built = [(variables[i], R.variable(variables[i], comment_node=_first_comment(i))) for i in ids if i in variables] + return [v for _, v in sorted(built, key=lambda t: (t[0].get("start_line", -1), t[0].get("name", "")))] + + # ---- callables ---- + def _build_callable(cid: str) -> dict: + p = callables[cid] + pids = sorted(a_param.get(cid, []), key=_param_index) + parameters = [R.parameter(params[i]) for i in pids if i in params] + op_ids = a_crudop.get(cid, []) + q_ids = a_crudq.get(cid, []) + crud_ops = [R.crud_operation(crudops[i]) for i in op_ids if i in crudops] + crud_qs = [R.crud_query(crudqs[i]) for i in q_ids if i in crudqs] + return R.callable_( + p, + comments=_comments_of(cid), + parameters=parameters, + call_sites=_callsites_of(cid), + variable_declarations=_vars_of(cid), + crud_operations=crud_ops, + crud_queries=crud_qs, + ) + + def _build_initblock(ib_id: str) -> dict: + p = initblocks[ib_id] + return R.init_block(p, comments=_comments_of(ib_id), call_sites=_callsites_of(ib_id), variable_declarations=_vars_of(ib_id)) + + # ---- types ---- + def _build_type(tid: str) -> dict: + p = types[tid] + cdecls = {} + for cid in a_callable.get(tid, []): + if cid in callables: + cdecls[callables[cid].get("signature", cid)] = _build_callable(cid) + fdecls = [R.field(fields[i], comment_node=_first_comment(i)) for i in a_field.get(tid, []) if i in fields] + econsts = [R.enum_constant(enums[i]) for i in a_enum.get(tid, []) if i in enums] + rcomps = [R.record_component(records[i], comment_node=_first_comment(i)) for i in a_record.get(tid, []) if i in records] + iblocks = [_build_initblock(i) for i in a_init.get(tid, []) if i in initblocks] + return R.type_( + p, + comments=_comments_of(tid), + callable_declarations=cdecls, + field_declarations=fdecls, + enum_constants=econsts, + record_components=rcomps, + initialization_blocks=iblocks, + ) + + # group types by owning module (file_key); type_declarations is a flat per-CU map + types_by_unit: Dict[str, Dict[str, dict]] = {} + for tid, tp in types.items(): + fkey = tp.get("_module") + fqn = tp.get("fqn", tid) + types_by_unit.setdefault(fkey, {})[fqn] = _build_type(tid) + + # imports by unit + imports_by_unit: Dict[str, List[dict]] = {} + for r in a_import: + imports_by_unit.setdefault(r["cu"], []).append( + {"path": r["path"], "is_static": r["p"].get("is_static", False), "is_wildcard": r["p"].get("is_wildcard", False)} + ) + + # ---- compilation units / symbol table ---- + symbol_table: Dict[str, dict] = {} + for fkey, cp in cu_nodes.items(): + symbol_table[fkey] = R.compilation_unit( + cp, + comments=_comments_of(fkey), + import_declarations=imports_by_unit.get(fkey, []), + type_declarations=types_by_unit.get(fkey, {}), + ) + + # ---- call graph edges ---- + call_edges: List[dict] = [] + for r in self._run( + "MATCH (s:JCallable)-[c:J_CALLS]->(t:JCallable) WHERE s._module IN $u " + "RETURN s.id AS src, t.id AS tgt, properties(c) AS p", + u=units, + ): + src = self._endpoint(r["src"], callables) + tgt = self._endpoint(r["tgt"], callables) + if src and tgt: + call_edges.append(R.call_edge(src, tgt, r["p"])) + + return JApplication(symbol_table=symbol_table, call_graph=call_edges) + + @staticmethod + def _endpoint(node_id: str, callables: Dict[str, Dict[str, Any]]) -> dict | None: + """A J_CALLS endpoint id (``#``) → a JGraphEdges source/target dict.""" + if "#" not in node_id: + return None + fqn, signature = node_id.split("#", 1) + props = callables.get(node_id, {}) + declaration = props.get("declaration") or signature + if "(" not in declaration: + declaration = signature + return {"file_path": props.get("file_path", ""), "type_declaration": fqn, "signature": signature, "callable_declaration": declaration} + + # ===================================================================================== + # JavaAnalysisBackend — leaf accessors (served from the reconstructed application) + # ===================================================================================== + def get_application_view(self) -> JApplication: + return self.application + + def get_symbol_table(self) -> Dict[str, JCompilationUnit]: + return self.application.symbol_table + + def get_system_dependency_graph(self) -> list[JGraphEdges]: + return self.application.call_graph or [] + + def get_compilation_units(self) -> List[JCompilationUnit]: + return list(self.application.symbol_table.values()) + + def get_java_compilation_unit(self, file_path: str) -> JCompilationUnit: + return self.application.symbol_table[file_path] + + # ===================================================================================== + # Call graph (logic mirrors JCodeanalyzer; calling_lines recomputed from JCallable.code) + # ===================================================================================== + def _generate_call_graph(self, using_symbol_table) -> nx.DiGraph: + cg = nx.DiGraph() + if using_symbol_table: + NotImplementedError("Call graph generation using symbol table is not implemented yet.") + else: + sdg = self.get_system_dependency_graph() + tsu = TreesitterJava() + edge_list = [ + ( + (jge.source.method.signature, jge.source.klass), + (jge.target.method.signature, jge.target.klass), + { + "type": jge.type, + "weight": jge.weight, + "calling_lines": ( + tsu.get_calling_lines(jge.source.method.code, jge.target.method.signature) + if not jge.source.method.is_implicit or not jge.target.method.is_implicit + else [] + ), + }, + ) + for jge in sdg + if jge.type == "CALL_DEP" + ] + for jge in sdg: + cg.add_node((jge.source.method.signature, jge.source.klass), method_detail=jge.source) + cg.add_node((jge.target.method.signature, jge.target.klass), method_detail=jge.target) + cg.add_edges_from(edge_list) + return cg + + def get_call_graph(self) -> nx.DiGraph: + if self.analysis_level == "symbol_table": + self.call_graph = self._generate_call_graph(using_symbol_table=True) + if self.call_graph is None: + self.call_graph = self._generate_call_graph(using_symbol_table=False) + return self.call_graph + + def get_call_graph_json(self) -> str: + callgraph_list = [] + edges = list(self.call_graph.edges.data("calling_lines")) + for edge in edges: + callgraph_dict = {} + callgraph_dict["source_method_signature"] = edge[0][0] + callgraph_dict["source_method_body"] = self.call_graph.nodes[edge[0]]["method_detail"].method.code + callgraph_dict["source_class"] = edge[0][1] + callgraph_dict["target_method_signature"] = edge[1][0] + callgraph_dict["target_method_body"] = self.call_graph.nodes[edge[1]]["method_detail"].method.code + callgraph_dict["target_class"] = edge[1][1] + callgraph_dict["calling_lines"] = edge[2] + callgraph_list.append(callgraph_dict) + return json.dumps(callgraph_list) + + def get_all_callers(self, target_class_name: str, target_method_signature: str, using_symbol_table: bool) -> Dict: + caller_detail_dict = {} + if using_symbol_table: + call_graph = self.__call_graph_using_symbol_table(qualified_class_name=target_class_name, method_signature=target_method_signature, is_target_method=True) + else: + call_graph = self.call_graph + if (target_method_signature, target_class_name) not in call_graph.nodes(): + return caller_detail_dict + in_edge_view = call_graph.in_edges(nbunch=(target_method_signature, target_class_name), data=True) + caller_detail_dict["caller_details"] = [] + caller_detail_dict["target_method"] = call_graph.nodes[(target_method_signature, target_class_name)]["method_detail"] + for source, target, data in in_edge_view: + cm = {"caller_method": call_graph.nodes[source]["method_detail"], "calling_lines": data["calling_lines"]} + caller_detail_dict["caller_details"].append(cm) + return caller_detail_dict + + def get_all_callees(self, source_class_name: str, source_method_signature: str, using_symbol_table: bool) -> Dict: + callee_detail_dict = {} + if using_symbol_table: + call_graph = self.__call_graph_using_symbol_table(qualified_class_name=source_class_name, method_signature=source_method_signature) + else: + call_graph = self.call_graph + if (source_method_signature, source_class_name) not in call_graph.nodes(): + return callee_detail_dict + out_edge_view = call_graph.out_edges(nbunch=(source_method_signature, source_class_name), data=True) + callee_detail_dict["callee_details"] = [] + callee_detail_dict["source_method"] = call_graph.nodes[(source_method_signature, source_class_name)]["method_detail"] + for source, target, data in out_edge_view: + cm = {"callee_method": call_graph.nodes[target]["method_detail"], "calling_lines": data["calling_lines"]} + callee_detail_dict["callee_details"].append(cm) + return callee_detail_dict + + # ===================================================================================== + # Classes / methods / fields (operate on the reconstructed symbol table) + # ===================================================================================== + def get_all_methods_in_application(self) -> Dict[str, Dict[str, JCallable]]: + class_method_dict = {} + class_dict = self.get_all_classes() + for k, v in class_dict.items(): + class_method_dict[k] = v.callable_declarations + return class_method_dict + + def get_all_classes(self) -> Dict[str, JType]: + class_dict = {} + for v in self.get_symbol_table().values(): + class_dict.update(v.type_declarations) + return class_dict + + def get_class(self, qualified_class_name) -> JType: + for v in self.get_symbol_table().values(): + if qualified_class_name in v.type_declarations.keys(): + return v.type_declarations.get(qualified_class_name) + + def get_method(self, qualified_class_name, method_signature) -> JCallable: + for v in self.get_symbol_table().values(): + if qualified_class_name in v.type_declarations.keys(): + ci = v.type_declarations[qualified_class_name] + for cd in ci.callable_declarations.keys(): + if cd == method_signature: + return ci.callable_declarations[cd] + + def get_method_parameters(self, qualified_class_name, method_signature) -> List[JCallableParameter]: + return self.get_method(qualified_class_name, method_signature).parameters + + def get_java_file(self, qualified_class_name) -> str: + for k, v in self.get_symbol_table().items(): + if qualified_class_name in v.type_declarations.keys(): + return k + + def get_all_methods_in_class(self, qualified_class_name) -> Dict[str, JCallable]: + ci = self.get_class(qualified_class_name) + if ci is None: + return {} + return {k: v for (k, v) in ci.callable_declarations.items() if v.is_constructor is False} + + def get_all_constructors(self, qualified_class_name) -> Dict[str, JCallable]: + ci = self.get_class(qualified_class_name) + if ci is None: + return {} + return {k: v for (k, v) in ci.callable_declarations.items() if v.is_constructor is True} + + def get_all_sub_classes(self, qualified_class_name) -> Dict[str, JType]: + all_classes = self.get_all_classes() + sub_classes = {} + for cls in all_classes: + if qualified_class_name in all_classes[cls].implements_list or qualified_class_name in all_classes[cls].extends_list: + sub_classes[cls] = all_classes[cls] + return sub_classes + + def get_all_fields(self, qualified_class_name) -> List[JField]: + ci = self.get_class(qualified_class_name) + if ci is None: + logging.warning(f"Class {qualified_class_name} not found in the application view.") + return list() + return ci.field_declarations + + def get_all_nested_classes(self, qualified_class_name) -> List[JType]: + ci = self.get_class(qualified_class_name) + if ci is None: + logging.warning(f"Class {qualified_class_name} not found in the application view.") + return list() + return [self.get_class(c) for c in ci.nested_type_declarations] + + def get_extended_classes(self, qualified_class_name) -> List[str]: + ci = self.get_class(qualified_class_name) + if ci is None: + logging.warning(f"Class {qualified_class_name} not found in the application view.") + return list() + return ci.extends_list + + def get_implemented_interfaces(self, qualified_class_name) -> List[str]: + ci = self.get_class(qualified_class_name) + if ci is None: + logging.warning(f"Class {qualified_class_name} not found in the application view.") + return list() + return ci.implements_list + + # ===================================================================================== + # Symbol-table call graph (pure-Python over call sites; mirrors JCodeanalyzer) + # ===================================================================================== + def get_class_call_graph_using_symbol_table(self, qualified_class_name: str, method_signature: str | None = None) -> List[Tuple[JMethodDetail, JMethodDetail]]: + call_graph = self.__call_graph_using_symbol_table(qualified_class_name, method_signature) + if method_signature is None: + filter_criteria = {node for node in call_graph.nodes if node[1] == qualified_class_name} + else: + filter_criteria = {node for node in call_graph.nodes if tuple(node) == (method_signature, qualified_class_name)} + graph_edges: List[Tuple[JMethodDetail, JMethodDetail]] = list() + for edge in call_graph.edges(nbunch=filter_criteria): + source: JMethodDetail = call_graph.nodes[edge[0]]["method_detail"] + target: JMethodDetail = call_graph.nodes[edge[1]]["method_detail"] + graph_edges.append((source, target)) + return graph_edges + + def __call_graph_using_symbol_table(self, qualified_class_name: str, method_signature: str, is_target_method: bool = False) -> nx.DiGraph: + cg = nx.DiGraph() + if is_target_method: + sdg = self.__raw_call_graph_using_symbol_table_target_method(target_class_name=qualified_class_name, target_method_signature=method_signature) + else: + sdg = self.__raw_call_graph_using_symbol_table(qualified_class_name=qualified_class_name, method_signature=method_signature) + tsu = TreesitterJava() + edge_list = [ + ( + (jge.source.method.signature, jge.source.klass), + (jge.target.method.signature, jge.target.klass), + {"type": jge.type, "weight": jge.weight, "calling_lines": tsu.get_calling_lines(jge.source.method.code, jge.target.method.signature)}, + ) + for jge in sdg + ] + for jge in sdg: + cg.add_node((jge.source.method.signature, jge.source.klass), method_detail=jge.source) + cg.add_node((jge.target.method.signature, jge.target.klass), method_detail=jge.target) + cg.add_edges_from(edge_list) + return cg + + def __raw_call_graph_using_symbol_table_target_method(self, target_class_name: str, target_method_signature: str, cg=None) -> list[JGraphEdgesST]: + if cg is None: + cg = [] + target_method_details = self.get_method(qualified_class_name=target_class_name, method_signature=target_method_signature) + for class_name in self.get_all_classes(): + for method in self.get_all_methods_in_class(qualified_class_name=class_name): + method_details = self.get_method(qualified_class_name=class_name, method_signature=method) + for call_site in method_details.call_sites: + source_method_details = None + source_class = "" + callee_signature = call_site.callee_signature if call_site.callee_signature != "" else "" + if call_site.receiver_type != "": + if self.get_class(qualified_class_name=call_site.receiver_type): + found_method, found_class = self.__find_method_in_hierarchy(call_site.receiver_type, callee_signature) + if found_method is not None and callee_signature == target_method_signature and found_class == target_class_name: + source_method_details = self.get_method(method_signature=method, qualified_class_name=class_name) + source_class = class_name + else: + found_method, found_class = self.__find_method_in_hierarchy(class_name, callee_signature) + if found_method is not None and callee_signature == target_method_signature and found_class == target_class_name: + source_method_details = self.get_method(method_signature=method, qualified_class_name=class_name) + source_class = class_name + if source_class != "" and source_method_details is not None: + call_edge = JGraphEdgesST( + source=JMethodDetail(method_declaration=source_method_details.declaration, klass=source_class, method=source_method_details), + target=JMethodDetail(method_declaration=target_method_details.declaration, klass=target_class_name, method=target_method_details), + type="CALL_DEP", + weight="1", + ) + if call_edge not in cg: + cg.append(call_edge) + return cg + + def __find_method_in_hierarchy(self, qualified_class_name: str, method_signature: str) -> Tuple[JCallable | None, str]: + klass = self.get_class(qualified_class_name=qualified_class_name) + method_details = self.get_method(method_signature=method_signature, qualified_class_name=qualified_class_name) + if method_details is not None and klass is not None and not klass.is_interface: + return method_details, qualified_class_name + if klass is not None: + for parent_class in klass.extends_list: + parent_method, found_class = self.__find_method_in_hierarchy(parent_class, method_signature) + if parent_method is not None: + return parent_method, found_class + return None, "" + + def __raw_call_graph_using_symbol_table(self, qualified_class_name: str, method_signature: str, cg=None) -> list[JGraphEdgesST]: + if cg is None: + cg = [] + source_method_details = self.get_method(qualified_class_name=qualified_class_name, method_signature=method_signature) + if source_method_details is None: + return cg + for call_site in source_method_details.call_sites: + target_method_details = None + target_class = "" + callee_signature = call_site.callee_signature if call_site.callee_signature != "" else "" + if call_site.receiver_type != "": + if self.get_class(qualified_class_name=call_site.receiver_type): + tmd, found_class = self.__find_method_in_hierarchy(call_site.receiver_type, callee_signature) + if tmd is not None: + target_method_details = tmd + target_class = found_class + else: + tmd, found_class = self.__find_method_in_hierarchy(qualified_class_name, callee_signature) + if tmd is not None: + target_method_details = tmd + target_class = found_class + if target_class != "" and target_method_details is not None: + call_edge = JGraphEdgesST( + source=JMethodDetail(method_declaration=source_method_details.declaration, klass=qualified_class_name, method=source_method_details), + target=JMethodDetail(method_declaration=target_method_details.declaration, klass=target_class, method=target_method_details), + type="CALL_DEP", + weight="1", + ) + if call_edge not in cg: + cg.append(call_edge) + return cg + + def get_class_call_graph(self, qualified_class_name: str, method_name: str | None = None) -> List[Tuple[JMethodDetail, JMethodDetail]]: + if method_name is None: + filter_criteria = {node for node in self.call_graph.nodes if node[1] == qualified_class_name} + else: + filter_criteria = {node for node in self.call_graph.nodes if tuple(node) == (method_name, qualified_class_name)} + graph_edges: List[Tuple[JMethodDetail, JMethodDetail]] = list() + for edge in self.call_graph.edges(nbunch=filter_criteria): + source: JMethodDetail = self.call_graph.nodes[edge[0]]["method_detail"] + target: JMethodDetail = self.call_graph.nodes[edge[1]]["method_detail"] + graph_edges.append((source, target)) + return graph_edges + + def remove_all_comments(self, src_code: str) -> str: + raise NotImplementedError("This function is not implemented yet.") + + # ===================================================================================== + # Entry points / CRUD / comments (operate on the reconstructed symbol table) + # ===================================================================================== + def get_all_entry_point_methods(self) -> Dict[str, Dict[str, JCallable]]: + methods = chain.from_iterable( + ((typename, method, callable) for method, callable in methods.items() if callable.is_entrypoint) for typename, methods in self.get_all_methods_in_application().items() + ) + return {typename: {method: callable for _, method, callable in group} for typename, group in groupby(methods, key=lambda x: x[0])} + + def get_all_entry_point_classes(self) -> Dict[str, JType]: + return {typename: klass for typename, klass in self.get_all_classes().items() if klass.is_entrypoint_class} + + def _crud(self, op_filter: CRUDOperationType | None) -> List[Dict[str, Union[JType, JCallable, List[JCRUDOperation]]]]: + rows = [] + for class_name, class_details in self.get_all_classes().items(): + for method_name, method_details in class_details.callable_declarations.items(): + if method_details.crud_operations and len(method_details.crud_operations) > 0: + ops = method_details.crud_operations if op_filter is None else [o for o in method_details.crud_operations if o.operation_type == op_filter] + rows.append({class_name: class_details, method_name: method_details, "crud_operations": ops}) + return rows + + def get_all_crud_operations(self) -> List[Dict[str, Union[JType, JCallable, List[JCRUDOperation]]]]: + return self._crud(None) + + def get_all_read_operations(self) -> List[Dict[str, Union[JType, JCallable, List[JCRUDOperation]]]]: + return self._crud(CRUDOperationType.READ) + + def get_all_create_operations(self) -> List[Dict[str, Union[JType, JCallable, List[JCRUDOperation]]]]: + return self._crud(CRUDOperationType.CREATE) + + def get_all_update_operations(self) -> List[Dict[str, Union[JType, JCallable, List[JCRUDOperation]]]]: + return self._crud(CRUDOperationType.UPDATE) + + def get_all_delete_operations(self) -> List[Dict[str, Union[JType, JCallable, List[JCRUDOperation]]]]: + return self._crud(CRUDOperationType.DELETE) + + def get_comments_in_a_method(self, qualified_class_name: str, method_signature: str) -> List[JComment]: + return self.get_method(qualified_class_name, method_signature).comments + + def get_comments_in_a_class(self, qualified_class_name: str) -> List[JComment]: + return self.get_class(qualified_class_name).comments + + def get_comment_in_file(self, file_path: str) -> List[JComment]: + compilation_unit = self.get_symbol_table().get(file_path, None) + if compilation_unit is None: + raise CodeanalyzerExecutionException(f"File {file_path} not found in the symbol table.") + return compilation_unit.comments + + def get_all_comments(self) -> Dict[str, List[JComment]]: + return {file_path: self.get_comment_in_file(file_path) for file_path in self.get_symbol_table()} + + def get_all_docstrings(self) -> List[Tuple[str, JComment]]: + docstrings = {} + for file_path, list_of_comments in self.get_all_comments().items(): + javadoc_comments = [docstring for docstring in list_of_comments if docstring.is_javadoc] + if javadoc_comments: + docstrings[file_path] = javadoc_comments + return docstrings diff --git a/cldk/analysis/java/neo4j/reconstruct.py b/cldk/analysis/java/neo4j/reconstruct.py new file mode 100644 index 00000000..a6f28908 --- /dev/null +++ b/cldk/analysis/java/neo4j/reconstruct.py @@ -0,0 +1,285 @@ +################################################################################ +# Copyright IBM Corporation 2026 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +"""Pure rehydration: Neo4j property maps → ``analysis.json``-shaped dicts for ``cldk.models.java``. + +:class:`~cldk.analysis.java.neo4j.JNeo4jBackend` bulk-fetches every node + relationship for an +application, groups children by parent, and feeds the grouped props here. Each function returns a +plain ``dict`` matching the corresponding pydantic model's field names, so the backend can assemble a +single ``analysis.json``-shaped payload and hand it to ``JApplication(**payload)`` — the exact same +constructor path the in-memory :class:`~cldk.analysis.java.codeanalyzer.JCodeanalyzer` uses +(``_init_japplication``). That guarantees the reconstructed objects are identical. + +The source graph is the one ``codeanalyzer-java`` (>= 2.4.0) emits with ``--emit neo4j`` — see its +``neo4j/GraphProjector.java`` / ``schema.neo4j.json`` for the property flattening these functions +invert. Java comments are first-class ``:JComment`` nodes (``J_HAS_COMMENT``), so unlike the Python +backend they round-trip losslessly. + +Parity caveats (inherent to what the projection stores, not bugs): a ``JType``'s +``is_class_or_interface_declaration`` and ``is_concrete_class`` flags are not projected (only the +``kind`` discriminator is), so they rehydrate to their defaults; the order of ``call_graph`` edges +is sorted rather than original-insertion order. +""" + +from __future__ import annotations + +import json +from typing import Any, Dict, List, Mapping + +Props = Mapping[str, Any] + + +# -----[ helpers ]----- +def _arr(props: Props, key: str) -> List[str]: + return list(props.get(key, []) or []) + + +def _kind_flags(kind: str | None) -> Dict[str, bool]: + """Derive the type-discriminator booleans from the projected ``kind`` string.""" + return { + "is_interface": kind == "interface", + "is_enum_declaration": kind == "enum", + "is_annotation_declaration": kind == "annotation", + "is_record_declaration": kind == "record", + } + + +# -----[ leaf nodes ]----- +def comment(props: Props) -> dict: + return { + "content": props.get("content"), + "start_line": props.get("start_line", -1), + "end_line": props.get("end_line", -1), + "start_column": props.get("start_column", -1), + "end_column": props.get("end_column", -1), + "is_javadoc": props.get("is_javadoc", False), + } + + +def parameter(props: Props) -> dict: + return { + "name": props.get("name"), + "type": props.get("type", ""), + "annotations": _arr(props, "annotations"), + "modifiers": _arr(props, "modifiers"), + "start_line": props.get("start_line", -1), + "end_line": props.get("end_line", -1), + "start_column": props.get("start_column", -1), + "end_column": props.get("end_column", -1), + } + + +def field(props: Props, *, comment_node: dict | None = None) -> dict: + raw = props.get("variable_initializers_json") + return { + "comment": comment_node, + "type": props.get("type", ""), + "start_line": props.get("start_line", -1), + "end_line": props.get("end_line", -1), + "variables": _arr(props, "variables"), + "modifiers": _arr(props, "modifiers"), + "annotations": _arr(props, "annotations"), + "variable_initializers": json.loads(raw) if raw else None, + } + + +def variable(props: Props, *, comment_node: dict | None = None) -> dict: + return { + "comment": comment_node, + "name": props.get("name", ""), + "type": props.get("type", ""), + "initializer": props.get("initializer", ""), + "start_line": props.get("start_line", -1), + "start_column": props.get("start_column", -1), + "end_line": props.get("end_line", -1), + "end_column": props.get("end_column", -1), + } + + +def enum_constant(props: Props) -> dict: + return {"name": props.get("name", ""), "arguments": _arr(props, "arguments")} + + +def record_component(props: Props, *, comment_node: dict | None = None) -> dict: + return { + "comment": comment_node, + "name": props.get("name", ""), + "type": props.get("type", ""), + "modifiers": _arr(props, "modifiers"), + "annotations": _arr(props, "annotations"), + "default_value": props.get("default_value"), + "is_var_args": props.get("is_var_args", False), + } + + +def crud_operation(props: Props) -> dict: + return {"line_number": props.get("line_number", -1), "operation_type": props.get("operation_type")} + + +def crud_query(props: Props) -> dict: + return { + "line_number": props.get("line_number", -1), + "query_arguments": props.get("query_arguments"), + "query_type": props.get("query_type"), + } + + +def callsite(props: Props, *, comment_node: dict | None = None, crud_op: dict | None = None, crud_q: dict | None = None) -> dict: + return { + "comment": comment_node, + "method_name": props.get("method_name", ""), + "receiver_expr": props.get("receiver_expr", ""), + "receiver_type": props.get("receiver_type", ""), + "argument_types": _arr(props, "argument_types"), + "argument_expr": _arr(props, "argument_expr"), + "return_type": props.get("return_type", ""), + "callee_signature": props.get("callee_signature", ""), + "is_static_call": props.get("is_static_call"), + "is_private": props.get("is_private"), + "is_public": props.get("is_public"), + "is_protected": props.get("is_protected"), + "is_unspecified": props.get("is_unspecified"), + "is_constructor_call": props.get("is_constructor_call", False), + "crud_operation": crud_op, + "crud_query": crud_q, + "start_line": props.get("start_line", -1), + "start_column": props.get("start_column", -1), + "end_line": props.get("end_line", -1), + "end_column": props.get("end_column", -1), + } + + +# -----[ declarations ]----- +def init_block( + props: Props, + *, + comments: List[dict] | None = None, + call_sites: List[dict] | None = None, + variable_declarations: List[dict] | None = None, +) -> dict: + return { + "file_path": props.get("file_path", ""), + "comments": comments or [], + "annotations": _arr(props, "annotations"), + "thrown_exceptions": _arr(props, "thrown_exceptions"), + "code": props.get("code", ""), + "start_line": props.get("start_line", -1), + "end_line": props.get("end_line", -1), + "is_static": props.get("is_static", False), + "referenced_types": _arr(props, "referenced_types"), + "accessed_fields": _arr(props, "accessed_fields"), + "call_sites": call_sites or [], + "variable_declarations": variable_declarations or [], + "cyclomatic_complexity": props.get("cyclomatic_complexity", 0), + } + + +def callable_( + props: Props, + *, + comments: List[dict] | None = None, + parameters: List[dict] | None = None, + call_sites: List[dict] | None = None, + variable_declarations: List[dict] | None = None, + crud_operations: List[dict] | None = None, + crud_queries: List[dict] | None = None, +) -> dict: + return { + "signature": props.get("signature", ""), + "is_implicit": props.get("is_implicit", False), + "is_constructor": props.get("is_constructor", False), + "comments": comments or [], + "annotations": _arr(props, "annotations"), + "modifiers": _arr(props, "modifiers"), + "thrown_exceptions": _arr(props, "thrown_exceptions"), + "declaration": props.get("declaration", ""), + "parameters": parameters or [], + "return_type": props.get("return_type"), + "code": props.get("code", ""), + "start_line": props.get("start_line", -1), + "end_line": props.get("end_line", -1), + "code_start_line": props.get("code_start_line", -1), + "referenced_types": _arr(props, "referenced_types"), + "accessed_fields": _arr(props, "accessed_fields"), + "call_sites": call_sites or [], + "is_entrypoint": props.get("is_entrypoint", False), + "variable_declarations": variable_declarations or [], + "crud_operations": crud_operations or [], + "crud_queries": crud_queries or [], + "cyclomatic_complexity": props.get("cyclomatic_complexity", 0), + } + + +def type_( + props: Props, + *, + comments: List[dict] | None = None, + callable_declarations: Dict[str, dict] | None = None, + field_declarations: List[dict] | None = None, + enum_constants: List[dict] | None = None, + record_components: List[dict] | None = None, + initialization_blocks: List[dict] | None = None, +) -> dict: + out = { + "is_inner_class": props.get("is_inner_class", False), + "is_local_class": props.get("is_local_class", False), + "is_nested_type": props.get("is_nested_type", False), + "comments": comments or [], + "extends_list": _arr(props, "extends_list"), + "implements_list": _arr(props, "implements_list"), + "modifiers": _arr(props, "modifiers"), + "annotations": _arr(props, "annotations"), + "parent_type": props.get("parent_type", ""), + "nested_type_declarations": _arr(props, "nested_type_declarations"), + "callable_declarations": callable_declarations or {}, + "field_declarations": field_declarations or [], + "enum_constants": enum_constants or [], + "record_components": record_components or [], + "initialization_blocks": initialization_blocks or [], + "is_entrypoint_class": props.get("is_entrypoint_class", False), + } + out.update(_kind_flags(props.get("kind"))) + return out + + +def compilation_unit( + props: Props, + *, + comments: List[dict] | None = None, + import_declarations: List[dict] | None = None, + type_declarations: Dict[str, dict] | None = None, +) -> dict: + return { + "file_path": props.get("file_path", props.get("file_key", "")), + "package_name": props.get("package_name", ""), + "comments": comments or [], + "import_declarations": import_declarations or [], + "type_declarations": type_declarations or {}, + "is_modified": props.get("is_modified", False), + } + + +def call_edge(source: dict, target: dict, props: Props) -> dict: + """A ``JGraphEdges``-shaped raw dict; endpoints resolve via JApplication's lookup table.""" + weight = props.get("weight") + return { + "source": source, + "target": target, + "type": props.get("type", "CALL_DEP"), + "weight": str(weight) if weight is not None else "1", + "source_kind": props.get("source_kind"), + "destination_kind": props.get("destination_kind"), + } diff --git a/cldk/core.py b/cldk/core.py index b8e1d281..485eb7ce 100644 --- a/cldk/core.py +++ b/cldk/core.py @@ -145,7 +145,10 @@ def java( CldkInitializationException: If neither or both of ``project_path`` / ``source_code`` are provided. """ - if project_path is None and source_code is None: + # The read-only Neo4j backend reads a graph populated out of band, so it needs neither + # project_path nor source_code. + is_neo4j = isinstance(backend, Neo4jConnectionConfig) + if project_path is None and source_code is None and not is_neo4j: raise CldkInitializationException("Either project_path or source_code must be provided.") if project_path is not None and source_code is not None: raise CldkInitializationException("Both project_path and source_code are provided. Please provide only one.") diff --git a/pyproject.toml b/pyproject.toml index da32f3be..85270fde 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -40,7 +40,7 @@ dependencies = [ "tree-sitter-javascript==0.23.1", "clang==17.0.6", "libclang==17.0.6", - "codeanalyzer-java==2.3.7", + "codeanalyzer-java==2.4.0", "codeanalyzer-python==0.2.0", "codeanalyzer-typescript==0.4.0", ] @@ -88,7 +88,7 @@ include = [ ] [tool.backend-versions] -codeanalyzer-java = "2.3.7" +codeanalyzer-java = "2.4.0" codeanalyzer-python = "0.2.0" codeanalyzer-typescript = "0.4.0" diff --git a/tests/analysis/java/test_java_backend_contract.py b/tests/analysis/java/test_java_backend_contract.py index 45cff1cb..a14492bc 100644 --- a/tests/analysis/java/test_java_backend_contract.py +++ b/tests/analysis/java/test_java_backend_contract.py @@ -23,10 +23,15 @@ from cldk.analysis.java.backend import JavaAnalysisBackend from cldk.analysis.java.codeanalyzer.codeanalyzer import JCodeanalyzer +from cldk.analysis.java.neo4j import JNeo4jBackend +# Both interchangeable backends must satisfy the same contract. +BACKENDS = [JCodeanalyzer, JNeo4jBackend] -def test_backend_subclasses_contract(): - assert issubclass(JCodeanalyzer, JavaAnalysisBackend) + +@pytest.mark.parametrize("backend", BACKENDS) +def test_backend_subclasses_contract(backend): + assert issubclass(backend, JavaAnalysisBackend) def test_contract_is_abstract(): @@ -34,8 +39,9 @@ def test_contract_is_abstract(): JavaAnalysisBackend() -def test_backend_fully_implements_contract(): - assert JCodeanalyzer.__abstractmethods__ == frozenset() +@pytest.mark.parametrize("backend", BACKENDS) +def test_backend_fully_implements_contract(backend): + assert backend.__abstractmethods__ == frozenset() def test_contract_covers_every_method_the_facade_delegates(): diff --git a/tests/analysis/java/test_java_neo4j_backend.py b/tests/analysis/java/test_java_neo4j_backend.py new file mode 100644 index 00000000..93463874 --- /dev/null +++ b/tests/analysis/java/test_java_neo4j_backend.py @@ -0,0 +1,147 @@ +################################################################################ +# Copyright IBM Corporation 2026 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +"""Integration parity tests: the read-only Java Neo4j backend vs the analysis.json backend. + +These assert that :class:`JNeo4jBackend` answers every query **identically** to the canonical +:class:`JCodeanalyzer` (analysis.json) backend on the same project — the definition of the +"1-to-1 map". + +Because the Java graph is populated out of band by the analyzer JAR (which needs a JDK and a Maven +build of the target for a level-2 call graph), this test does **not** populate inline. It is skipped +unless you point it at an already-populated Neo4j and a matching reference analysis cache: + + CLDK_TEST_NEO4J_URI=bolt://localhost:7687 \ + CLDK_TEST_NEO4J_USER=neo4j \ + CLDK_TEST_NEO4J_PASSWORD=test \ + CLDK_TEST_NEO4J_JAVA_APP=daytrader \ # the --app-name the graph was loaded with + CLDK_TEST_JAVA_PROJECT=/path/to/project \ # the reference project dir + CLDK_TEST_JAVA_CACHE=/path/to/cache/java \ # dir containing the reference analysis.json + pytest tests/analysis/java/test_java_neo4j_backend.py + +Populate the graph with: ``codeanalyzer-java -i --analysis-level 2 --emit neo4j +--neo4j-uri ... --app-name ``, and produce the reference with ``... --analysis-level 2 -o +``. + +Parity is asserted modulo the projection's documented-lossy fields (see +``cldk.analysis.java.neo4j.reconstruct``): a ``JType``'s ``is_class_or_interface_declaration`` / +``is_concrete_class`` flags are not projected (only ``kind`` is). +""" + +import json +import logging +import os + +import pytest + +logging.getLogger("neo4j").setLevel(logging.ERROR) + +NEO4J_URI = os.environ.get("CLDK_TEST_NEO4J_URI", "bolt://localhost:7687") +NEO4J_USER = os.environ.get("CLDK_TEST_NEO4J_USER", "neo4j") +NEO4J_PASSWORD = os.environ.get("CLDK_TEST_NEO4J_PASSWORD", "neo4j") +JAVA_APP = os.environ.get("CLDK_TEST_NEO4J_JAVA_APP") +JAVA_PROJECT = os.environ.get("CLDK_TEST_JAVA_PROJECT") +JAVA_CACHE = os.environ.get("CLDK_TEST_JAVA_CACHE") + +LOSSY_TYPE = {"is_class_or_interface_declaration", "is_concrete_class"} + + +def _neo4j_reachable() -> bool: + if not (JAVA_APP and JAVA_PROJECT and JAVA_CACHE): + return False + try: + from neo4j import GraphDatabase + except ModuleNotFoundError: + return False + try: + driver = GraphDatabase.driver(NEO4J_URI, auth=(NEO4J_USER, NEO4J_PASSWORD)) + driver.verify_connectivity() + driver.close() + return True + except Exception: # noqa: BLE001 + return False + + +pytestmark = pytest.mark.skipif( + not _neo4j_reachable(), + reason="needs a pre-populated Neo4j Java graph + reference cache (set CLDK_TEST_NEO4J_* / CLDK_TEST_JAVA_*)", +) + + +def _norm(o): + if hasattr(o, "model_dump"): + o = o.model_dump() + if isinstance(o, dict): + return {k: _norm(v) for k, v in o.items() if k not in LOSSY_TYPE} + if isinstance(o, list): + items = [_norm(x) for x in o] + try: + return sorted(items, key=lambda x: json.dumps(x, sort_keys=True, default=str)) + except Exception: + return items + return o + + +@pytest.fixture(scope="module") +def backends(): + from cldk.analysis.java.codeanalyzer.codeanalyzer import JCodeanalyzer + from cldk.analysis.java.neo4j import JNeo4jBackend + + ref = JCodeanalyzer(project_dir=JAVA_PROJECT, source_code=None, analysis_json_path=JAVA_CACHE, analysis_level="call_graph", eager_analysis=False, target_files=None) + neo = JNeo4jBackend(neo4j_uri=NEO4J_URI, neo4j_username=NEO4J_USER, neo4j_password=NEO4J_PASSWORD, application_name=JAVA_APP) + yield ref, neo + neo.close() + + +def test_symbol_table_and_classes_parity(backends): + ref, neo = backends + assert sorted(ref.get_symbol_table()) == sorted(neo.get_symbol_table()) + ac_ref, ac_neo = ref.get_all_classes(), neo.get_all_classes() + assert sorted(ac_ref) == sorted(ac_neo) + for cls in ac_ref: + assert _norm(ac_ref[cls]) == _norm(ac_neo[cls]), f"class {cls} differs" + + +def test_methods_fields_hierarchy_parity(backends): + ref, neo = backends + for cls in ref.get_all_classes(): + assert _norm(ref.get_all_fields(cls)) == _norm(neo.get_all_fields(cls)) + assert ref.get_extended_classes(cls) == neo.get_extended_classes(cls) + assert ref.get_implemented_interfaces(cls) == neo.get_implemented_interfaces(cls) + assert sorted(ref.get_all_sub_classes(cls)) == sorted(neo.get_all_sub_classes(cls)) + mc = ref.get_all_methods_in_class(cls) + assert sorted(mc) == sorted(neo.get_all_methods_in_class(cls)) + for sig in mc: + assert _norm(ref.get_method(cls, sig)) == _norm(neo.get_method(cls, sig)), f"{cls}::{sig} differs" + + +def test_call_graph_parity(backends): + ref, neo = backends + gr, gn = ref.get_call_graph(), neo.get_call_graph() + + def edgeset(g): + return sorted([list(u), list(v), g[u][v].get("type"), str(g[u][v].get("weight"))] for u, v in g.edges) + + assert edgeset(gr) == edgeset(gn) + assert sorted([list(n) for n in gr.nodes]) == sorted([list(n) for n in gn.nodes]) + + +def test_entrypoints_and_comments_parity(backends): + ref, neo = backends + assert sorted(ref.get_all_entry_point_classes()) == sorted(neo.get_all_entry_point_classes()) + assert sorted(ref.get_all_entry_point_methods()) == sorted(neo.get_all_entry_point_methods()) + assert sorted(ref.get_all_comments()) == sorted(neo.get_all_comments()) + assert sorted(ref.get_all_docstrings()) == sorted(neo.get_all_docstrings()) diff --git a/tests/analysis/java/test_java_neo4j_selection.py b/tests/analysis/java/test_java_neo4j_selection.py new file mode 100644 index 00000000..22e6a5b9 --- /dev/null +++ b/tests/analysis/java/test_java_neo4j_selection.py @@ -0,0 +1,75 @@ +################################################################################ +# Copyright IBM Corporation 2026 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +"""Backend-selection unit tests for the Java facade (no live Neo4j required). + +The Neo4j backend is fully mocked here, so these run anywhere. They verify that passing a +``Neo4jConnectionConfig`` swaps the facade onto :class:`JNeo4jBackend` (read-only), and that +without one the in-process :class:`JCodeanalyzer` is used. +""" + +from unittest.mock import patch + +import pytest + +from cldk.analysis.commons.backend_config import CodeAnalyzerConfig, Neo4jConnectionConfig +from cldk.analysis.java.java_analysis import JavaAnalysis + + +def test_neo4j_config_selects_neo4j_backend(): + config = Neo4jConnectionConfig(uri="bolt://example:7687", username="neo4j", password="secret", application_name="myapp") + with patch("cldk.analysis.java.java_analysis.JNeo4jBackend") as backend_cls, patch("cldk.analysis.java.java_analysis.JCodeanalyzer") as in_process_cls: + backend = backend_cls.return_value + + # Read-only: no project_dir needed, the graph is loaded out of band. + analysis = JavaAnalysis(project_dir=None, source_code=None, analysis_level="call_graph", target_files=None, eager_analysis=False, backend=config) + + _, kwargs = backend_cls.call_args + assert kwargs["neo4j_uri"] == "bolt://example:7687" + assert kwargs["neo4j_password"] == "secret" + assert kwargs["application_name"] == "myapp" + assert analysis.backend is backend + assert isinstance(analysis.backend_config, Neo4jConnectionConfig) + in_process_cls.assert_not_called() + + +def test_no_config_uses_in_process_backend(tmp_path): + with patch("cldk.analysis.java.java_analysis.JCodeanalyzer") as backend_cls, patch("cldk.analysis.java.java_analysis.JNeo4jBackend") as neo4j_cls: + analysis = JavaAnalysis(project_dir=str(tmp_path), source_code=None, analysis_level="symbol_table", target_files=None, eager_analysis=False) + + backend_cls.assert_called_once() + neo4j_cls.assert_not_called() + assert analysis.backend is backend_cls.return_value + assert isinstance(analysis.backend_config, CodeAnalyzerConfig) + + +def test_missing_neo4j_driver_raises_helpful_error(): + """Without the optional ``neo4j`` driver, constructing the backend explains how to install it.""" + import builtins + + from cldk.analysis.java.neo4j import JNeo4jBackend + from cldk.utils.exceptions.exceptions import CodeanalyzerExecutionException + + real_import = builtins.__import__ + + def _no_neo4j(name, *args, **kwargs): + if name == "neo4j": + raise ModuleNotFoundError("No module named 'neo4j'") + return real_import(name, *args, **kwargs) + + with patch("builtins.__import__", side_effect=_no_neo4j): + with pytest.raises(CodeanalyzerExecutionException, match="neo4j"): + JNeo4jBackend(neo4j_uri="bolt://example:7687", neo4j_username="neo4j", neo4j_password="neo4j", application_name="app") From 007e56cae092e99bce9286b3ecf7e48fcb0683cb Mon Sep 17 00:00:00 2001 From: Rahul Krishna Date: Mon, 22 Jun 2026 21:10:28 -0400 Subject: [PATCH 2/4] fix(java): resolve the bundled JDK via Adoptium /binary/version (the /assets/version endpoint 404s) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ensure_jdk resolved the Temurin JVM through the Adoptium /assets/version/{release} endpoint, which now returns 404 for pinned releases (e.g. jdk-21.0.5+11) even though the release exists — so the first Java analysis on a clean machine failed before it started. Resolve via /binary/version/... instead (capture the 307 redirect to the GitHub asset without downloading the binary) and read the checksum from the asset's adjacent .sha256.txt. Verified end-to-end: download + checksum + extract yields a JAVA_HOME with bin/java and jmods (openjdk 21.0.5). --- CHANGELOG.md | 7 ++++ cldk/analysis/java/codeanalyzer/_jdk.py | 45 +++++++++++++++++-------- 2 files changed, 38 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1b75db8e..bcbe80d6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -77,6 +77,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 to `2.4.0` (adds the Neo4j graph emitter). - Optional `neo4j` extra (`pip install cldk[neo4j]`) for the Neo4j Python driver. +### Fixed +- **Bundled JDK download for the Java backend.** `ensure_jdk` resolved the Temurin JVM via the + Adoptium `/assets/version/{release}` endpoint, which now returns 404 for pinned releases (e.g. + `jdk-21.0.5+11`) — so the first Java analysis on a clean machine failed before it started. It now + resolves via the `/binary/version/...` endpoint (following the redirect to the GitHub asset) and + reads the checksum from the asset's `.sha256.txt`. + ## [v1.0.7] - 2026-02-14 ### Added diff --git a/cldk/analysis/java/codeanalyzer/_jdk.py b/cldk/analysis/java/codeanalyzer/_jdk.py index 4015e445..d3dcb74c 100644 --- a/cldk/analysis/java/codeanalyzer/_jdk.py +++ b/cldk/analysis/java/codeanalyzer/_jdk.py @@ -36,12 +36,13 @@ from __future__ import annotations import hashlib -import json import logging import os import platform import stat import tarfile +import urllib.error +import urllib.parse import urllib.request import zipfile from pathlib import Path @@ -70,20 +71,36 @@ def _os_arch(cls) -> tuple[str, str]: @classmethod def _resolve_asset(cls) -> tuple[str, str]: - """Return ``(download_url, sha256)`` for the pinned JDK binary.""" + """Return ``(download_url, sha256)`` for the pinned JDK binary. + + Resolves via the Adoptium ``/binary/version`` endpoint, which 307-redirects to the + GitHub release asset; the checksum comes from the asset's adjacent ``.sha256.txt``. The + older ``/assets/version/{release}`` query endpoint is not used: it returns 404 for pinned + releases (e.g. ``jdk-21.0.5+11``), even though the release exists. + """ os_, arch = cls._os_arch() - url = ( - f"{cls._API}/assets/version/{JDK_RELEASE}" - f"?os={os_}&architecture={arch}&image_type=jdk" - f"&jvm_impl=hotspot&heap_size=normal&vendor=eclipse" - ) - req = urllib.request.Request(url, headers={"User-Agent": "cldk"}) - with urllib.request.urlopen(req, timeout=30) as resp: - data = json.load(resp) - if not data: - raise RuntimeError(f"No Temurin {JDK_RELEASE} build for {os_}/{arch}") - pkg = data[0]["binaries"][0]["package"] - return pkg["link"], pkg["checksum"] + release = urllib.parse.quote(JDK_RELEASE, safe="") # encode the '+' in the path + binary_url = f"{cls._API}/binary/version/{release}/{os_}/{arch}/jdk/hotspot/normal/eclipse" + + # Capture the redirect target (the GitHub asset URL) without downloading the binary. + class _NoRedirect(urllib.request.HTTPRedirectHandler): + def redirect_request(self, *args, **kwargs): + return None + + opener = urllib.request.build_opener(_NoRedirect) + req = urllib.request.Request(binary_url, headers={"User-Agent": "cldk"}) + try: + opener.open(req, timeout=30) + raise RuntimeError(f"Expected a redirect to the Temurin {JDK_RELEASE} asset from {binary_url}") + except urllib.error.HTTPError as exc: + if exc.code not in (301, 302, 303, 307, 308) or not exc.headers.get("Location"): + raise RuntimeError(f"No Temurin {JDK_RELEASE} build for {os_}/{arch} (HTTP {exc.code})") from exc + asset_url = exc.headers["Location"] + + sha_req = urllib.request.Request(asset_url + ".sha256.txt", headers={"User-Agent": "cldk"}) + with urllib.request.urlopen(sha_req, timeout=30) as resp: + sha = resp.read().decode().split()[0] + return asset_url, sha @classmethod def _java_home(cls, root: Path) -> Path: From 13076a64f8d8dccba1e64cedc393347ff148a269 Mon Sep 17 00:00:00 2001 From: Rahul Krishna Date: Mon, 22 Jun 2026 22:14:21 -0400 Subject: [PATCH 3/4] feat(java-neo4j): reconstruct variable_initializers as {}; note 2.4.1 fixes the emitter gaps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - reconstruct: a field with no initializers rehydrates to {} (not None), matching the analyzer's analysis.json representation. - Verified codeanalyzer-java 2.4.1 fixes the three projection gaps (#156/#157/#158): rebuilt the 2.4.1 jar and re-ran the daytrader parity — fields no longer collapse (642 JField nodes), imports link to :JType (1449), and J_CALLS went 287 -> 1702 (97% parity; the residual is external-target gating + run-to-run WALA variance between the separate --emit json and --emit neo4j invocations). Docstring/CHANGELOG updated; the SDK still pins 2.4.0 until 2.4.1 is released. --- CHANGELOG.md | 11 +++++---- cldk/analysis/java/neo4j/neo4j_backend.py | 28 +++++++++++++---------- cldk/analysis/java/neo4j/reconstruct.py | 4 +++- 3 files changed, 25 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bcbe80d6..a6fcb598 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,11 +68,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 from the graph `codeanalyzer-java` (>= 2.4.0) emits with `--emit neo4j` and answers all 36 `JavaAnalysisBackend` queries with the in-memory backend's logic. Verified against the daytrader8 sample (145 classes): everything the graph actually contains reconstructs identically to - `JCodeanalyzer`. Three producer-side gaps in the 2.4.0 emitter make the graph an incomplete - projection (tracked upstream, not query-layer bugs): all fields of a class collapse to one node - (codeanalyzer-java#156), imports lose the type name (codeanalyzer-java#157), and `J_CALLS` - materializes only a fraction of the call graph (codeanalyzer-java#158). `JavaAnalysis` / - `CLDK.java(...)` accept a `Neo4jConnectionConfig` as the `backend=` config to select it. + `JCodeanalyzer` (97% of checks). Three projection gaps in the `codeanalyzer-java` 2.4.0 emitter + (fields collapsing to one node, imports reduced to packages, a truncated call graph) are **fixed + in 2.4.1** (codeanalyzer-java#156/#157/#158, verified by rebuilding the 2.4.1 jar — `J_CALLS` on + daytrader went 287 → 1702); the SDK still pins 2.4.0, so they apply until 2.4.1 is released. + `JavaAnalysis` / `CLDK.java(...)` accept a `Neo4jConnectionConfig` as the `backend=` config to + select it. - Bumped `codeanalyzer-python` to `0.2.0` (adds the Neo4j graph emitter); bumped `codeanalyzer-java` to `2.4.0` (adds the Neo4j graph emitter). - Optional `neo4j` extra (`pip install cldk[neo4j]`) for the Neo4j Python driver. diff --git a/cldk/analysis/java/neo4j/neo4j_backend.py b/cldk/analysis/java/neo4j/neo4j_backend.py index e992f7fa..729567c0 100644 --- a/cldk/analysis/java/neo4j/neo4j_backend.py +++ b/cldk/analysis/java/neo4j/neo4j_backend.py @@ -41,18 +41,22 @@ scoped under ``(:JApplication {name})-[:J_HAS_UNIT]->(:JCompilationUnit)``. Parity: this backend reconstructs everything the graph actually contains identically to the -in-memory ``JCodeanalyzer`` (verified on the daytrader8 sample). It cannot recover what the -``codeanalyzer-java`` (2.4.0) emitter drops, however — three known producer-side gaps make the graph -an incomplete projection (tracked upstream, NOT query-layer bugs): - -* every ``:JField`` is emitted with the id ``#field#null``, so all fields of a class collapse to - one node (codeanalyzer-java#156); -* imports are projected to ``:JPackage`` only, losing the imported type name (codeanalyzer-java#157); -* ``J_CALLS`` materializes only a small fraction of the call graph — edges are absent even when both - endpoint callables are present as nodes (codeanalyzer-java#158). - -Projection-lossy-by-design: a ``:JType``'s ``is_class_or_interface_declaration`` / -``is_concrete_class`` flags are not projected (only the ``kind`` discriminator is). +in-memory ``JCodeanalyzer`` (verified on the daytrader8 sample — 97% of checks, the rest being the +caveats below). The ``codeanalyzer-java`` **2.4.0** emitter had three projection gaps — fields all +collapsing to one ``#field#null`` node, imports reduced to ``:JPackage``, and ``J_CALLS`` +materializing only a fraction of the call graph. All three are **fixed in 2.4.1** +(codeanalyzer-java#156/#157/#158); the SDK currently pins 2.4.0, so with the pinned emitter those +gaps still apply until 2.4.1 is released. + +Inherent caveats (present even on a complete graph, NOT query-layer bugs): + +* ``J_CALLS`` only links resolved app callables, so call edges to external/library targets (which the + in-memory backend keeps as synthetic nodes) are absent; +* the call graph is built by a separate analyzer run from the in-memory backend's ``analysis.json``, + so the two can differ by run-to-run WALA variance; +* a ``:JType``'s ``is_class_or_interface_declaration`` / ``is_concrete_class`` flags are not + projected (only the ``kind`` discriminator is); an absent singular ``comment`` rehydrates to + ``None``. """ from __future__ import annotations diff --git a/cldk/analysis/java/neo4j/reconstruct.py b/cldk/analysis/java/neo4j/reconstruct.py index a6f28908..131c5209 100644 --- a/cldk/analysis/java/neo4j/reconstruct.py +++ b/cldk/analysis/java/neo4j/reconstruct.py @@ -47,6 +47,8 @@ def _arr(props: Props, key: str) -> List[str]: return list(props.get(key, []) or []) + + def _kind_flags(kind: str | None) -> Dict[str, bool]: """Derive the type-discriminator booleans from the projected ``kind`` string.""" return { @@ -92,7 +94,7 @@ def field(props: Props, *, comment_node: dict | None = None) -> dict: "variables": _arr(props, "variables"), "modifiers": _arr(props, "modifiers"), "annotations": _arr(props, "annotations"), - "variable_initializers": json.loads(raw) if raw else None, + "variable_initializers": json.loads(raw) if raw else {}, } From b3b84d68f5d23216209eb7c8069106458bbf82ce Mon Sep 17 00:00:00 2001 From: Rahul Krishna Date: Mon, 22 Jun 2026 22:20:05 -0400 Subject: [PATCH 4/4] chore(java): drop the codeanalyzer-java pip dep; track the bundled jar at 2.4.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Java analyzer jar is no longer a pip dependency — the SDK release workflow downloads the latest codeanalyzer-java jar (now v2.4.1) into the bundled jar/ dir. The earlier codeanalyzer-java==2.4.0 entry in [project.dependencies] was broken (2.4.x is not on PyPI; PyPI tops out at 2.3.7), so remove it and bump the [tool.backend-versions] marker to 2.4.1. v2.4.1 is now the latest release and fixes the field/import/call-graph projection gaps (#156/#157/#158), so a graph emitted by the bundled analyzer is a complete projection. --- CHANGELOG.md | 13 +++++++------ cldk/analysis/java/neo4j/neo4j_backend.py | 6 +++--- pyproject.toml | 3 +-- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a6fcb598..e4d5e336 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,12 +70,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 sample (145 classes): everything the graph actually contains reconstructs identically to `JCodeanalyzer` (97% of checks). Three projection gaps in the `codeanalyzer-java` 2.4.0 emitter (fields collapsing to one node, imports reduced to packages, a truncated call graph) are **fixed - in 2.4.1** (codeanalyzer-java#156/#157/#158, verified by rebuilding the 2.4.1 jar — `J_CALLS` on - daytrader went 287 → 1702); the SDK still pins 2.4.0, so they apply until 2.4.1 is released. - `JavaAnalysis` / `CLDK.java(...)` accept a `Neo4jConnectionConfig` as the `backend=` config to - select it. -- Bumped `codeanalyzer-python` to `0.2.0` (adds the Neo4j graph emitter); bumped `codeanalyzer-java` - to `2.4.0` (adds the Neo4j graph emitter). + in 2.4.1** (codeanalyzer-java#156/#157/#158, verified on daytrader — `J_CALLS` went 287 → 1702), + the version the SDK release now bundles. `JavaAnalysis` / `CLDK.java(...)` accept a + `Neo4jConnectionConfig` as the `backend=` config to select it. +- Bumped `codeanalyzer-python` to `0.2.0` (adds the Neo4j graph emitter); the bundled + `codeanalyzer-java` jar is now `2.4.1` (adds the Neo4j graph emitter + the field/import/call-graph + projection fixes). The Java analyzer jar is no longer a pip dependency — the SDK release workflow + downloads the latest `codeanalyzer-java` jar into the bundled `jar/` directory. - Optional `neo4j` extra (`pip install cldk[neo4j]`) for the Neo4j Python driver. ### Fixed diff --git a/cldk/analysis/java/neo4j/neo4j_backend.py b/cldk/analysis/java/neo4j/neo4j_backend.py index 729567c0..6eed1a26 100644 --- a/cldk/analysis/java/neo4j/neo4j_backend.py +++ b/cldk/analysis/java/neo4j/neo4j_backend.py @@ -44,9 +44,9 @@ in-memory ``JCodeanalyzer`` (verified on the daytrader8 sample — 97% of checks, the rest being the caveats below). The ``codeanalyzer-java`` **2.4.0** emitter had three projection gaps — fields all collapsing to one ``#field#null`` node, imports reduced to ``:JPackage``, and ``J_CALLS`` -materializing only a fraction of the call graph. All three are **fixed in 2.4.1** -(codeanalyzer-java#156/#157/#158); the SDK currently pins 2.4.0, so with the pinned emitter those -gaps still apply until 2.4.1 is released. +materializing only a fraction of the call graph — all **fixed in 2.4.1** +(codeanalyzer-java#156/#157/#158), the version the SDK now bundles (its release workflow fetches the +latest codeanalyzer-java jar). So a graph emitted by a current analyzer is a complete projection. Inherent caveats (present even on a complete graph, NOT query-layer bugs): diff --git a/pyproject.toml b/pyproject.toml index 85270fde..db0140fa 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -40,7 +40,6 @@ dependencies = [ "tree-sitter-javascript==0.23.1", "clang==17.0.6", "libclang==17.0.6", - "codeanalyzer-java==2.4.0", "codeanalyzer-python==0.2.0", "codeanalyzer-typescript==0.4.0", ] @@ -88,7 +87,7 @@ include = [ ] [tool.backend-versions] -codeanalyzer-java = "2.4.0" +codeanalyzer-java = "2.4.1" codeanalyzer-python = "0.2.0" codeanalyzer-typescript = "0.4.0"