Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
where calls to a bare module name that is also imported (e.g. `os`/`re`/`json`) are dropped from
the emitted call graph. `PythonAnalysis` / `CLDK.analysis(language="python")` accept the same
optional `neo4j_config`.
- Bumped `codeanalyzer-python` to `0.2.0` (adds the Neo4j graph emitter).
- Read-only Neo4j-backed **Java** analysis backend (`cldk.analysis.java.neo4j.JNeo4jBackend`),
completing Neo4j parity across all three languages. It reconstructs the canonical `JApplication`
from the graph `codeanalyzer-java` (>= 2.4.0) emits with `--emit neo4j` and answers all 36
`JavaAnalysisBackend` queries with the in-memory backend's logic. Verified against the daytrader8
sample (145 classes): everything the graph actually contains reconstructs identically to
`JCodeanalyzer` (97% of checks). Three projection gaps in the `codeanalyzer-java` 2.4.0 emitter
(fields collapsing to one node, imports reduced to packages, a truncated call graph) are **fixed
in 2.4.1** (codeanalyzer-java#156/#157/#158, verified on daytrader — `J_CALLS` went 287 → 1702),
the version the SDK release now bundles. `JavaAnalysis` / `CLDK.java(...)` accept a
`Neo4jConnectionConfig` as the `backend=` config to select it.
- Bumped `codeanalyzer-python` to `0.2.0` (adds the Neo4j graph emitter); the bundled
`codeanalyzer-java` jar is now `2.4.1` (adds the Neo4j graph emitter + the field/import/call-graph
projection fixes). The Java analyzer jar is no longer a pip dependency — the SDK release workflow
downloads the latest `codeanalyzer-java` jar into the bundled `jar/` directory.
- Optional `neo4j` extra (`pip install cldk[neo4j]`) for the Neo4j Python driver.

### Fixed
- **Bundled JDK download for the Java backend.** `ensure_jdk` resolved the Temurin JVM via the
Adoptium `/assets/version/{release}` endpoint, which now returns 404 for pinned releases (e.g.
`jdk-21.0.5+11`) — so the first Java analysis on a clean machine failed before it started. It now
resolves via the `/binary/version/...` endpoint (following the redirect to the GitHub asset) and
reads the checksum from the asset's `.sha256.txt`.

## [v1.0.7] - 2026-02-14

### Added
Expand Down
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ pip install cldk
Optional extras:

```bash
pip install "cldk[neo4j]" # read-only Neo4j graph backend (Python / TypeScript)
pip install "cldk[neo4j]" # read-only Neo4j graph backend (Java / Python / TypeScript)
```

## Quick Start
Expand Down Expand Up @@ -117,11 +117,11 @@ classes = analysis.get_all_classes()

## Supported Languages & Backends

Each language is analyzed by a dedicated `codeanalyzer-*` engine; CLDK normalizes the result into typed models exposed through the same API.
Each language is analyzed by a dedicated `codeanalyzer-*` engine; CLDK normalizes the result into typed models exposed through the same API. All three also support an optional **read-only Neo4j backend** — pass a `Neo4jConnectionConfig` and the SDK answers the same queries with Cypher over a graph the analyzer populates out of band (`--emit neo4j`).

| Language | Analysis engine | What it provides |
| --- | --- | --- |
| **Java** | [`codeanalyzer-java`](https://github.com/codellm-devkit/codeanalyzer-java) | WALA + JavaParser. Bytecode-level call graphs, type hierarchies, symbol resolution, and method/field declarations. |
| **Java** | [`codeanalyzer-java`](https://github.com/codellm-devkit/codeanalyzer-java) | WALA + JavaParser. Bytecode-level call graphs, type hierarchies, symbol resolution, CRUD-operation and entry-point detection. Optional read-only **Neo4j** graph backend. |
| **Python** | [`codeanalyzer-python`](https://github.com/codellm-devkit/codeanalyzer-python) | Jedi with optional CodeQL augmentation. Symbol tables, call graphs, and class/method resolution. Optional read-only **Neo4j** graph backend. |
| **TypeScript / JavaScript** | [`codeanalyzer-typescript`](https://github.com/codellm-devkit/codeanalyzer-typescript) | ts-morph with Jelly-based call graphs. Symbols, call graph, types, decorators, and call sites. Optional read-only **Neo4j** graph backend. |

Expand All @@ -147,13 +147,14 @@ graph TD
P --> EP[codeanalyzer-python<br/>Jedi · CodeQL]
T --> ET[codeanalyzer-typescript<br/>ts-morph · Jelly]

P -. read-only .-> N[(Neo4j)]
J -. read-only .-> N[(Neo4j)]
P -. read-only .-> N
T -. read-only .-> N
```

**Data models** — each language has its own set of Pydantic models under `cldk.models` (`cldk.models.java`, `cldk.models.python`, `cldk.models.typescript`). They give you structured, typed, dot-accessible representations of classes, methods, fields, and statements, with JSON serialization and shared conventions across languages.

**Analysis backends** — each language has a backend under `cldk.analysis.<language>` that coordinates its engine (see the table above) and maps the result onto the data models. Backends are orchestrated internally; you only call high-level methods such as `get_symbol_table()`, `get_method_body(...)`, and `get_call_graph(...)`, and CLDK handles tool coordination, parsing, and marshalling under the hood.
**Analysis backends** — each language has a backend under `cldk.analysis.<language>` that coordinates its engine (see the table above) and maps the result onto the data models. The read-only Neo4j backends (`cldk.analysis.<language>.neo4j`) reconstruct the *same* models from a Cypher graph, so they are drop-in interchangeable with the in-process analyzers. Backends are orchestrated internally; you only call high-level methods such as `get_symbol_table()`, `get_method_body(...)`, and `get_call_graph(...)`, and CLDK handles tool coordination, parsing, and marshalling under the hood.

## Documentation

Expand Down
5 changes: 2 additions & 3 deletions cldk/analysis/commons/backend_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,9 +99,8 @@ class Neo4jConnectionConfig:
application_name: str | None = None


# Per-language discriminated unions the facades match on. Java has no Neo4j backend yet, so its
# only admissible config is the codeanalyzer one.
JavaBackend = CodeAnalyzerConfig
# Per-language discriminated unions the facades match on.
JavaBackend = Union[CodeAnalyzerConfig, Neo4jConnectionConfig]
PyBackend = Union[PyCodeAnalyzerConfig, Neo4jConnectionConfig]
TSBackend = Union[CodeAnalyzerConfig, Neo4jConnectionConfig]

Expand Down
45 changes: 31 additions & 14 deletions cldk/analysis/java/codeanalyzer/_jdk.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,13 @@
from __future__ import annotations

import hashlib
import json
import logging
import os
import platform
import stat
import tarfile
import urllib.error
import urllib.parse
import urllib.request
import zipfile
from pathlib import Path
Expand Down Expand Up @@ -70,20 +71,36 @@ def _os_arch(cls) -> tuple[str, str]:

@classmethod
def _resolve_asset(cls) -> tuple[str, str]:
"""Return ``(download_url, sha256)`` for the pinned JDK binary."""
"""Return ``(download_url, sha256)`` for the pinned JDK binary.

Resolves via the Adoptium ``/binary/version`` endpoint, which 307-redirects to the
GitHub release asset; the checksum comes from the asset's adjacent ``.sha256.txt``. The
older ``/assets/version/{release}`` query endpoint is not used: it returns 404 for pinned
releases (e.g. ``jdk-21.0.5+11``), even though the release exists.
"""
os_, arch = cls._os_arch()
url = (
f"{cls._API}/assets/version/{JDK_RELEASE}"
f"?os={os_}&architecture={arch}&image_type=jdk"
f"&jvm_impl=hotspot&heap_size=normal&vendor=eclipse"
)
req = urllib.request.Request(url, headers={"User-Agent": "cldk"})
with urllib.request.urlopen(req, timeout=30) as resp:
data = json.load(resp)
if not data:
raise RuntimeError(f"No Temurin {JDK_RELEASE} build for {os_}/{arch}")
pkg = data[0]["binaries"][0]["package"]
return pkg["link"], pkg["checksum"]
release = urllib.parse.quote(JDK_RELEASE, safe="") # encode the '+' in the path
binary_url = f"{cls._API}/binary/version/{release}/{os_}/{arch}/jdk/hotspot/normal/eclipse"

# Capture the redirect target (the GitHub asset URL) without downloading the binary.
class _NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, *args, **kwargs):
return None

opener = urllib.request.build_opener(_NoRedirect)
req = urllib.request.Request(binary_url, headers={"User-Agent": "cldk"})
try:
opener.open(req, timeout=30)
raise RuntimeError(f"Expected a redirect to the Temurin {JDK_RELEASE} asset from {binary_url}")
except urllib.error.HTTPError as exc:
if exc.code not in (301, 302, 303, 307, 308) or not exc.headers.get("Location"):
raise RuntimeError(f"No Temurin {JDK_RELEASE} build for {os_}/{arch} (HTTP {exc.code})") from exc
asset_url = exc.headers["Location"]

sha_req = urllib.request.Request(asset_url + ".sha256.txt", headers={"User-Agent": "cldk"})
with urllib.request.urlopen(sha_req, timeout=30) as resp:
sha = resp.read().decode().split()[0]
return asset_url, sha

@classmethod
def _java_home(cls, root: Path) -> Path:
Expand Down
44 changes: 28 additions & 16 deletions cldk/analysis/java/java_analysis.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,12 +52,13 @@

from tree_sitter import Tree

from cldk.analysis.commons.backend_config import CodeAnalyzerConfig, JavaBackend, cache_subdir
from cldk.analysis.commons.backend_config import CodeAnalyzerConfig, JavaBackend, Neo4jConnectionConfig, cache_subdir
from cldk.analysis.commons.treesitter import TreesitterJava
from cldk.models.java import JCallable
from cldk.models.java import JApplication
from cldk.models.java.models import JCRUDOperation, JComment, JCompilationUnit, JMethodDetail, JType, JField
from cldk.analysis.java.codeanalyzer import JCodeanalyzer
from cldk.analysis.java.neo4j import JNeo4jBackend
from cldk.analysis.java.backend import JavaAnalysisBackend


Expand Down Expand Up @@ -149,22 +150,33 @@ def __init__(
self.eager_analysis = eager_analysis
self.target_files = target_files
self.backend_config: JavaBackend = backend if backend is not None else CodeAnalyzerConfig()
# Java has a single backend family; the config only carries the cache root. analysis.json
# is cached under <cache_dir>/java (None in source_code mode, where the analyzer streams
# results over a pipe).
cache_path = cache_subdir(self.backend_config.cache_dir, project_dir, "java")
if cache_path is not None:
cache_path.mkdir(parents=True, exist_ok=True)
self.treesitter_java: TreesitterJava = TreesitterJava()
# Initialize the analysis backend
self.backend: JavaAnalysisBackend = JCodeanalyzer(
project_dir=self.project_dir,
source_code=self.source_code,
eager_analysis=self.eager_analysis,
analysis_level=self.analysis_level,
analysis_json_path=cache_path,
target_files=self.target_files,
)
self.backend: JavaAnalysisBackend
if isinstance(self.backend_config, Neo4jConnectionConfig):
# Read-only: the graph is populated out of band; the SDK only polls it.
cfg = self.backend_config
application_name = cfg.application_name or (Path(project_dir).name if project_dir else None)
self.backend = JNeo4jBackend(
neo4j_uri=cfg.uri,
neo4j_username=cfg.username,
neo4j_password=cfg.password,
neo4j_database=cfg.database,
application_name=application_name,
)
else:
# The config only carries the cache root. analysis.json is cached under <cache_dir>/java
# (None in source_code mode, where the analyzer streams results over a pipe).
cache_path = cache_subdir(self.backend_config.cache_dir, project_dir, "java")
if cache_path is not None:
cache_path.mkdir(parents=True, exist_ok=True)
self.backend = JCodeanalyzer(
project_dir=self.project_dir,
source_code=self.source_code,
eager_analysis=self.eager_analysis,
analysis_level=self.analysis_level,
analysis_json_path=cache_path,
target_files=self.target_files,
)

def get_imports(self) -> List[str]:
"""Return all import statements in the source code.
Expand Down
22 changes: 22 additions & 0 deletions cldk/analysis/java/neo4j/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
################################################################################
# Copyright IBM Corporation 2026
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################

"""Read-only Neo4j-backed Java analysis backend (Cypher queries over the codeanalyzer-java graph)."""

from cldk.analysis.java.neo4j.config import Neo4jConnectionConfig
from cldk.analysis.java.neo4j.neo4j_backend import JNeo4jBackend

__all__ = ["JNeo4jBackend", "Neo4jConnectionConfig"]
27 changes: 27 additions & 0 deletions cldk/analysis/java/neo4j/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
################################################################################
# Copyright IBM Corporation 2026
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################

"""Connection settings for the read-only Neo4j-backed Java analysis backend.

The definition has been hoisted to :mod:`cldk.analysis.commons.backend_config`; it is re-exported
here for symmetry with the Python and TypeScript backends.
"""

from __future__ import annotations

from cldk.analysis.commons.backend_config import Neo4jConnectionConfig

__all__ = ["Neo4jConnectionConfig"]
Loading