Repository navigation
docs: update developer guide - #850
Merged
Merged
Conversation
yizhouxw
reviewed
Nov 17, 2023
yhilmare
added a commit
that referenced
this pull request
Jan 15, 2024
* docs: modify developer guide * uodate submodule * response to cr comments
sjjian
pushed a commit
to actiontech/odc
that referenced
this pull request
May 20, 2026
…ithout catalog_name (issue oceanbase#850) Bug: DMS 创建 PG 数据源时仅传 default_schema（典型值 public）而不传 catalogName， ODC 后端 PostgresConnectionExtension#generateJdbcUrl 抛 `IllegalArgumentException: catalog name can not be null`，导致 DatabaseService.syncDataSourceSchemas 100% 失败、前端资源树 PG 节点没有 caret-down， 无法展开到 schema 层（dms-ee#850 / Task-001 烟雾测试 EXC-2）。 Root cause: PG 的 JDBC URL 形如 jdbc:postgresql://host:port/<catalog>?currentSchema=<schema>， catalog 必须非空，但 ODC `ConnectionConfig` 把 catalogName / defaultSchema 分离持久化， 上游（DMS）侧只填了 default_schema，未提供 database name；ODC 当前没有兜底。 Fix: 在 OBConsoleDataSourceFactory 新增 `resolveEffectiveCatalogName(dialectType, catalogName, defaultSchema)` 静态方法做 PG 专属兜底——仅当 dialectType=POSTGRESQL 且 catalogName 为空时生效： 1. defaultSchema 非空且 !equalsIgnoreCase("public") → 用 defaultSchema 兜底（兼容 用户在 default_schema 字段中实际填了 database 名的场景）； 2. 否则使用 PG 标准内置数据库 `postgres`（POSTGRESQL_DEFAULT_DATABASE， 在所有 PG 标准安装中默认存在）。 不直接用 defaultSchema=public 作 catalog 兜底，因为 PG 中 public 是 schema 名而不是 database 名，强行用之会抛 FATAL: database "public" does not exist。 对其他数据源类型（MySQL/Oracle/SQLServer/OceanBase 等）保持原行为不变：catalogName 原样透传，不影响其既有 JDBC URL 生成逻辑（这些 dialect 的 ConnectionExtension 未对 catalogName 做非空校验）。 Files: - odc-core/OdcConstants: 新增 POSTGRESQL_DEFAULT_DATABASE = "postgres" 常量 - odc-service/OBConsoleDataSourceFactory: getJdbcUrlProperties() 通过新增的 resolveEffectiveCatalogName 做兜底；保留原 connectionConfig 持久化字段不变 - odc-service/ConnectionTesting#getJdbcUrlProperties: 同步走 resolveEffectiveCatalogName 兜底，让测试连接（POST /api/v2/datasource/.../connection_test）也修复 - odc-service/DataSourceInfoMapper#getJdbcUrl: DLM 数据迁移任务路径同步兜底 - odc-service test: 新增 OBConsoleDataSourceFactoryTest 12 条用例，覆盖 PG/MySQL/Oracle/OBMysql/SqlServer 各种 catalog/schema 组合，含 issue oceanbase#850 现场复现 Validation: - mvn -pl server/odc-core install + mvn -pl server/odc-service compile 全通过 - 单测 OBConsoleDataSourceFactoryTest 12/12 OK（独立 JUnit runner） - ODC 重启后日志 `Create datasource success, jdbcUrl: jdbc:postgresql://10.186.16.126:5433/postgres?...¤tSchema=public&...` 证明 jdbc URL 生成成功；之前 `catalog name can not be null` 错误**不再出现** Refs: dms-ee#850, EXC-2 (Task-001 烟雾测试)
4 tasks
sjjian
pushed a commit
to actiontech/odc
that referenced
this pull request
May 29, 2026
…ale databases on JDBC sync failure (issue oceanbase#850) Two PG datasource compatibility fixes follow-up the existing oceanbase#850 work: 1. resolveEffectiveCatalogName(): drop the defaultSchema fallback. The previous logic returned defaultSchema as catalog when it was non-empty and not "public", on the assumption that users sometimes put the database name in default_schema. That assumption breaks for the common case where default_schema is a real PG schema name (e.g. schema_a) and there is no PG database named schema_a, which triggers FATAL: database "schema_a" does not exist and blocks every sync / connection test for that datasource. Now we always fall back to OdcConstants.POSTGRESQL_DEFAULT_DATABASE ("postgres") when catalogName is blank. The front-end already keeps catalogName as a required field for PG datasource, so this is purely a defence-in-depth path for older imports and third-party callers. Unit test OBConsoleDataSourceFactoryTest is realigned: the customSchema_usesSchemaAsCatalog case is renamed to customSchema_fallsBackToPostgresDb to match the new contract; the other 11 cases (explicit catalog returns as-is; non-PG dialects untouched; etc.) continue to assert the previous behaviour. 2. DatabaseService.handleSyncException(): also clean connect_database on generic JDBC connection failure. The OceanBase-specific clauses (cluster not exist / No tenants found) were the only ones that triggered deleteDatabaseIfInstanceNotExists(). When PG syncing failed because the catalog did not exist (or the network was unreachable, or auth failed), the old database rows kept is_existed=1 and the user could not refresh the list from the UI. We add a third branch matching common JDBC connection failure markers (CannotGetJdbcConnectionException / "Failed to obtain JDBC Connection" / "FATAL: database") and reuse the same cleanup helper. Failed-reason stays UNKNOWN so the user sees the raw JDBC error in connect_sync_history but the stale database list is cleared. Refs: dms-ee#850
LordofAvernus
added a commit
to actiontech/odc
that referenced
this pull request
Jun 1, 2026
oceanbase#850, compat-RISK-1) Refs: dms-ee#850 Risk: compat-RISK-1 data-upgrade: B-V_4_3_4_14__fix_postgresql_version_diff_config_idempotent 存量 ODC 部署因 V_4_3_4_13 末尾 `ON DUPLICATE KEY UPDATE config_key=config_key` （自我赋值即 NOOP），无法把已存在的 PostgreSQL 三项配置 support_trigger / support_sequence / support_type 从 'true' 覆盖为 'false'，导致升级后 PG 后端 PostgreSQLFeatures 仍按 true 加载，与前端收敛不一致（compat_risks.md §1 compat-RISK-1 / design.md §8.4）。 新增独立 Flyway DML 脚本 V_4_3_4_14，对存量行执行 UPDATE： - WHERE `db_mode` = 'POSTGRESQL'：仅命中 PG，不影响 MySQL/Oracle 等其它 db_mode - WHERE `config_key` IN ('support_trigger','support_sequence','support_type')： 仅修复三项 bug 配置，不影响 PG 其它 config_key（support_view / support_function 等） - WHERE `config_value` = 'true'：仅修复老错误状态，二次执行 / 全新部署均零变更 （幂等），不抹掉运维已手工修正的状态 V_4_3_4_13 文本未改动（Flyway checksum 约束）；本次新增 SQL 与 V_4_3_4_13 同 目录，Flyway 同 location 扫描，按版本号顺序执行。
LordofAvernus
added a commit
to actiontech/odc
that referenced
this pull request
Jun 1, 2026
…ithout catalog_name (issue oceanbase#850) Bug: DMS 创建 PG 数据源时仅传 default_schema（典型值 public）而不传 catalogName， ODC 后端 PostgresConnectionExtension#generateJdbcUrl 抛 `IllegalArgumentException: catalog name can not be null`，导致 DatabaseService.syncDataSourceSchemas 100% 失败、前端资源树 PG 节点没有 caret-down， 无法展开到 schema 层（dms-ee#850 / Task-001 烟雾测试 EXC-2）。 Root cause: PG 的 JDBC URL 形如 jdbc:postgresql://host:port/<catalog>?currentSchema=<schema>， catalog 必须非空，但 ODC `ConnectionConfig` 把 catalogName / defaultSchema 分离持久化， 上游（DMS）侧只填了 default_schema，未提供 database name；ODC 当前没有兜底。 Fix: 在 OBConsoleDataSourceFactory 新增 `resolveEffectiveCatalogName(dialectType, catalogName, defaultSchema)` 静态方法做 PG 专属兜底——仅当 dialectType=POSTGRESQL 且 catalogName 为空时生效： 1. defaultSchema 非空且 !equalsIgnoreCase("public") → 用 defaultSchema 兜底（兼容 用户在 default_schema 字段中实际填了 database 名的场景）； 2. 否则使用 PG 标准内置数据库 `postgres`（POSTGRESQL_DEFAULT_DATABASE， 在所有 PG 标准安装中默认存在）。 不直接用 defaultSchema=public 作 catalog 兜底，因为 PG 中 public 是 schema 名而不是 database 名，强行用之会抛 FATAL: database "public" does not exist。 对其他数据源类型（MySQL/Oracle/SQLServer/OceanBase 等）保持原行为不变：catalogName 原样透传，不影响其既有 JDBC URL 生成逻辑（这些 dialect 的 ConnectionExtension 未对 catalogName 做非空校验）。 Files: - odc-core/OdcConstants: 新增 POSTGRESQL_DEFAULT_DATABASE = "postgres" 常量 - odc-service/OBConsoleDataSourceFactory: getJdbcUrlProperties() 通过新增的 resolveEffectiveCatalogName 做兜底；保留原 connectionConfig 持久化字段不变 - odc-service/ConnectionTesting#getJdbcUrlProperties: 同步走 resolveEffectiveCatalogName 兜底，让测试连接（POST /api/v2/datasource/.../connection_test）也修复 - odc-service/DataSourceInfoMapper#getJdbcUrl: DLM 数据迁移任务路径同步兜底 - odc-service test: 新增 OBConsoleDataSourceFactoryTest 12 条用例，覆盖 PG/MySQL/Oracle/OBMysql/SqlServer 各种 catalog/schema 组合，含 issue oceanbase#850 现场复现 Validation: - mvn -pl server/odc-core install + mvn -pl server/odc-service compile 全通过 - 单测 OBConsoleDataSourceFactoryTest 12/12 OK（独立 JUnit runner） - ODC 重启后日志 `Create datasource success, jdbcUrl: jdbc:postgresql://10.186.16.126:5433/postgres?...¤tSchema=public&...` 证明 jdbc URL 生成成功；之前 `catalog name can not be null` 错误**不再出现** Refs: dms-ee#850, EXC-2 (Task-001 烟雾测试)
LordofAvernus
added a commit
to actiontech/odc
that referenced
this pull request
Jun 1, 2026
…ale databases on JDBC sync failure (issue oceanbase#850) Two PG datasource compatibility fixes follow-up the existing oceanbase#850 work: 1. resolveEffectiveCatalogName(): drop the defaultSchema fallback. The previous logic returned defaultSchema as catalog when it was non-empty and not "public", on the assumption that users sometimes put the database name in default_schema. That assumption breaks for the common case where default_schema is a real PG schema name (e.g. schema_a) and there is no PG database named schema_a, which triggers FATAL: database "schema_a" does not exist and blocks every sync / connection test for that datasource. Now we always fall back to OdcConstants.POSTGRESQL_DEFAULT_DATABASE ("postgres") when catalogName is blank. The front-end already keeps catalogName as a required field for PG datasource, so this is purely a defence-in-depth path for older imports and third-party callers. Unit test OBConsoleDataSourceFactoryTest is realigned: the customSchema_usesSchemaAsCatalog case is renamed to customSchema_fallsBackToPostgresDb to match the new contract; the other 11 cases (explicit catalog returns as-is; non-PG dialects untouched; etc.) continue to assert the previous behaviour. 2. DatabaseService.handleSyncException(): also clean connect_database on generic JDBC connection failure. The OceanBase-specific clauses (cluster not exist / No tenants found) were the only ones that triggered deleteDatabaseIfInstanceNotExists(). When PG syncing failed because the catalog did not exist (or the network was unreachable, or auth failed), the old database rows kept is_existed=1 and the user could not refresh the list from the UI. We add a third branch matching common JDBC connection failure markers (CannotGetJdbcConnectionException / "Failed to obtain JDBC Connection" / "FATAL: database") and reuse the same cleanup helper. Failed-reason stays UNKNOWN so the user sees the raw JDBC error in connect_sync_history but the stale database list is cleared. Refs: dms-ee#850
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
there is a discussion #336 about how to startup odc and we find that our developer guide leaks installation of plugin and relied modules which may lead to this issue.
so that I update the developer guide and add these contents.
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Additional documentation e.g., usage docs, etc.: