You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
schema.rebuild_lineage() fails on PostgreSQL, after emptying ~lineage #1641
schema.rebuild_lineage() fails on PostgreSQL. rebuild_schema_lineage (lineage.py) reads keys from information_schema with MySQL-only names:
foreign keys from KEY_COLUMN_USAGE.REFERENCED_TABLE_SCHEMA, REFERENCED_TABLE_NAME and REFERENCED_COLUMN_NAME, columns PostgreSQL does not have;
primary keys by CONSTRAINT_NAME = 'PRIMARY', while PostgreSQL names a primary-key constraint <table>_pkey.
The foreign-key query raises, so the rebuild stops before writing anything. It has already run DELETE FROM ~lineage by then, so the call leaves the schema with an empty ~lineage table: every join on the schema's tables is then refused, until each table is decorated again and its rows are refilled.
The docs and several messages recommend schema.rebuild_lineage() as the repair for lineage problems, and dj.migrate.rebuild_lineage calls the same function.
Reproduction
DataJoint master, PostgreSQL 15:
importdatajointasdjschema=dj.Schema("scratch_rebuild_lineage")
@schemaclassImage(dj.Manual):
definition=""" image_id : int32 """schema.rebuild_lineage()
# UnknownAttributeError: column "referenced_table_schema" does not exist# ~lineage held 1 row before the call and 0 after it
Expected behavior
rebuild_lineage() works on both backends. It could read primary and foreign keys through the adapters' existing load_primary_keys_sql and load_foreign_keys_sql, as Dependencies.load does, and leave ~lineage unchanged if it fails.
Activity