Skip to content

fix(e6): emit plain LATERAL for lateral subqueries, not LATERAL VIEW - #288

Merged
suyashkhare1403 merged 1 commit into
e6data:mainfrom
tkaunlaky-e6:fix/e6-lateral-subquery
Aug 4, 2026
Merged

fix(e6): emit plain LATERAL for lateral subqueries, not LATERAL VIEW#288
suyashkhare1403 merged 1 commit into
e6data:mainfrom
tkaunlaky-e6:fix/e6-lateral-subquery

Conversation

@tkaunlaky-e6

Copy link
Copy Markdown

Problem

E6.Generator.lateral_sql unconditionally set view=True on every Lateral node and always emitted LATERAL VIEW. So a standard LATERAL (subquery) (a correlated derived table) was rendered as LATERAL VIEW (subquery) — invalid in every dialect, since LATERAL VIEW requires a generator function, not a subquery. Nothing to do with explode/arrays; any plain lateral subquery hits it.

in : SELECT * FROM t1 LEFT JOIN LATERAL (SELECT x FROM t2 WHERE t2.a = t1.a) s ON TRUE
out: SELECT * FROM t1 LEFT JOIN LATERAL VIEW (SELECT ...) AS s ON TRUE     -- unparseable

in : SELECT * FROM t1, LATERAL (SELECT x FROM t2 WHERE t2.a = t1.a) s
out: SELECT * FROM t1 CROSS JOIN LATERAL VIEW (SELECT ...) AS s            -- unparseable

Root cause

The parser already distinguishes the two — a Spark LATERAL VIEW <generator> gets view=True; a plain LATERAL (subquery) does not. The base Generator.lateral_sql branches on that flag (viewLATERAL VIEW, else plain LATERAL via lateral_op). The e6 override discarded the flag and forced LATERAL VIEW on everything.

Fix

Guard before forcing the view flag: when the lateral wraps an exp.Subquery and view is not already set, delegate to the base generator (plain LATERAL). LATERAL VIEW / explode laterals are untouched.

if not expression.args.get("view") and isinstance(expression.this, exp.Subquery):
    return super().lateral_sql(expression)

Validation

  • Live E6 engine: ... CROSS JOIN LATERAL (SELECT 2 AS x) t and ... LEFT JOIN LATERAL (...) ON TRUE parse and return rows; the old LATERAL VIEW (subquery) fails with Encountered "JOIN" ... expecting "APPLY".
  • Coverage: all 37 LATERAL (subquery) queries in the migration run now emit plain LATERAL (0 still produce the invalid form).
  • tests/dialects/test_e6.py: 59 passed / 901 subtests, incl. 2 new lateral-subquery regression cases; LATERAL VIEW EXPLODE unchanged.

Root-cause analysis and fix by Atharv Rastogi.

E6.Generator.lateral_sql unconditionally forced view=True on every Lateral
node and always emitted "LATERAL VIEW", so a standard "LATERAL (subquery)"
correlated derived table was rendered as "LATERAL VIEW (subquery)" -- invalid
in every dialect, since LATERAL VIEW requires a generator function. Both
"LEFT JOIN LATERAL (subquery)" and comma / "CROSS JOIN LATERAL (subquery)"
were affected; it has nothing to do with explode/arrays.

Guard before forcing the view flag: when the lateral wraps a Subquery and
view is not already set, delegate to the base generator, which emits plain
LATERAL. LATERAL VIEW / explode laterals are untouched.

Validated on the live E6 engine: plain "LATERAL (subquery)" parses and runs;
"LATERAL VIEW (subquery)" does not. 37 queries in the migration run use it.
@suyashkhare1403
suyashkhare1403 merged commit e6f89d4 into e6data:main Aug 4, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants