fix(e6): emit plain LATERAL for lateral subqueries, not LATERAL VIEW - #288
Merged
Merged
Conversation
E6.Generator.lateral_sql unconditionally forced view=True on every Lateral node and always emitted "LATERAL VIEW", so a standard "LATERAL (subquery)" correlated derived table was rendered as "LATERAL VIEW (subquery)" -- invalid in every dialect, since LATERAL VIEW requires a generator function. Both "LEFT JOIN LATERAL (subquery)" and comma / "CROSS JOIN LATERAL (subquery)" were affected; it has nothing to do with explode/arrays. Guard before forcing the view flag: when the lateral wraps a Subquery and view is not already set, delegate to the base generator, which emits plain LATERAL. LATERAL VIEW / explode laterals are untouched. Validated on the live E6 engine: plain "LATERAL (subquery)" parses and runs; "LATERAL VIEW (subquery)" does not. 37 queries in the migration run use it.
suyashkhare1403
approved these changes
Aug 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
E6.Generator.lateral_sqlunconditionally setview=Trueon everyLateralnode and always emittedLATERAL VIEW. So a standardLATERAL (subquery)(a correlated derived table) was rendered asLATERAL VIEW (subquery)— invalid in every dialect, sinceLATERAL VIEWrequires a generator function, not a subquery. Nothing to do with explode/arrays; any plain lateral subquery hits it.Root cause
The parser already distinguishes the two — a Spark
LATERAL VIEW <generator>getsview=True; a plainLATERAL (subquery)does not. The baseGenerator.lateral_sqlbranches on that flag (view→LATERAL VIEW, else plainLATERALvialateral_op). The e6 override discarded the flag and forcedLATERAL VIEWon everything.Fix
Guard before forcing the view flag: when the lateral wraps an
exp.Subqueryandviewis not already set, delegate to the base generator (plainLATERAL).LATERAL VIEW/ explode laterals are untouched.Validation
... CROSS JOIN LATERAL (SELECT 2 AS x) tand... LEFT JOIN LATERAL (...) ON TRUEparse and return rows; the oldLATERAL VIEW (subquery)fails withEncountered "JOIN" ... expecting "APPLY".LATERAL (subquery)queries in the migration run now emit plainLATERAL(0 still produce the invalid form).tests/dialects/test_e6.py: 59 passed / 901 subtests, incl. 2 new lateral-subquery regression cases;LATERAL VIEW EXPLODEunchanged.Root-cause analysis and fix by Atharv Rastogi.