Skip to content

how do I use TiParser lexer? #1157

Open

Description

@david-sail

Question

how do I call TiParser lexer for sql word split just like vitess token not ast parser, the sample code is as follows

var tokens []Token
tkn := sqlparser.NewStringTokenizer(sql)
typ, val := tkn.Scan()

Activity

  1. tangenta commented on Jan 21, 2021

    @tangenta
    Contributor

    Here is an example:

    cat example_test.go
    package parser
    
    import (
    	"fmt"
    	. "github.com/pingcap/check"
    )
    
    var _ = Suite(&testExampleSuite{})
    
    type testExampleSuite struct {
    }
    
    func (s *testExampleSuite) TestExample(c *C) {
    	l := NewScanner("create table t (a int);")
    	var v yySymType
    	for {
    		result := l.Lex(&v)
    		if result == 0 || result == invalid {
    			break
    		}
    		fmt.Printf("%d\n", result)
    	}
    }
    go test -v github.com/pingcap/parser -check.f TestExample
    === RUN   TestT
    57382
    57531
    57346
    40
    57346
    57446
    41
    59
    PASS: example_test.go:13: testExampleSuite.TestExample  0.001s

    The meaning of these numbers can be found in parser.go.

  2. zjcxc commented on Sep 13, 2022

    @zjcxc

    yySymType is a private struct, how can i use it in my program?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions