Skip to content

Batch processing - #635

Merged
zhicwu merged 4 commits into
ClickHouse:developfrom
zhicwu:batch-process
Apr 21, 2021
Merged

Batch processing#635
zhicwu merged 4 commits into
ClickHouse:developfrom
zhicwu:batch-process

Conversation

@zhicwu

@zhicwu zhicwu commented Apr 20, 2021

Copy link
Copy Markdown
Contributor
  • enable batch processing in Statement
    try (Statement s = connection.createStatement()) {
      s.addBatch("select 1; select 2"); // same as s.addBatch("select 1"); s.addBatch("select 2");
      s.addBatch("select 3");
      ...
      s.executeBatch();
    }
  • arbitray sql in PreparedStatement
    try (PreparedStatement s = connection.prepareStatement("alter table my_table update value=? where key=2")) {
      s.setInt(1, 233);
      s.addBatch(); // addBatch(String) will result in SQLException according to JDBC spec
      ...
      s.executeBatch();
    }
  • performance is slightly improved because generated batch insert statement will no longer be parsed again
    Note: Basic.insertOneRandomNumber(normal) dropped to ~274.899 from ~543463.393 because addBatch(String) was an empty method.

@zhicwu zhicwu changed the title Enable batch execution in Statement Batch processing Apr 20, 2021
@github-actions

Copy link
Copy Markdown
Benchmark                           (client)  (statement)   Mode  Cnt       Score       Error  Units
Basic.insertOneRandomNumber  clickhouse-jdbc       normal  thrpt   20  539761.864 ± 12407.335  ops/s
Basic.selectOneRandomNumber  clickhouse-jdbc       normal  thrpt   20     874.187 ±    64.683  ops/s
Basic.selectOneRandomNumber  clickhouse-jdbc     prepared  thrpt   20     841.559 ±    70.248  ops/s

@enqueue

enqueue commented Apr 20, 2021

Copy link
Copy Markdown
Contributor

Sorry that my PR was not ready, yet

@zhicwu

zhicwu commented Apr 20, 2021

Copy link
Copy Markdown
Contributor Author

Sorry that my PR was not ready, yet

Not a problem. I changed some code in order to add support for multi-statement SQL, so I figured that I'd better carry on to tackle this one too. To be honest, I spent too much time playing MHS lately, so I really need to hurry up :p

@github-actions

Copy link
Copy Markdown
Benchmark                           (client)  (statement)   Mode  Cnt       Score      Error  Units
Basic.insertOneRandomNumber  clickhouse-jdbc       normal  thrpt   20  548317.483 ± 6943.436  ops/s
Basic.selectOneRandomNumber  clickhouse-jdbc       normal  thrpt   20    1080.404 ±   70.840  ops/s
Basic.selectOneRandomNumber  clickhouse-jdbc     prepared  thrpt   20    1055.015 ±  100.579  ops/s

@zhicwu
zhicwu merged commit af73772 into ClickHouse:develop Apr 21, 2021
@zhicwu
zhicwu deleted the batch-process branch April 21, 2021 01:49
@zhicwu zhicwu linked an issue Apr 21, 2021 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants