Support for Mongo 3.6

[ngbalk]

MemoryBackend currently is coded to emulate Mongo 3.0.0 - any plans on supporting newer versions (3.6.0)?

This would require updating the OpCode handling as 3.6 introduces a new OpCode that is not currently supported (#2013)

mongo-java-server/core/src/main/java/de/bwaldvogel/mongo/wire/OpCode.java

Line 16 in 7449324

[bwaldvogel]

Actually some time ago I started implementation for opcode 2013 (OP_MSG).

However, I de-prioritized the work since I’m currently not aware of use-case where you are able to notice the difference. If you have such a case, could you provide a minimal test case?

[ngbalk]

Thanks @bwaldvogel
We wanted to test how our application would perform with a "flaky mongo backend", i.e. we would wrap the existing backend with some logic that fails every n requests. The latest mongo-java-driver > 3.6 has retry-able writes, and mongo-java-driver > 3.11.0 has retry-able reads baked into the driver's code. We wanted to verify in a unit test that our application would be resilient and silently retry an intermittent failure.

The MongoClient asks the database for its "version" on initial connection so it knows which opcodes to use, but it also enables/disables client-side retries if the database is newer / older than 3.6.0. Perhaps there are other client-side features that are enabled / disabled depending on the backend version, but our team was particular interested in retryReads and retryWrites.

Hope that gives you some context :)

[freakbite]

On the topic of 3.6 support I was wondering about support for

{
   $lookup:
     {
       from: <collection to join>,
       let: { <var_1>: <expression>, …, <var_n>: <expression> },
       pipeline: [ <pipeline to execute on the collection to join> ],
       as: <output array field>
     }
}

Where let is optional.

[bwaldvogel]

@freakbite: I’ve started to implement support for $lookup stages with pipelines in 59fb1f4.
Could you test your use-cases with the latest version?

[bwaldvogel]

@freakbite: The change is released in version 1.20.0

[snava10]

@bwaldvogel I'm looking at implementing transactions which requires the implementation of OP_MSG. I'm happy to pick this up, if you have made any progress and have something to share I would appreciate it.

[bwaldvogel]

I was finally able to finish my earlier work on support for OP_MSG (wire protocol version 6) in 26a681c.

Note that mongo-java-server currently does not enable wire protocol version 6 by default!

@ngbalk, @snava10:
I would be glad if you could test it. You can enable it manually with:

MemoryBackend backend = new MemoryBackend().version(ServerVersion.MONGO_3_6);
MongoServer mongoServer = new MongoServer(backend);

[ngbalk]

Awesome @bwaldvogel ! I'm excited to test this out!

[bwaldvogel]

The change is available in version 1.30.0