CAMEL-14929: Changes S3Object manipulation to avoid consume large S3 into memory

[avaz]

This PR addresses CAMEL-14929 which the goal is to give an option to do not only consume S3 objects into the memory and let the S3Object open so them can be manipulated by the next steps in the pipeline.

With such behaviour is possible to process large S3 objects in a more memory friendly fashion (streaming).

[oscerd]

Can you please add unit tests by using localstack approach? It should be pretty easy to test all the combinations. Thanks.

[oscerd]

And the javadoc you changed must be changed at javadoc level on the s3 configuration class, after the first build the table will be regenerated.

[oscerd]

@avaz any news?

[avaz]

Hi @oscerd ,

TL;DR
Integration tests are added for the changes related to this PR as I couldn't get to add localstack tests.

I couldn't add tests using the localstack due the fact that localstack test seems to interact with camel in a different way, essentially the functional class the I made the change (src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Endpoint.java) is not being called by localstack. I checked that by debugging one of localstack tests (src/test/java/org/apache/camel/component/aws2/s3/localstack/S3ConsumerLocalstackTest.java) and the debugger didn't hit the AWS2S3Endpoint.java class. Therefore I decided to add integration test instead and I did that in the class src/test/java/org/apache/camel/component/aws2/s3/integration/S3ConsumerIntegrationTest.java.

Please, let me know if this is enough or if possible provide some guidance on how to implement the same tests using localstack. I would expect if I copy and paste the tests from the integration test class to the localstack class would as is.

Another thing is that I didn't is about the java docs, where they are generated and what kind of build I run to generate them? So far I'm run mvn clean install -Pfastinstall at the project root to have the build done. However I don't see any java doc being generated. Could you provide guidance here as well?

[oscerd]

The endpoint is used anyway, so I don't know why in the test with localstack should not be used. Do you have docker running? By the way, the Pr is enough. For the javadoc, you need to do clean install -DskipTests, btw I'll do that once I merge. Thanks

[oscerd]

Thanks, merged on master.