[Python] Add AwsOptions and deprecate S3Options

[shunping]

Motivation

S3Options holds settings for creating an AWS service client: credentials, region, endpoint and SSL settings. None of them are specific to S3. Upcoming AWS integrations, such as AWS Secrets Manager support, need the same settings, so this PR adds a general AwsOptions and deprecates S3Options.

Changes

• New AwsOptions in apache_beam.options.pipeline_options, with the same fields as S3Options under an aws_ prefix:

  New flag	Deprecated flag
  --aws_access_key_id	--s3_access_key_id
  --aws_secret_access_key	--s3_secret_access_key
  --aws_session_token	--s3_session_token
  --aws_endpoint_url	--s3_endpoint_url
  --aws_region_name	--s3_region_name
  --aws_api_version	--s3_api_version
  --aws_verify	--s3_verify
  --aws_disable_ssl	--s3_disable_ssl

• Backward compatibility
  • The --s3_* flags are still accepted. Each aws_* option falls back to its s3_* counterpart when it isn't set, and logs a one-time deprecation warning. If both are set, --aws_* wins.
  • The fallback also covers keyword arguments (e.g. PipelineOptions(s3_endpoint_url=...)) and the dict form accepted by the S3 client.
  • S3Options is now a deprecated subclass of AwsOptions. Its s3_* attributes return the corresponding aws_* values, so existing code like options.view_as(S3Options).s3_region_name keeps working.
  • Both flag sets are registered on AwsOptions only. Since S3Options is no longer a direct subclass of PipelineOptions, no flag is registered twice.
• S3 client: boto3_client.Client now reads its settings from AwsOptions.

This is one of the steps to address #40528

[jrmccluskey]

LGTM, thanks!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.73%. Comparing base (76f6e6d) to head (de09a52).
⚠️ Report is 127 commits behind head on master.

Additional details and impacted files

@@              Coverage Diff              @@
##             master   #40529       +/-   ##
=============================================
+ Coverage     56.16%   79.73%   +23.56%     
=============================================
  Files          1124      487      -637     
  Lines        178336    82285    -96051     
  Branches       1489        0     -1489     
=============================================
- Hits         100170    65610    -34560     
+ Misses        75645    16675    -58970     
+ Partials       2521        0     -2521     

Flag	Coverage Δ
go	?
java	?
python	79.73% <100.00%> (+0.17%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Assigning reviewers:

R: @tvalentyn for label python.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).