Skip to content

reduce search results size #641

Open
Open
reduce search results size#641
Assignees
jradcliff
Labels
enhancementNew feature or request

Description

@schabe77
schabe77
opened on Jun 23, 2022

Describe the problem you are trying to solve
I migrated from old Adwords API to new Ads API. I recognized that loading some of my reports take longer than before. First I thought it was an server issues, but then I found out that the responses are much larger. An example: when doing the request

SELECT campaign.status, ad_group.status, ad_group_criterion.status, ad_group_criterion.type
 FROM ad_group_criterion
 WHERE ad_group_criterion.negative = false
 AND campaign.status != REMOVED
 AND ad_group.status != REMOVED
 AND ad_group_criterion.status != REMOVED

on a specific account, the response stream contains 280MB of data. When I did the same request with the old Adwords Reporting API (using GZIPPED_CSV as download format) the download size was only 200KB.

Describe the solution you would like
It would be great if the response size of the Ads API could be similar to the report size of the Adwords API and not differ by a factor of 1400. The old API returned a compressed csv file, the response of the new API seems to be uncompressed. It would be great if the Ads API would communicate to the server using compressed messages.

Describe alternatives you've considered
There is not much I can do on my side, but I hope the library could

  • use compressed requests/responses. As far as I could see gRPC supports gzip compression: io.grpc.stub.AbstractStub.withCompression("gzip")

Additional context
I copied the most of this ticket's content from #438 that I wrote one year ago. The old issue was closed, but only one part of the request was implemented (thanks for that again). @nwbirnie wrote that I should check back in a few months.

Activity

  1. schabe77
    added
    enhancementNew feature or request
    triage
    on Jun 23, 2022
  2. jradcliff
    self-assigned this
    on Jun 28, 2022
  3. jradcliff
    removed
    triage
    on Jun 28, 2022

  4. jradcliff commented on Jun 28, 2022

    @jradcliff
    jradcliff
    on Jun 28, 2022
    Member

    Could you share a recent request-id from that request that contained 280 MB of data, along with the query you used in the AdWords API? I'd like to investigate that case.

    Also, are you using the new OMIT_UNSELECTED_RESOURCE_NAMES parameter? I know you ran into some issues with that parameter early on, but those issues were resolved a while back.

    Thanks

  5. schabe77 commented on Jun 30, 2022

    @schabe77
    schabe77
    on Jun 30, 2022
    Author

    Hi Josh,

    yes, I use the new parameter (thanks again) from the very beginning.

    What I did to find some examples is to sum up SearchGoogleAdsStreamResponse.getSerializedSize() for each request. I didn't make it to 280MB, but here are three examples:

    requestId=qTODgOHB8he5i0kXYfU5Rw, size=226.617.663 bytes, query=

    SELECT ad_group_ad.resource_name
 FROM ad_group_ad
 WHERE campaign.status != REMOVED
 AND ad_group.status != REMOVED
 AND ad_group_ad.status != REMOVED
 AND ad_group_ad.ad.system_managed_resource_source != AD_VARIATIONS
 PARAMETERS omit_unselected_resource_names=true

    requestId=R7ZPX_DmbxpDlpNlLInbXg, size=175.483.361 bytes, query=

    SELECT ad_group_ad.resource_name, ad_group_ad.ad.expanded_text_ad.headline_part1, ad_group_ad.ad.expanded_text_ad.headline_part2, ad_group_ad.ad.expanded_text_ad.headline_part3, ad_group_ad.ad.expanded_text_ad.description, ad_group_ad.ad.expanded_text_ad.description2, ad_group_ad.ad.expanded_text_ad.path1, ad_group_ad.ad.expanded_text_ad.path2, ad_group_ad.ad.type, ad_group_ad.ad.final_urls, ad_group_ad.ad.url_custom_parameters
 FROM ad_group_ad
 WHERE campaign.status != REMOVED
 AND ad_group.status != REMOVED
 AND ad_group_ad.status != REMOVED
 AND ad_group_ad.ad.type = EXPANDED_TEXT_AD
 PARAMETERS omit_unselected_resource_names=true

    requestId=lamT2O1-dPfI4o0P67VyeQ, size=195.195.731 bytes, query=

    SELECT asset.id, asset.page_feed_asset.page_url, asset.page_feed_asset.labels
 FROM asset
 WHERE asset.type = PAGE_FEED
 PARAMETERS omit_unselected_resource_names=true

    The problem is not that I get this big results sizes for a single account, the problem is, that I do the same request for hundred(s) of accounts - so even if the result is "only" 100MB large, for >100 accounts you easily run into Gigabytes.

    Best regards,

    Christian

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions