Skip to content

Sharrow mode not accessing xarray skim object as expected #662

Description

@i-am-sijia

Describe the bug

The 2-zone profiling exercise profiled full size placeholder_sandag and placeholder_psrc models, and found out that Accessibility Calculation is the memory peak for the two example 2-zone models. After tracing line by line, it turns out the following line gets triggered to access the xarray skim object, causing the memory spike. See more details in the presentation at Meeting 2023.03.28, the slides include a chart of memory comparison between sharrow turned on vs off.

result = self.dataset.iat(**x, _name=key) # iat strips data encoding
# if 'digital_encoding' in self.dataset[key].attrs:
# result = array_decode(result, self.dataset[key].attrs['digital_encoding'])

image

Jeff pointed out that this line should not be triggered when it's running sharrow mode, something must be failing in the sharrow implementation. If implemented correctly, there should not be a memory spike when accessing skim object in sharrow mode.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

BugSomething isn't working/bug f

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions