Skip to content

kickstarting python lib #1

Open

Description

@kpwebb

@migurski just moving this thread over from sharedstreets/sharedstreets-ref-system#16 From the ticket:

So the SharedStreets tiles use length delimited encoding (the technique $GOOG recommends for files over ~1mb to allow stream processing of data). Turns out the Python Protobuf library doesn't have helper functions to parse these messages, but I found this example on how do to it:

https://www.datadoghq.com/blog/engineering/protobuf-parsing-in-python/

Just posted a quick example here using your compiled version of the .proto file:

https://github.com/sharedstreets/sharedstreets-python/blob/master/test.py

Let's use your example code as starting point for building tile loading functionality in the python lib.

Also @DenisCarriere want to create a punch list based on the core functionality from the JS lib so we can try and keep parity in the python version?

Activity

  1. migurski commented on Feb 28, 2018

    @migurski
    Collaborator

    I tried out the test code, and it worked well. Should be possible to run with this.

  2. DenisCarriere commented on Mar 1, 2018

    @DenisCarriere
    Contributor

    to create a punch list based on the core functionality from the JS lib so we can try and keep parity in the python version

    👍 Sounds good.

    I'll try to build up the documentation to have both Python & Javascript sync up.

    A few method names might need to change to follow the Python idioms.

    Ex: forwardReference => forward_reference

    Should we also divide the Python "core" library with the "pbf" parser (eventually writer).

    • sharedstreets
    • sharedstreets-pbf
  3. migurski commented on Mar 1, 2018

    @migurski
    Collaborator

    I’d keep them together; there’s little benefit to prematurely carving up libraries just yet!

    A few things I’d like to see in this library, that I’m willing to personally work on:

    • Parsing the hosted PBF files into Python data structures
    • Unified GeoJSON output
    • Basic tile server with slippy map preview so people can see it working
  4. kpwebb commented on Mar 1, 2018

    @kpwebb
    MemberAuthor

    @migurski this sounds great! and big +1 on the tile server with slippy preview. Having a CLI we can point at local tile directory would be fantastic.

    I'm curious if we can leverage any of the the existing React UI (http://platform.sharedstreets.io/) to boostrap this. I'm about to flip the switch on the public repo of this. The extensible layer infrastructure still needs a lot of work but hopefully a good starting point.

  5. migurski commented on Mar 1, 2018

    @migurski
    Collaborator

    What would the local tile directory CLI do? Fill it up, or convert it out?

  6. kpwebb commented on Mar 1, 2018

    @kpwebb
    MemberAuthor

    I was thinking something like the node "serve" command to spin up a slippy map viewer for local files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions