ObjectRef.full_path leaves out the store's location for s3, gcs and azure stores #1646

Summary

On an s3, gcs or azure store with a location, ObjectRef.full_path points to a path where the object is not. ref.fs.exists(ref.full_path) returns False and ref.fs.open(ref.full_path) raises FileNotFoundError, although ref.read() returns the object.

The backend stores an object under <bucket>/<location>/<path> (StorageBackend._full_path). full_path builds s3://<bucket>/<path>, and the same for gs:// and az://, without the location. Its docstring says it includes the "bucket/location", and its file branch does add location.

ref.fsmap builds its root with _full_path, so it is not affected. On a store with an empty location, full_path is correct.

Reproduction

DataJoint 2.3.4 and master, MySQL 8.0, MinIO:

import pathlib
import tempfile

import datajoint as dj

dj.config.stores["outbox"] = {
    "protocol": "s3", "endpoint": "localhost:9000", "bucket": "bkt", "location": "proj/outbox",
    "access_key": "datajoint", "secret_key": "datajoint", "secure": False,
}
schema = dj.Schema("scratch_full_path")


@schema
class Result(dj.Manual):
    definition = """
    result_id : int32
    ---
    data : <object@outbox>
    """


src = pathlib.Path(tempfile.mkdtemp()) / "result.txt"
src.write_text("hello")
Result.insert1({"result_id": 1, "data": str(src)})

ref = (Result & {"result_id": 1}).fetch1("data")
ref.read()                       # b'hello'
ref.full_path                    # 's3://bkt/_schema/scratch_full_path/result/result_id=1/data_....txt'
ref.fs.exists(ref.full_path)     # False
ref.fs.exists(ref.fsmap.root)    # True
ref.fs.open(ref.full_path)       # FileNotFoundError

Workaround

Use ref.open(), ref.read() or ref.fsmap instead of ref.fs with ref.full_path.

Expected behavior

full_path includes the store's location, so ref.fs.open(ref.full_path) opens the object. For s3, gcs and azure it could return self._backend.get_url(self.path), which already prefixes the protocol to _full_path.

Where

ObjectRef.full_path in objectref.py.