Custom transformer - create_instance invoked incorrectly?

[UnitedMarsupials]

Hello!

I'm trying to parse a file serializing an ArrayList of custom functions -- and I cannot even get the basics going...

Below is my entire script. Without the default value (such as None) for the class_desc, I get an error:

  File "/home/mi/.local/lib/python3.9/site-packages/javaobj/v2/core.py", line 528, in _do_object
    instance = self._create_instance(class_desc)
  File "/home/mi/.local/lib/python3.9/site-packages/javaobj/v2/core.py", line 503, in _create_instance
    instance = transformer.create_instance(class_desc)
TypeError: create_instance() missing 1 required positional argument: 'class_desc'

If I put the default value, the call to create_instance goes through, and the method prints the value of self, which indicates, that it is the class_desc -- instead of referring to the transformer:

INFO:javaobj.parser:Reading next content
DEBUG:javaobj.parser:Reading new object: handle 7e0001, classdesc [classdesc 0x7e0000: name java.util.ArrayList, uid 8683452581122892189]
Self is [classdesc 0x7e0000: name java.util.ArrayList, uid 8683452581122892189], class_desc is None
Traceback ...

Please, advise. Thank you!

import javaobj.v2 as javaobj
import sys
import os
import json
import pprint
import logging

logging.basicConfig(level = logging.DEBUG)

class odrnavsFunctionInstance(javaobj.beans.JavaInstance):
	def __init__(self):
		javaobj.beans.JavaInstance.__init__(self)

	def load_from_instance(self):
		if not self.classdesc or self.classdesc not in self.field_data:
			return False

		fields = self.classdesc.fields_names
		values = [
			self.field_data[self.classdesc][self.classdesc.fields[i]]
			for i in range(len(fields))
		]
		self.field_data = dict(zip(fields, values))
		if not self.classdesc.super_class or \
		    self.classdesc.super_class not in self.annotations:
			return True
		super_class = self.annotations[self.classdesc.super_class][0]
		self.annotations = dict(zip(super_class.fields_names, super_class.field_data))
		return True

class odrnavsTransformer(javaobj.transformers.ObjectTransformer):

	def __init__(self, handled_classes = {
		'my.apps.datalayer.client.navs.shared.api.Function':
		    odrnavsFunctionInstance
	}):
		super(JavaRandomTransformer, self).__init__()
		self.instance = None
		self.handled_classes = handled_classes or {}

	def create_instance(self, class_desc = None):
		print('Self is %s, class_desc is %s' % (self, class_desc))
		if class_desc.name not in self.handled_classes:
			return None
		self.instance = self.handled_classes[class_desc.name]()
		return self.instance

if sys.argv[1].endswith('.gz'):
	import gzip
	opener = gzip.open
else:
	opener = open

obj = None
with opener(sys.argv[1], 'rb') as inp:
		obj = javaobj.load(inp, odrnavsTransformer)

pprint.pprint(obj)

[tcalmant]

Hi
Sorry for the long delay, I seem to have missed the notifications for this project.

Regarding your issue: the transformers given to javaobj.v2.load are supposed be instances, not classes.
As a result, you should call obj = javaobj.load(inp, odrnavsTransformer()) instead of obj = javaobj.load(inp, odrnavsTransformer).

[UnitedMarsupials]

Thank you, that seems to work now, but I'm still not getting much useful from the file I'm trying to decode. It seems to contain java.util.ArrayList with several dozens entries of the same vendor-specific items in it: vendor.apps.datalayer.client.navs.shared.api.Function.

Do I need to know the fields of that vendor's class -- and put that knowledge into my Python script as the example in README.md is doing -- or would javaobj do that for me automatically? At least, for the fields of known Java-types?

My goal is to be able to turn the binary Java-serialization data into a human-readable text -- ideally, in JSON-format -- and back.

[tcalmant]

On the original report: transformers are expected as instances, and passing
the class gave a confusing error from inside the parser. #71 makes v2 and v3
say what is wrong instead:

TypeError: Transformers must be given as instances, not as classes:
got the class MyTransformer, did you mean MyTransformer() ?

On your follow-up question, whether javaobj can turn a vendor class into
readable JSON without writing a transformer: partly, and it is worth knowing
where the limit is.

Everything a class writes through the default mechanism is recovered without
any transformer: the declared fields, of every class of the hierarchy, and
the collections, which are already converted to list / dict. You only
need a transformer when a class defines its own writeObject(), because what
it wrote is then opaque bytes whose layout only that Java class knows. No
library can guess that from the stream alone.

So a generic dump gets you quite far. With v3:

import json
import javaobj.v3 as javaobj
from javaobj.v3.beans import JavaInstance, JavaArray, JavaEnum

def to_python(obj):
    # Collections first: the transformed ones are dict/list *and* JavaInstance
    if isinstance(obj, dict):
        return {str(to_python(k)): to_python(v) for k, v in obj.items()}
    if isinstance(obj, (list, JavaArray)):
        return [to_python(o) for o in obj]
    if isinstance(obj, JavaEnum):
        return obj.constant
    if isinstance(obj, JavaInstance):
        out = {"__class__": obj.classdesc.name}
        for fields in obj.field_data.values():
            for field, value in fields.items():
                out[field.name] = to_python(value)
        return out
    if isinstance(obj, bytes):
        return list(obj)
    return obj

with open("dump.ser", "rb") as fd:
    print(json.dumps(to_python(javaobj.load(fd)), indent=2, default=str))

On an ArrayList of vendor objects this gives you the list, and for each
element its class name and its fields, the inherited ones included. If some
field comes back empty, that class writes it in a custom writeObject() and
that one needs a transformer.

Going back to Java from that JSON is another matter: v3 can write streams,
but rebuilding a class which has a custom writeObject() means reproducing
its format by hand.