init hooks

Sometimes I want to write an attr.s class that has some behavior in its constructor. Particularly:

I want to validate between/among different related values passed to the c'tor
I want to establish a bidirectional relationship to one of the arguments passed to the c'tor (for example: protocol/transport hook-up)
I want to do some gross I/O because it's a handle for some external gross I/O object (i.e. I want to open something, for example) (sorry)

Arguably, many of these things could be expressed as a function, but some of them involve preserving valid structural relationships between the object and its dependencies, so I'd really prefer to be able to use __init__ in some cases.

I know, I can write my own __init__. I know! But if I do that, I give up not just on the convenience of attr.s writing the __init__ for me, but also the useful behavior of attr.s: i.e. validators, default factories.

All of my use-cases here would be easily accomplished with either a pre-__init__ and post-__init__ hook. Most of the ones I'm actually interested in would all be achievable with a post-__init__ hook; I think pre-__init__ might be pointless given the availability of convert=.

This has been discussed already, but some of the other issues (#33 #24 #38 #38 #58) which have raised the possibility of a post-__init__ hook have been somewhat conflated with better support for inheritance. As we all know, inheritance is deprecated and will be removed in python 3.7 so we don't need to focus on that.

As we all know, inheritance is deprecated and will be removed in python 3.7 so we don't need to focus on that.

I think the problem is that many libraries are based on inheritance. Gtk-python, PyQt, Django... I think it would reduce too much the use cases of attr if inheritance was totally not supported.

Re: __pre_init__ could be useful if you need to set up some state on the instance before validating for some reason

@Insoleet

@Insoleet I’m not sure what you’re trying to say; is there anything about this proposal contrary to that?

I'm not sure, I would just like to be sure that you guys takes inheritance into account and not just don't care about it :)

The __pre_init__ hook is important for PyQt for example. It is the first thing to do before doing any PyQt related thing in the constructor (see for example : http://stackoverflow.com/questions/12280371/python-runtimeerror-super-class-init-of-s-was-never-called )

Random idea time! What do y’all think about:

import attr

class C:
    x = attr.ib()

   @attr.hooks.post_init
   def i_dont_care_how_its_called(self):
       # ...

Positive:

It’s much cleaner than adding another argument to attr.s
More explicit. Someone who never heard of attrs can deduce the meaning immediately.
you can have multiple of them which is useful in subclassing

Negative:

can be rather confusing if multiple hooks are registered?

+1

I'd say just call them in order of definition within class... For
subclasses, after super().init()

On Aug 22, 2016 4:49 AM, "Hynek Schlawack" notifications@github.com wrote:

Random idea time! What do y’all think about:

import attr
class C:
x = attr.ib()

@attr.hooks.post_init
def i_dont_care_how_its_called(self):
# ...

Positive:

It’s much cleaner than adding another argument to attr.s

you can have multiple of them which is useful in subclassing

Negative:

can be rather confusing if multiple hooks are registered?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/hynek/attrs/issues/68#issuecomment-241350493, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAe2AJABwv2cDbn6tlD6dPimcn-9FhCrks5qiWKzgaJpZM4JpO43
.

I like the cleanness of the decorator idea, although I'd alias it attr.post_init.

attrs has never played too well with non-attrs superclasses (since we never call super), so I'm not too worried there, and this feature might even improve the situation by allowing users to manually do it themselves.

What about attrs superclasses? Do we call their hooks, and if so in which order? (Presumably MRO or reverse MRO. My gut feeling is reverse MRO, so the basest class first.)

What about attrs superclasses? Do we call their hooks, and if so in which order? (Presumably MRO or reverse MRO. My gut feeling is reverse MRO, so the basest class first.)

MRO. The only right way to cooperatively call an overriden method is super(MyClass, self).__init__(). In py3 this is built in to the language; it's what super().__init__(...) does. Reversing the MRO is just a mistake; the basest class is always object and you never want to upcall to its __init__.

I very much like the idea of this hook decorator, though! I don't like typing underscores, so my suggestion for the spelling would be @attr.after.init and (maybe later) @attr.before.init ("pre" and "post" are more jargon-y than "before" and "after", and are also prefixes rather than words, which means preinit would be closer to valid, but looks weird).

Regarding "if multiple hooks are registered" - my suggestion for the first iteration of this would be to just make registering a second hook into an immediate exception at registration time. If we come up with a non-confusing way to register multiple hooks, it's always fine to add it later.

MRO. The only right way to cooperatively call an overriden method is super(MyClass, self).init(). In py3 this is built in to the language; it's what super().init(...) does. Reversing the MRO is just a mistake; the basest class is always object and you never want to upcall to its init.

Just to clarify my gut feeling: usually when you have a class hierarchy, the first line in your __init__ will be a call to super() (in Java, for example, not doing this on the first line is a compile error if I remember correctly from a past life), then the business logic of that particular __init__. So __init__s get called in the MRO, but the business logic gets executed reverse MRO, with the most base class setting itself up first. Then subclasses can count on the superclass invariants holding.

Since attrs will be in charge of invoking hooks, the user doesn't get to have a choice.

class A:
    x = attr.ib()
    y = attr.ib()

    @after_init
    def after(self):
        # Set me up.
        # If I'm an instance of B, B.after has been called already.
        # Maybe I want to make sure y is greater than x.
        if self.x > self.y:
            self.x, self.y = self.y, self.x

class B(A):
    z = attr.ib()

    @after_init
    def after(self):
        # This gets called before A.after
        # Has the superclass set itself up?
        # Maybe I want to make sure z is greater than y, but I can't yet, because A.after might flip them.

Of course, even in Java you can call super() with the result of an expression or function IIRC, so the truth is you get to execute some code before the call to super anyway. And of course Python isn't Java.
¯_(ツ)_/¯

Have you guys thought about using __new__ for the attr logic, and then letting users use __init__ for themselves?

I've built a similar library (not nearly as impressive, but prior to this coming along), and that technique serves us very well

For serialization and deserialization, the expected protocol for __new__ is that it never requires any arguments. Violating this expectation is possible, but ugly. Not that you should use Pickle for most things, but it would be annoying to break the ability to use attrs with multiprocessing, for example, without a bunch of additional explicit work.

Two ideas.

The order of field declarations is known. So, you could initialize your fields deterministically with a factory, if that factory were given a self attribute.
But since "1." isn't entirely generic, having the generated __init__() call __post_init__() makes sense to me. It is explicit enough already that you use the class decorator so forcing users to put another decorator every time is counter-productive. Isn't attrs all about avoiding boilerplate?

Subclassing and the custom super() ideas all sound inferior to me. They are far less obvious which means I couldn't recommend using them to regular Joes (like myself).

A new painter appears! :D

At this I think the question is whether we go:

magic dunder method, or
decorator.

I’m not sure what you mean by subclassing/custom super, because it’s all just about people wanting to subclass non-attrs classes and call super on them. It’s not meant as a mean to solve the problem here but a goal to enable. :)

Thoughts

Magic Dunder

It’s more error prone since mistyping the name will result in nothing happening.
One hook per class. Or do we traverse the MRO to collect them all? I guess we have to, if the super classes depend on it?

Decorator

More explicit.
Allows for multiple hooks even throughout hierarchies. Meaning: attrs can collect all hooks and execute them.
That has the downside of possible conflicts though?

So personally, I feel like a decorator would be more attrs-style because it’s more explicit and less error prone and plays well WRT to intra-attrs subclassing.

I’d love thoughts on these two approaches.

I guess if someone offered me a good PR for Lukasz’s 1. point I’d begrudgingly merge it.

Well, a counter-argument to the typos in the name is that you can also mistype init, str, etc. and they will all happily do nothing.

As for allowing multiple hooks with a decorator, then you run into issues with ordering. I'd rather only have to override one hook, possibly calling super() inside it. While this also has problems, it's less custom and magical than decorators.

Yes but what about subclassing? What if someone relies on the hook being run to initialize an attribute? :-/

If you go with __post_init__ I think it's actually a lot more straightforward to deal with subclassing.

…reads through the whole previous discussion to make sure nobody else has raised this already…

The problem with __init__ and subclassing is that it doesn't have a fixed signature. So if the MRO might get reordered by diamond inheritance or what have you, you don't know what to call super().__init__ with; it's literally impossible to know in the general case. attrs actually has more information in __init__ than regular old Python code does, so it might actually be able to solve this problem, especially if the other superclasses are also attrs classes.

Let me posit such a solution, actually. Instead of trying to make __init__ cooperative, let's say attrs just makes all inherited attr.ibs part of your constructor. (Maybe it does this already? I dunno.) The important thing is that attrs defines an __init__ that doesn't upcall other __init__s at all, sidestepping the inheritance problem entirely. This uber-__init__ then calls __post_init__ once, at the end of the __init__.

Making __post_init__ itself cooperative is then very easy: it does have a fixed signature. Just super().__post_init__() at the beginning or end of your __post_init__ (depending on your behavioral dependencies); this will correctly invoke __post_init__ up and down the MRO, and you can either depend on your superclasses's post-init behavior, in which case you invoke it early, or you know you're dealing only with your own subclass's state, in which case you put it at the end.

After typing that all out and swapping the whole previous conversation into my brain again, I'm coming down more and more in favor of just a __post_init__ special method. In many ways it's less magical than the decorator-based solution.

attrs initializes parents attrs class's attributes without ever calling super(). It just inspects the list of arguments, including the parents.

So should pre-init and post-init follow the same rule as attrs' __init__? It gets called magically in the parent class by the generated __init__?

Should attr's generated init be like this:

def __init__(self, ...):
  self.__pre_init__()
  super(Ba, self).__pre_init__()  # do this for all parent classes
  # init attributes
  super(Ba, self).__post_init__()  # do this for all parent classes
  self.__post_init__()

Or should it be like this:

def __init__(self, ...):
  self.__pre_init__()
  # init attributes
  self.__post_init__()

And the defined __pre_init__ and __post_init__ have to call super().__pre_init__ and super().__post_init__, if needed?

@mathieulongtin

@mathieulongtin Okay so I just re-invented what it already does? :) (I have very lightly used inheritance with attrs once or twice, but never had a complex hierarchy and never inherited anything that required an __init__, or one attrs class from another.)

__pre_init__ is a separate problem; I'm not entirely sure we need it.

__post_init__, though, should be called exactly once at the end of the generated __init__. super() works fine for methods other than __init__; you either rely on your superclass's behavior or not, you either call it first or call it last. So let's not do anything magic, just let users implement inheritance with the regular semantics that are usually used for method overloading.

Honestly, the more I think about it, the more I think that with this hook attrs might actually fix the second biggest problem with subclassing that is specific to Python. (The first biggest being confusion about who owns the self namespace when inheriting across library boundaries...)

Hm how do you mean “other than __init__”? It would be nice if the feature could stop people from opening issues, that they want to use attrs with PyQT etc.

Fixed in #111! 🚲 🏠 🌈

Semi-related: Here's a ticket (and comment) describing how to run all post-init hooks in a full class inheritance hierarchy with attrs:

#167 (comment)

PS, for anyone as confused as I was about the end of the OP's post:

This has been discussed already, but some of the other issues (#33 #24 #38 #38 #58) which have raised the possibility of a post-__init__ hook have been somewhat conflated with better support for inheritance. As we all know, inheritance is deprecated and will be removed in python 3.7 so we don't need to focus on that.

^ That comment is a joke. Class inheritance is not deprecated in python and is not being removed. In fact the python docs at https://docs.python.org/3/tutorial/classes.html#inheritance say "Python classes provide all the standard features of Object Oriented Programming: the class inheritance mechanism allows multiple base classes, a derived class can override any methods of its base class or classes, and a method can call the method of a base class with the same name." and "Of course, a language feature would not be worthy of the name "class" without supporting inheritance."

Python is not removing class inheritance.

Python is not removing class inheritance.

The joke is only funny because it should, though.

Looks like we missed the boat for 3.7 but there's always Python 4!

Navigation Menu

Repository navigation

init hooks #68

72 remaining items

Thoughts

Magic Dunder

Decorator

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

init hooks #68

Description

Activity

Insoleet commented on Aug 21, 2016

wearpants commented on Aug 21, 2016

hynek commented on Aug 21, 2016

Insoleet commented on Aug 21, 2016

hynek commented on Aug 22, 2016

wearpants commented on Aug 22, 2016

Tinche commented on Aug 22, 2016

glyph commented on Aug 22, 2016

glyph commented on Aug 22, 2016

glyph commented on Aug 22, 2016

Tinche commented on Aug 22, 2016

max-sixty commented on Aug 22, 2016

glyph commented on Aug 23, 2016

72 remaining items

ambv commented on Nov 2, 2016

hynek commented on Nov 3, 2016

Thoughts

Magic Dunder

Decorator

ambv commented on Nov 5, 2016

hynek commented on Nov 5, 2016

glyph commented on Nov 5, 2016

glyph commented on Nov 5, 2016

mathieulongtin commented on Nov 5, 2016

glyph commented on Nov 5, 2016

glyph commented on Nov 5, 2016

hynek commented on Nov 7, 2016

hynek commented on Nov 20, 2016

Arcitec commented on Sep 29, 2019

Arcitec commented on Sep 29, 2019

glyph commented on Sep 30, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions