Typeshed for WebOb is missing `webob.compat` module

[fmigneault]

For example, the following is valid from webob.compat import cgi_FieldStorage, but is flagged by IDEs such as PyCharm as missing from the stubs.

Other stubs from webob are found accordingly.

[AlexWaygood]

PR welcome!

[AlexWaygood]

On second thought, though: are you sure the .compat submodule is meant to be public API? Are you meant to be importing things from webob.compat, as an end user? It might be a regresion in user experience if IDEs started suggesting this module in autocompletions if it's not intended to be public API. It seems like it might be mainly a module providing compatibility for Python 2? https://github.com/Pylons/webob/blob/0a69ff129df2da9bba347981c4ef632a8b7c2ddb/src/webob/compat.py#L12

[JelleZijlstra]

Our general policy is to add private objects when users ask for them, so I think we should include this. I agree it's questionable whether it's a good idea to use this module in user code.

[fmigneault]

The specific situation I have is that I employ request.POST.get("file") for an endpoint that supports uploading a file, and that results into this cgi_FieldStorage object. Without the explicit type cast with cgi_FieldStorage, I get more linting errors and missing properties than if I omitted it. I agree in hiding away some magic that could happen to hide interfaces of _cgi_FieldStorage (note underscore), but the methods (notably read_multi for multipart content) are important interfaces for my use case.

What would be the recommended strategy to define this type, since it mostly derives from https://github.com/python/typeshed/blob/main/stdlib/cgi.pyi that is already defined (except the extra read_multi) ?

[AlexWaygood]

Our general policy is to add private objects when users ask for them

Absolutely -- I just wanted to double-check that it was actually needed. It sounds like it is, so I consider my concerns as having been addressed :)

[Daverball]

@fmigneault I would suggest not doing this and instead use the dummy type webob.request._FieldStorageWithFile I created for this:

if TYPE_CHECKING:
    from webob.request import _FieldStorageWithFile

It's worth noting that the cgi Module is deprecated and a future version of WebOb will likely replace this with a different object located somewhere else. This way the stubs can just update this internal alias and point to the new type, so you don't have to change all your imports.

Apart from cgi_FieldStorage the entire compat module has already been removed upstream. It's really an implementation detail and should not be exposed in typing information.

It's also worth noting that you don't need to cast at all. You can ensure the type checker infers the correct attributes this way:

file = request.POST.get("file")
if file is not None:
    assert not isinstance(file, str)  # has to be a file upload
    file.file  # works
    file.filename  # works

The only time you really should have to worry about importing this type, is when you want to write a function that takes a fieldstorage that has a file attached to it, otherwise you can just use cgi.FieldStorage and it will pass as well, since webob.request._FieldStorageWithFile is compatible with cgi.FieldStorage.

It might be worth adding a comment to the type definition, so it's easier for people to find out how to best make use of these stubs.

Or alternatively we do add it, but with a deprecated decorator explaining why you shouldn't use it, and what you should do instead. Although I'm not sure if deprecated triggers for types being used inside an annotation or if it only triggers for runtime use.

[fmigneault]

I'm fine with @Daverball's suggestion.