Repository navigation
Closed
Labels
area/unmarshallingIndicates an issue on unmarshalling area.Indicates an issue on unmarshalling area.kind/bug/confirmedkind/missing-implementation
Description
I'm still working on minimal reproducer to prove this out so apologies for the lack of a one in advance.
I've got a schema like this:
openapi: '3.1.0'
# ...
components:
schemas:
PatchList:
type: 'object'
properties:
# ...
pull_url:
title: 'Pull URL'
type:
- 'string'
- 'null'
format: 'uri'
maxLength: 255
# ...If my server returns a response with pull_url set to null, then the call to openapi_core.validation.response.validators.ResponseValidator.validate fails with the following traceback:
Traceback (most recent call last):
File "/dev/patchwork/patchwork/tests/api/test_relation.py", line 267, in test_remove_one_patch_from_relation_good
resp = self.client.patch(
File "/dev/patchwork/patchwork/tests/api/utils.py", line 241, in patch
validator.validate_data(
File "/dev/patchwork/patchwork/tests/api/validator.py", line 141, in validate_data
result.raise_for_errors()
File "/dev/openapi-core/openapi_core/validation/datatypes.py", line 12, in raise_for_errors
raise error
File "/dev/openapi-core/openapi_core/validation/response/validators.py", line 245, in validate
data = self._get_data(response, operation_response)
File "/dev/openapi-core/openapi_core/validation/response/validators.py", line 88, in _get_data
data = self._unmarshal(schema, casted)
File "/dev/openapi-core/openapi_core/validation/validators.py", line 76, in _unmarshal
return unmarshaller(value)
File "/dev/openapi-core/openapi_core/unmarshalling/schemas/unmarshallers.py", line 91, in __call__
return self.unmarshal(value)
File "/dev/openapi-core/openapi_core/unmarshalling/schemas/unmarshallers.py", line 434, in unmarshal
return unmarshaller.unmarshal(value)
File "/dev/openapi-core/openapi_core/unmarshalling/schemas/unmarshallers.py", line 316, in unmarshal
properties = self.format(value)
File "/dev/openapi-core/openapi_core/unmarshalling/schemas/unmarshallers.py", line 325, in format
return self._unmarshal_properties(formatted, schema_only=schema_only)
File "/dev/openapi-core/openapi_core/unmarshalling/schemas/unmarshallers.py", line 354, in _unmarshal_properties
all_of_properties = self._clone(all_of_schema).format(
File "/dev/openapi-core/openapi_core/unmarshalling/schemas/unmarshallers.py", line 325, in format
return self._unmarshal_properties(formatted, schema_only=schema_only)
File "/dev/openapi-core/openapi_core/unmarshalling/schemas/unmarshallers.py", line 373, in _unmarshal_properties
properties[prop_name] = self.unmarshallers_factory.create(
File "/dev/openapi-core/openapi_core/unmarshalling/schemas/unmarshallers.py", line 91, in __call__
return self.unmarshal(value)
File "/dev/openapi-core/openapi_core/unmarshalling/schemas/unmarshallers.py", line 433, in unmarshal
unmarshaller = self._get_best_unmarshaller(value)
File "/dev/openapi-core/openapi_core/unmarshalling/schemas/unmarshallers.py", line 430, in _get_best_unmarshaller
raise UnmarshallerError("Unmarshaller not found for type(s)")
openapi_core.unmarshalling.schemas.exceptions.UnmarshallerError: Unmarshaller not found for type(s)If I comment out format, things work as expected. I suspect it's attempting to apply a formatter for the null values. As far as I can tell, this should work though.
Metadata
Metadata
Assignees
Labels
area/unmarshallingIndicates an issue on unmarshalling area.Indicates an issue on unmarshalling area.kind/bug/confirmedkind/missing-implementation
Activity
stephenfin commentedon Jan 24, 2023
This post and this SO answer suggests I could use
oneOfto do this, but it seems we're not able to do so here:This yields the exact same exception.
stephenfin commentedon Jan 24, 2023
Things do, however, work if I specify
typetwice:This shouldn't be necessary though, right?
p1c2u commentedon Jan 25, 2023
Hi @stephenfin
thanks for reporting the issue.
The first example doesn't work because uri is string type format only so if you define multiple types it will fail with others (in this case null). This works as expected (but still not ideal).
The second example here is null is not considered as valid if type is not defined in main schema. Here we have the issue.
The third one is just a workaround for the second example issue which (as you noticed) shouldn't be necessary.
stephenfin commentedon Jan 25, 2023
@p1c2u Thanks for confirming that the latter two examples are what we'd expect: the OpenAPI spec is seriously lacking when it comes to discussions on nullable attributes 😞 Is there anything I can do to help? I did try to address this but it seems there have been significant changes in the 0.17 beta and my test suite has yet to be adapted for this so I was kind of flying blind.
stephenfin commentedon Jan 25, 2023
For what it's worth, the first example is rather common too, even if it's technically correct. For example, the OpenAPI 3.1 spec for GitHub's own API contains things like this:
In this case,
itemsonly applies fortype: array, nottype: null, so this doesn't seem valid either. It's common though. Given the spec is silent on this matter, I wonder if this is something we want to choose to accept also?stephenfin commentedon Jan 25, 2023
Hmm, I actually think JSONSchema allows the first example (henceforth option A). From the JSON Schema Core spec:
If this is true, I guess we should support it somehow?
EDIT: Also, from the JSON Schema Validation spec:
(emphasis mine)
p1c2u commentedon Jan 25, 2023
@stephenfin 0.17 should be mostly compatible except validation errors. Can you tell which changes do you have in mind?
First option is definitely correct the problem is current implementation of format validators and unmarshallers which don't allow to handle multiple types with format correctly. I started the refactor of this part.
stephenfin commentedon Jan 25, 2023
I'm using the Django validators and seeing the following exceptions when installing the
openapi-core-0.17.0a1pre-release (pip install --pre --upgrade openapi-core):I suspect I just need to update how I'm creating my validators. No big deal.
Excellent. Happy to test this when it's available! 🙋
p1c2u commentedon Jan 25, 2023
Oh I see you use low level integration. You must pass spec at first argument to
RequestValidator/ResponseValidatorbut probably I will make it backward compatible in the future pre-rereleases.stephenfin commentedon Jan 25, 2023
Spot on. Thank you as always. This is an excellent collection of tools 😊