Juno Design Summit has ended
Back To Schedule
Thursday, May 15 • 1:30pm - 2:10pm
API tests with JSONSchema

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Now Tempest contains two kinds of API definitions with JSONSchema for Nova project.
One is used for generating negative tests automatically.
The other one is used for checking the response body in API test.

API definitions of the first one represent both the request body formats and error
response codes(HTTP 404, etc.) of each API. Tempest operates negative API tests based
on them and check whether the response code is the expected one. The API definitions
is being implemented now.

The second ones represent both the response body formats and success response codes(HTTP200,
etc.) of each API. The REST clients check the response based on each API definition and if
receiving a bad response(lack of necessary parameters, etc.), Tempest raises an exception.
By this behavior, Tempest is blocking the backward incompatible changes for Nova project.

I'd like to discuss some points for this topic:

* How to integrate API definitions
API definitions of negative tests are stored under etc/schemas/compute/ and the other
ones are done under tempest/api_schema/compute/. The description formats are different,
but it would be nice to describe both of them in each API file for its maintenance.

* Port API definitions
Nova also contains API definitions of JSONSchema for the request validation of v3 API.
So we will be able to port API definitions of request from Nova to Tempest for negative tests.

On the other hand, it would be good to port API definitions of response from Tempest to Nova
for API documentation. Nova contains template files of API response body and they have/should
been used for Nova original integrated tests(not Tempest), but these template files sometimes
were wrong because some tests did not use them and Nova tests could not detect them.
If porting from Tempest and using them for the tests, it would make Nova test quality better
because these definitions are already verified.

* Apply this framework to the other projects?
JSONSchema is a common python library and we use it for Tempest.
As the above description, we can reuse API definitions, which are described with JSONSchema,
for multiple use cases. However, the other project(Ceilometer, Ironic, etc.) is implemented
with WSME instead of JSONSchema. So if applying this to API tests of these projects, we use
the definitions once only.

(Session proposed by Ken'ichi Ohmichi)

Thursday May 15, 2014 1:30pm - 2:10pm EDT

Attendees (0)