Functional API tests for OpenStack core projects are currently maintained in the Tempest project. This is in keeping with Tempest's role in performing post-development QA, and may work well if a given API is developed monolithically. And while tests maintained in Tempest may duplicate some of the testing that is done by an individual project, a common perception is that this duplication serves as a valuable check against unintended regressions in backwards compatibility.
The current approach has some drawbacks, however. Reviewers for a feature that has an API impact can't require that API testing be ready before approval, since Tempest tests can only be run post-merge. The arrangement also complicates test-driven development (TDD) since a feature and its API tests have to be maintained in separate repos. And while a monolithic API may suit the current approach of post-dev QA, it hinders an iterative approach to API development that is being considered by projects like Neutron.
The intent of this session is to discuss the possibility of allowing functional api tests to be maintained in the tree of a given core project instead of in Tempest. Such tests would still need to be runnable by Tempest (e.g. by copying ala oslo or by depending on the core project), and would rely on Tempest-provided infrastructure. Technical considerations aside, the specifics of a project's maintenance of these critical tests would need to be agreed upon to ensure that removing test duplication did not increase the potential for regressions in backwards-compatibility.
