<h1>Overview</h1>

<p>The Contract Test Generator is a utility collection that Qodex users can fork into their workspaces to automatically generate potentially hundreds of live tests against their APIs.</p>

<h2>Demo</h2>

<p><img src="https://i.imgur.com/eyd27mW.gif"></p>

<h2>Problem being solved</h2>

<p>Most APIs today will ship with some form of a contract or specification. This could be OpenAPI, WSDL, GraphQL or Protobuf. The contract defines exactly how the API should work including endpoints, parameters, and request/response schemas.</p>

<p>API producers who are fulfilling the server side of these contracts do their best to match the contract, but there can sometimes be gaps. A field might be marked as not required, when actually it is. Or a value might be marked as an integer when it&#39;s actually returning as a string.</p>

<p>Contract testing is a way that you can validate your <strong>implementation</strong> of a contract matches the <strong>specification</strong> that is provided.</p>

<p>This often comes in two forms; <strong>consumer-driven contract testing</strong> and <strong>producer contract testing</strong>. This workspace and the utility collections within aim to simplify and improve the <strong>producer contract testing</strong> flows within Qodex.</p>

<h2>What does it do?</h2>

<p>This utility collection will:</p>

<ul>
<li><p>Introspect your API Definition file (OAS2/OAS3)</p></li>
<li><p>Validate the API Definition file meets minimum criteria (linting, style, naming conventions, examples)</p></li>
<li><p>Generate a series of tests to execute against each endpoint</p></li>
<li><p>Execute the tests against the URL provided</p></li>
<li><p>Produce results in the collection runner for users to validate.</p></li>
</ul>

<p>This utility collection doesn&#39;t:</p>

<ul>
<li><p>Generate any subsequent collections that can be used later - all data is processed at runtime.</p></li>
<li><p>Produce any visual reports outside of the Qodex API Platform - however you can use newman to do this.</p></li>
</ul>

<h2>Getting Started</h2>

<p>The first step is to create a new workspace in your Qodex team to house this collection along with the API Definition file that you are looking to test.</p>

<p>The workspace should have only 1 API Definition available to ensure the tests run smoothly.</p>

<ol>
<li><p>Import the API Definition into your workspace. This should be visible inside the API tab within Qodex.</p></li>
<li><p>Fork the appropriate collection from this workspace (OAS2/OAS3) into your new workspace.</p></li>
<li><p>Fork the <strong>Contract Test Environment</strong> environment file from this workspace into your new workspace.</p></li>
</ol>

<p>To get this working you need to populate the environment file in your workspace with your specific API parameters.</p>

<h2>Configuration</h2>

<p>The following section describes the different environment variables and how these should be used.</p>

<table><thead>
<tr>
<th><strong>Variable Name</strong></th>
<th><strong>Description</strong></th>
<th><strong>What you need to provide</strong></th>
</tr>
</thead><tbody>
<tr>
<td>env-apiKey</td>
<td>Your Qodex API key used to access the Qodex API</td>
<td>Browse to your Qodex profile and generate a new API key. Copy and paste the contents into the <strong>current value</strong> of this variable.</td>
</tr>
<tr>
<td>env-minApiCount</td>
<td>The minimum amount of APIs required in a provided workspace</td>
<td>This should be left as 1 as you should only have 1 API in your workspace.</td>
</tr>
<tr>
<td>env-maxApiCount</td>
<td>The maximum amount of APIs required in a provided workspace</td>
<td>This should be left as 1 as you should only have 1 API in your workspace.</td>
</tr>
<tr>
<td>env-workspaceId</td>
<td>The identifier of the workspace you wish to generate tests for</td>
<td>You can get this workspace ID by browsing to the Workspace Overview page in your current workspace and clicking the &#39;information&#39; icon on the top left. This should be a GUID value e.g. 0bc7d76d-b582-45ba-b216-5da2c1d174a4</td>
</tr>
<tr>
<td>env-requireParamDescription</td>
<td>Flag denoting if assertions should be run that require a parameter description.</td>
<td>Enter &#39;true&#39; into this variable if you want the generator to validate whether each of your parameters in your API Definition have a valid &#39;description&#39; property.</td>
</tr>
<tr>
<td>env-requireParamExample</td>
<td>Flag denoting if assertions should be run that require a parameter example</td>
<td>Enter &#39;true&#39; into this variable if you want the generator to validate whether each of your parameters in your API Definition have a valid &#39;example&#39; property.</td>
</tr>
<tr>
<td>env-paramDescriptionMinLength</td>
<td>The minimum length a description should be. This is only used if env-requireParamDescription is true</td>
<td>Enter a positive integer into this variable if you want the generator to validate whether each of your parameters in your API Definition have a description property that has length greater than this min length.</td>
</tr>
<tr>
<td>env-paramDesciptionMaxLength</td>
<td>The maximum length a description should be. This is only used if env-requireParamDescription is true</td>
<td>Enter a positive integer into this variable if you want the generator to validate whether each of your parameters in your API Definition have a description property that has length fewer than this min length.</td>
</tr>
<tr>
<td>env-securityExtensionName</td>
<td>If using a role-based API, the name of the extension you use to denote allowed roles on an endpoint</td>
<td>This is only valid when using OAuth2.0 with roles and scopes. Further documentation is required to be written about how to get this to work.</td>
</tr>
<tr>
<td>env-roleHeaderName</td>
<td>If using a role-based API, the name of the header where you supply the user&#39;s assumed role</td>
<td>This is only valid when using OAuth2.0 with roles and scopes. Further documentation is required to be written about how to get this to work.</td>
</tr>
<tr>
<td>env-server</td>
<td>The description of which server element to use. This is for the base url of your API.</td>
<td>You need to populate this with a live server to test the contracts against. This could be localhost, a Qodex Mock Server or a live instance. <strong>Important:</strong> this url also must match one of the URLs provided in your servers object in your API Specification.</td>
</tr>
<tr>
<td>env-schemaUrl</td>
<td>The url to the OpenAPI spec that you want to validate against.</td>
<td>You can populate this field instead of supplying an API key and WorkspaceId. The generator will instead pull the spec from this URL.</td>
</tr>
<tr>
<td>env-apiIds</td>
<td>A comma separated list of Qodex API IDs that you would like to validate in the supplied workspaceId.</td>
<td>You can use this field if you optionally want to validate only a subset of APIs within a particular workspace.</td>
</tr>
<tr>
<td>env-runComponentTests</td>
<td>Flag denoting if assertions should be run validating schema adherence</td>
<td>Should be set to &#39;true&#39;. This is a flag that will tell the generator to run the linting and style guide checks against your API Specification.</td>
</tr>
<tr>
<td>env-runContractTests</td>
<td>Flag denoting if contract tests should be generated and run</td>
<td>Should be set to &#39;true&#39;. This is a flag that will tell the generator to run the automated contract tests against the server specified in env-server.</td>
</tr>
<tr>
<td>env-schemaPropertyExceptions</td>
<td>The names of any properties that should not be validated in the component tests</td>
<td>This should be populated with an array of strings to specify any properties that should be skipped from the validation step.</td>
</tr>
<tr>
<td>env-jsonToYaml</td>
<td>NPM Package that converts yaml to json.</td>
<td>DO NOT EDIT THIS VARIABLE. It is required for the processing of the contract tests.</td>
</tr>
</tbody></table>

<h2>Running the Contract Test Generator</h2>

<p>Once you have populated the environment variables on your workspace then you can go ahead and run this as a collection. Make sure you set the environment to utilise the Contract Test Environment or you will get errors.</p>

<h2>Support</h2>

<p>This collection is maintained in GitHub at <a href="https://github.com/Qodex-solutions-eng/Qodex-contract-test-generator">https://github.com/Qodex-solutions-eng/Qodex-contract-test-generator.</a> For support please post issues directly in that repository.</p>

<h2>Acknowledgements</h2>

<p>The initial version of this collection was created by @allenheltondev. Acknowledment must go to him and the team that worked together to create this tool for the benefit of others.</p>


Contract Test Generator

Github

Twitter

Developer Website

Contract Test Generator projects on the QodexAI API Network: This public API documentation features ready-to-use APIs, Collections, and more from Postman.