Pact 101 — Getting started with Pact and Consumer Driven Contract Testing
So you heard about Pact and want to get started. This guide should hopefully get you going in the right direction.
What is Pact?
The Pact family of testing frameworks (Pact-JVM, Pact Ruby, Pact .NET, Pact Go, Pact.js, Pact Swift etc.) provide support for Consumer Driven Contract Testing between dependent systems where the integration is based on HTTP (or message queues for some of the implementations). They are particularly useful for µ-services, where there can be lots of inter-dependent services and integration testing quickly becomes unfeasible.
Before we can get deeper into Pact, you need to get an understanding of Consumer Driven Contracts.
What are Consumer Driven Contracts?
Consumer Driven Contracts is a pattern or methodology where contracts between inter-dependent services are designed from the perspective of a service consumer. The main article for it is Consumer-Driven Contracts: A Service Evolution Pattern, but it is a bit dated with respect to technology (it talks a lot about XML and Schemas). But the concepts expressed in the article are relevant to the computer systems we build today.
I find a lot of the time the teams who are building µ-services or internal APIs follow the same thinking as for external or public APIs. The service provider or API team think of all the things that any consumer of their service would need (making many assumptions along the way) and then provide the service along with documentation on how to use it. They build it and hope the consumers will come.
When they later need to change it, they make the changes and then publish it along with instructions that all consumers need to follow to use the updated API. They may include multiple versions of the service to protect consumers from breaking changes.
I witnessed first-hand two teams developing and consuming an API. It took them a number of days to get the interaction working in a integration testing environment after building the consumer and provider because the consumer team assumed the API would be based on posting a JSON document (it was being invoked from a web browser), while the provider team implemented an application/x-www-form-urlencoded POST API.
But the irony is in the case for internal APIs and services, it is not that hard to work out who all the consumers of your service are and what their requirements may be. Most of the time, they all work for the same organisation.
Consumer Driven Contracts turns this all around. The contract for the interaction is developed by the consumer of a service.
The benefits to this approach are that you know who all the consumers of your service are, you’ll know when you make breaking changes so it is easier to make changes and the contract is also a form of documentation. Secondly, you’ll know exactly what functionality needs to be provided so you can implement the right services needed by your consumers and can follow the YAGNI principle.
The development of the interaction starts from the consumer side, hopefully with a test. The consumer team defines the contract for the interaction and implements the service consumer based on the contract.
This contract is then provided to the provider team (in some cases it is the same team) and the provider service is implemented to fulfill the contract.
What do I need to do to get started?
Most of the information can be found on the Github pages for the various Pact implementations. The Ruby Pact one has a lot of information in the wikis.
- Ruby Pact: https://github.com/realestate-com-au/pact
- JVM Pact: https://github.com/DiUS/pact-jvm
- .Net Pact: https://github.com/SEEK-Jobs/pact-net
- Pact Go: https://github.com/pact-foundation/pact-go
- Pact.js: https://github.com/pact-foundation/pact-js
- Pact Swift: https://github.com/DiUS/pact-consumer-swift
- Pact Python: https://github.com/pact-foundation/pact-python
In particular, to get an understanding of how Pact works, read: https://github.com/realestate-com-au/pact#how-does-it-work.
Start with a Consumer Test
It is all about the Consumer, so start there. There is support for a number of test frameworks, so depending what test tooling you use, read up on the docs for that.
For Ruby and RSpec, read Simplifying micro-service testing with pacts for an example.
For JVM based test frameworks (JUnit, Groovy, etc.), have a look at https://github.com/DiUS/pact-jvm#service-consumers. The JUnit one seems to be the most popular. There are lots of example tests.
Once you have a running consumer test, you should have pact files being generated. These are the contracts in the Consumer Driven Contracts sense. You can publish these files in a number of ways. Common ways would be to commit them to your source repository, upload them to a file server, store them as artifacts in your CI build or upload them to a Pact Broker.
Verify your provider
The second half of the approach, is to verify that your provider actually behaves in the way that the consumers expect. For each consumer that has implemented a consumer test, you should have a pact file published somewhere. Typically, you want to verify your provider in its CI build, so you’ll know when changes have broken the interaction with a consumer, and you’ll also know which consumer you have impacted.
Given a set of published pact files, there are two main ways to verify that your provider adheres to the contracts encapsulated in the pact files.
Using a build plug-in to verify pacts
The first way is to use a build plug-in that can execute the requests in a pact file against your provider. In some cases (like the Ruby Rake tasks), the provider is started from within a test harness (Rack Test for ruby), the requests are replayed and then the responses compared to the expected responses from the pact file. In other cases (like the Maven and Gradle plugins), the actual provider needs to be running and the build plug-in makes the actual requests to the running provider.
For an example of using the Ruby Rake tasks for a Ruby provider, read “The Provider”section of Simplifying micro-service testing with pacts.
The readmes for the JVM build plug-ins (Maven, Gradle, Leiningen and SBT) contain more information on how to use those tools. In most cases, you need to be able to start your provider beforehand and stop it afterwards, and have a way of providing test data required by the pacts. Refer to the Service Providers section of the Pact-JVM read-me for more information.
Pact has a state change mechanism for controlling the state of the provider during the verification test run. Essentially, this is a hook that can be called before each request with a description of what the expected state that provider needs to be in to be able to successfully handle the request. Examples would be things like “User Andy should exist”or “The order table should be empty”. The Ruby implementation allows setup and tear-down blocks to be defined, while the JVM build plug-ins allow a state change URL to be defined which will receive a POST request with the state description before the actual request is made.
Using a testing framework to verify pacts
The biggest disadvantage in using some of the build plug-ins, is that you need to have your provider running, and you need to be able to setup test data. This involves either preloading fixture data required by all the pacts beforehand or using the state change mechanism to change the provider’s state on the fly. Both methods require a fair amount of orchestration, especially in a CI environment.
Thanks to a community contribution, a Pact JUnit provider runner is available for people using a JVM based provider. It allows you to exercise only the portion of your provider code that deals with handling the requests, and you can use the JUnit setup and tear-down mechanisms as well as a state annotation marker to setup data required for the pacts. You can also use the standard mocking frameworks to stub out dependencies, although I would add a caution to make sure you are not affecting behavior with your mocks and stubs. The README contains more information.
Where can I get support?
As with most open-source projects, Pact has a vibrant online community. There is a Google group and a Gitter channel for the Ruby and JVM versions (click on the Gitter images on the Github pages). I would recommend searching through or posting your questions there, someone will be sure to answer.
- Stack Overflow: https://stackoverflow.com/questions/tagged/pact
- Google users group: https://groups.google.com/forum/#!forum/pact-support
- Gitter: https://gitter.im/realestate-com-au/pact
- Twitter: @pact_up