Contract Testing Serverless and Asynchronous Applications

Ron Holsausen

With the move to microservices, or µ-services for the more stylish, Consumer Driven Contract testing with Pact evolved as a strategy to be able to determine if all of our services will talk correctly to each other. Without requiring all of our services to talk to each other. It also allowed us to identify who the consumers of any service were. This was helpful, because if you know who your consumers are, you know who to talk to when you want to break something. Or when they break something.

We did this for HTTP-based microservices. Then we added some message queues and lambda functions to the mix. Just because we could.

Asynchronous communication via message queues

Some of the services we were developing started communicating over message queues in an asynchronous manner. It became obvious that it was now even more important to know who the consumers of a message are. With synchronous communications, if there is a failure it is obvious who the participants are. With message queues, you may not know who is consuming your message or if any failure was caused by a change to your message.

It felt to me that the typical attitude was to generate the message, throw it onto the queue and it was no longer your problem. Seems pretty similar to the thinking we were trying to change with Restful APIs and microservices.

I witnessed a system failure with the services between two teams caused by a incorrectly formatted date in a message. This took days to resolve with multiple deployments to a QA environment.

The question on how to ensure the services communicating over a message queue were not impacted by incorrectly formatted messages got me thinking about contact tests. Having a contract test to ensure that both the message producer and consumer could communicate correctly seemed like the right idea. But the problem was more complex than for HTTP-based services, where we could mock things at the HTTP level. Everything spoke HTTP. When it comes to message queues, depending on the message queue used, the protocol would be different. And I didn’t want to have to create a mock ActiveMQ, RabbitMQ, SQS, Kafka and Kinesis servers, to name but a few.

This lead to the realisation that for the purpose of verifying a contract, the underlying message queue is mostly irrelevant. As long as you can ensure that the message producer is generating a correctly formatted message, and the consumer is able to consume it, you don’t need an actual message queue. This made me happy.

So I created the message pact.

This works in a similar mechanism to regular Pact tests. We create a consumer test, generate and publish a pact file and then verify that our message provider generates the correct message.

The Consumer Test

Most of the message queue consumers are implemented using a callback mechanism, normally some kind of message listener that receives the message from the message queue via integration with a SDK library. To be able to test my consumer, I divided the consumer into two classes, a message listener that integrates with the message queue libraries, and a message handler which processes the message. The message listener does nothing apart from passing the message on to the handler. In this way, I could write a consumer contract test with just the handler that does not need a message queue.

The consumer contract test works in a similar way. I specify the messages that I expect to receive, and then get the Pact test framework to invoke the message handler with each message. So far this has worked for testing Kafka, SQS and Kinesis message consumers.

Below is an example of a JVM message handler that will process message from a Kafka topic as a Spock test. But you should be able to use any testing framework, like JUnit or TestNG.

Step 1 — Define the message expectations

The consumer test starts with defining the message expectations. Basically, we’re setting up the message we expect to receive. We use the PactMessageBuilder to setup the pact and messages.

given:
def messageStream = new PactMessageBuilder().call {
    serviceConsumer 'messageConsumer'
    hasPactWith 'messageProducer'
    given 'order with id 10000004 exists'
    expectsToReceive 'an order confirmation message'
    withMetaData(type: 'OrderConfirmed') // Can define any key-value pairs here
    withContent(contentType: 'application/json') {
        type 'OrderConfirmed'
        audit {
            userCode string('messageService')
        }
        origin string('message-service')
        referenceId regexp('\\d+\\-\\d', '10000004-2')
        timeSent timestamp
        value {
            orderId regexp('\\d+', '10000004')
            value decimal(10.00)
            fee decimal(10.00)
            gst decimal(15.00)
        }
    }
}

Step 2 — Call the message handler with the generated messages

This example tests a message handler that gets messages from a Kafka topic. In this case the Pact message is wrapped as a Kafka MessageAndMetadata class.

when:
messageStream.run { Message message ->
    messageHandler.handleMessage(new MessageAndMetadata('topic', 1,
        new kafka.message.Message(message.contentsAsBytes()), 0, null, valueDecoder))
}

Similarly, for an example where the handler receives the message from SQS as an Object that has been de-serialised by a library like Jackson, you could do something like:

when:
messageStream.run { Message message ->
  def order = objectMapper.readValue(message.contentsAsBytes(), Order)
  assert messageHandler.handleMessage(order) != null
}

Here objectMapper is the ObjectMapper from the Jackson library. It is important that the message get de-serialised inthe same way as when the actual message is received.

Step 3 — Validate that the message was handled correctly

Our handler should have taken the message and persisted the order details to our order repository, so we can check that to see that it worked ok.

then:
def order = orderRepository.getOrder('10000004')
assert order.status == 'confirmed'
assert order.value == 10.0

Pact is not doing anything fancy here, it is just converting the expectation from the DSL into a JSON document, and passing that to some code to be executed. In essence, it is pretending to be a message queue. If everything passes, the pact file is generated. This is the important bit, because we can now publish it.

Verify our Message Provider

The next step to close the loop is to verify that the provider service generates the message correctly. We do this by getting Pact to work in the opposite fashion as in the consumer test. Again, Pact will pretend to be a message queue and get the message provider to send it a message. This will be matched against the published pact file.

We do this by splitting the message sending code from the message generation code. That way the class that is responsible for putting the message on the message queue delegates to another class to generate the actual message. We can then call the latter class to generate a message during the verification test to determine if it generates the message correctly. The only caveat is that the class tested in this way must be the one that is actually used to generate the real message.

Pact-JVM uses Java annotations to find a test class that can call the message generator and return the generated message. If you are using JUnit, you can use an AmqpTarget to drive this behaviour instead of the HttpTarget used for regular pact verification tests. See also Verifying a message provider in Gradle and Verifying a message provider in Maven for information on how to enable this functionality in those tools.

Now when we verify the message pact file, Pact-JVM will look for methods annotated with @PactVerifyProvider in the test class path that have a matching description to what is in the pact file. In our previous example, it was an order confirmation message.

So, here is an example to verify the message from the previous consumer test.

class ConfirmationKafkaMessageBuilderTest {
  @PactVerifyProvider('an order confirmation message')
  String verifyMessageForOrder() {
      Order order = new Order()
      order.setId(10000004)
      order.setExchange('ASX')
      order.setSecurityCode('CBA')
      order.setPrice(BigDecimal.TEN)
      order.setUnits(15)
      order.setGst(15.0)
      order.setFees(BigDecimal.TEN)
      def message = new ConfirmationKafkaMessageBuilder()
              .withOrder(order)
              .build()
      JsonOutput.toJson(message)
  }
}

In this case, ConfirmationKafkaMessageBuilder is the class that is used to generate the message that is sent to theKafka topic. But this can be used for any message queue, as we are not using anything Kafka specific. Pact-JVM will now call the verifyMessageForOrder method and validate that the returned contents matches the contents for the message in the pact file.

In Summary

We can use a strategy like contact tests to verify that a consumer and provider communicating over a message queue work without requiring an actual running message queue. We do this with the following steps:

1. Split out message consumer into a queue specific message handler and a class that processes the message.
2. Write a consumer test that calls the message processing class.
3. Publish the resulting pact file.
4. Split our message provider into a queue specific publisher and a message generator.
5. Setup our verification (with Pact-JVM) to invoke methods on test classes annotated with @PactVerifyProvider
6. Get our annotated test method to return the message JSON from the message generator.

Now we can catch those date format issues before they’re even deployed.

In part two of this blog, I’ll show you how I used this same technique to verify contracts with AWS lambda functions.