Good design from the ground up — how I/O library design can make or break IoT solutions

Recently, as more and more players are entering the Internet of Things (IoT) field, there has been an influx of not-so-well-thought-out networking or general I/O libraries. Even certain large vendors which claim to be focusing on small embedded systems have been missing the mark significantly by providing libraries that are at best cumbersome and memory intensive, and at worst not really fit for purpose.

Rather than naming and shaming, I wanted to put together a short post on what it is that so many are missing, because I suspect it simply comes down to a lack of awareness of the alternative. It’s one of those things which is obvious once you’ve been exposed to it, but until then might not cross your mind.

So what am I talking about? It’s the iovec concept, known both as “vectored I/O” and “scatter/gather I/O”.

What this means is simply that instead of assuming that all the data to be sent resides in contiguous memory, it needs to be gathered from several separate buffers. Conversely, for receiving incoming data, the destination buffer is not assumed to be contiguous but rather the read data needs to be scattered to several buffers. Instead of passing a single buffer to the read or write function, a vector of buffers is passed (hence the term “vectored I/O”). In short it allows a single call to read or write data to and from many buffers at once; useful for bunching together fields of different data structures to form one I/O transaction, or slicing protocol headers.

A similar but slightly alternative approach is to design the library with an interface which allows incremental composing of requests, though in practice this in turn often requires extra memory within the library for buffering there instead. The gains are not as easy as with an iovec style interface.

When it comes to I/O library interfaces then, the lowest building block should be one which takes a vector of buffers, rather than a single buffer. There are two big reasons for this. The first is that it is trivial to implement an interface for sending a single buffer on top of an iovec interface, but often impossible to implement an iovec interface on top of a single buffer interface. The second reason cuts to the heart of the matter — an iovec interface can allow for vastly reduced memory usage, depending on the protocol/situation.

Typically, when you’re sending network protocol data you’re having to include a few logically separate parts — some sort of header followed by a body payload, with perhaps also a trailer for checksums or such. And even within each of those, there may be logical separation, such as HTTP request which has the request line followed by a number of header field lines, and an optional body after that. Some of these parts are often constant and do not change.

As a somewhat contrived and seriously cut down example, consider a case where you have a small, memory-limited IoT device collecting some sort of sensor data, and uploading that data using HTTP. A request might look like:

In that request, only a few parts are actually variable (between devices and/or times):

  • the device identifier in the request line
  • the timestamp and sensor reading value in the body JSON
  • possibly the authentication token

Note that I’m deliberately overlooking the possibility of a varying content-length, because it doesn’t add anything of value to the discussion but would make the examples less readable. These examples are not for real world use, only for demonstrating concepts. With that said, back to the topic at hand.

If the http I/O library only presents a send function which takes a single buffer, you have to put the full request into RAM. On the other hand, if it presents an iovec style send function, only the variable parts have to reside in RAM — the rest can be kept in flash (assuming memory-mapped flash, of course).

To illustrate with some code, if we have:

then to send that request we have to do something like:

As we have to have everything available in contiguous memory in order to call http_send_request(), there’s no avoiding allocating a memory block of significant size. Depending on circumstances, it might be possible to statically allocate it, but the memory demand is still there.

Compare that to if there’s an iovec interface available:

in which case we could keep most things in flash:

Here you can see the difference. Even with this trivial example, we went from needing around ~300 bytes in RAM to ~70 (assuming each iovec entry is 8 bytes), effectively relieving 75% of the memory pressure. For real world scenarios the gain can be even bigger, especially where the body payload contains a lot of boilerplate formatting. More importantly, it can be difference between being able to compose the request or not when you are working on a severely RAM constrained device, where your total memory is measured in kilobytes. Having to spend big chunks of precious RAM on temporary buffers just because the underlying communications library wasn’t sufficiently well designed is at best a downer, and at worst a show stopper.

My exhortation is thus: Don’t be the person who lets your company ship non-iovec style I/O libraries for small embedded systems! Pretty please with a cherry on top :).

We specialise in using emerging tech to solve difficult problems, get new ideas to market & disrupt business models.