EVA ICS v4 is finally out. We have been working on it for years and are finally proud to introduce the new-generation automation platform to all our customers.
In this article I want to explain, why we marked RESTful as a deprecated API in 3.4.2 and completely removed it in v4.
Sometimes RESTful approach is good, sometimes it is just amazing. However, we have found that it really does not fit the industrial systems paradigm, historically based on RPC variations, from ancient Modbus (yep, that’s also a kind of RPC, no kidding) to nowadays HTTP.
Let us review a small part of EVA ICS architecture: there are units (can have states and execute actions), there are sensors (can have states) and there are lvars (same as sensors but not linked to any equipment or fieldbus PLC registers). How can we go with RESTful?
Actually, in the way we have got in EVA ICS v3:
- GET /r/unit/lamp1 – get a unit status
- PATCH /r/unit/lamp1 – forcibly set a unit status or some its parameters
- POST /r/action – execute a unit action (returns something like Location: /r/action/UUID)
- GET /r/action/UUID – get running action status
The above is fine, the above is truly RESTful. Additionally, V3 also has got special methods to create/destroy items via HTTP API, which can be mapped to PUT and DELETE.
But. As soon as API becomes more powerful, we face more and more RESTful limitations, advanced methods are hard or even impossible to be mapped in RESTful way.
In example, we want to get item states by a mask. This can be performed e.g. with MQTT-style masks, like
GET /r/unit/+/subgroup/#
and can work properly but not quite RESTful (the result is an array of states). Until a client wants to get multiple masks in bulk. EVA ICS v4 HMI service method “item.state” allows to specify unlimited number of masks and get states for all matching items with a single request. Can this be done with RESTful? No way.
The next topic is about state history. In EVA ICS we have got two history methods: “item.state_log” (returns historical states as-is) and “item.state_history” (the history can be processed as a time-series dataframe). What is the RESTful way of it?
GET /r/unit/lamp1@log – for state log
GET /r/unit/lamp1@history – for state history
Pretty clear and common for such type of API. But let us add some filters and grouping methods:
GET /r/unit/lamp1@log?t_start=2022-05-29:11:22:00&t_end=2022-05-29:11:28:00&limit=20&xopts=<OOPS_WE_CAN_NOT_PUT_A_MAP_HERE_UNLESS_UGLY_ENCODED>&database=db1
Pretty hard to call, hard to read and hard to process. If we have started working this way, sooner or later we would have also faced with 2048-characters limit of GET request. Furthermore, “item.state_history” has got even more parameters and allows to specify multiple item OIDs at once to get their history as a combined time-series data-frame, which can be easily used e.g. to display charts. With RESTful we could try getting history data-frames one-by-one and combine them on the client side. Which is definitely an awful idea, especially when the back-end has got methods to perform grouping with built-in database functions but we need to manually parse again the data-frames by ourselves.
Last but not least are bus calls. EVA ICS v4 IPC is based on BUS/RT, which makes the whole platform insanely scalable and extendable. V4 HTTP API allows system administrators to perform direct bus calls to EVA ICS core or any service, connected to the core broker on the local machine or in the local cluster.
What is a bus call, according to RESTful paradigm? It can create objects, destroy objects, return their states or modify properties. It can send a email or deploy dozens of new services. We never know.
How can be a bus call sent to IPC, Fieldbus or other low-level layer in a RESTful way? No way, again. The RESTful API ends at the moment when HTTP ends. While a RPC payload can be quickly re-packed in anything and sent via anything. Have you got a two-wire RS-485? No problem, keep calm and give me your data.
BUS/RT: MessagePack vs strict-structure-RPCs
At the end, let me explain why have we chosen MessagePack (and alternatively JSON for HTTP API for web-based clients) over gRPC, capnp and other cooler-faster-neater modern serialization protocols.
Well. Despite BUS/RT uses MessagePack for certain built-in broker RPC methods, it allows clients to exchange data in any form. The best data serialization is always no data serialization. If there is an object, which has got a fully strict structure (e.g. two 100-byte strings and two 64-bit integers), the best way is to send it as-is, in binary (C) representation and cast back at the receiver side. But this is an example from utopia, usually unsafe for two low-level-language-written parts and nearly impossible for others.
If there is a simple structure (like EVA ICS item state: 5-10 simple fields with almost no sub-fields), simple versatile serialization protocols, such as MessagePack or JSON could give 2-3 times advantage in comparison with Protobuf and other trendy stuff. And no matter, how many ones are serialized inside a coming list. Why? Just because they are simpler. And keep in mind: in any IoT system push/pull state exchange is always the primary kind of data exchange, all others are quite rare or even are not used at all.
If there are complicated structures, sooner or later MessagePack and JSON become slower, however unless it is necessary to serialize gargantuan binary trees, the simple data packers still do their job fine, with maybe about 5-10% worse speed. But despite of that, they still have their primary advantage: they stay simple, clear and can be de/serialized to/from objects, maps or whatever is preferred by a communication side.
In his famous book The Art of Unix Programming, Eric S. Raymond (well-known open source advocate and programmer) summarizes the Unix philosophy as KISS Principle: “Keep it Simple, Stupid!”. We have no desire to make stupid platforms. The “S” in EVA ICS stands for “Smart”.