Data pullers
Data pullers are external programs, used to pull state from equipment as fast as possible. Data pullers are usually used in high-load environments.
Configuration
Data pullers are created either with “eva uc datapuller” CLI command or with corresponding UC API methods.
Execution
All data pullers are started automatically at Universal Controller start. The data pullers can be stopped or restarted later using UC API functions or CLI (eva uc datapuller).
Universal Controller always tries to kill a data puller and its child subprocesses on stop / shutdown. However, to avoid critical situations, data puller SHOULD exit manually if STDOUT / STDERR are not available any longer (standard programs do this by default).
Data exchange
Basics
Data puller is a program, or a shell script, written in any language, which repeatedly pulls the equipment, collects and processes data and then sends it to stdout. Data pulling should be done in an infinite loop. Start / stop operations are handled by Universal Controller.
When stop (SIGTERM signal) is received, data puller has 1 second to shut down itself gracefully, otherwise it (and its child processes) will be forcibly killed.
If a data puller process exits or crashes, it will be automatically restarted. The automatic restart is performed with 1-second delay.
The idea of data pullers is developing small programs in low-level programming languages to let EVA ICS receive only modified data. Data pullers are good for Ethernet/IP and Modbus-slave equipment, as well as for SNMP-enabled equipment without a support of SNMP traps or any other equipment, which is unable to send any state events.
Health check
If there are no events, a data puller SHOULD send .ping or empty string to STDOUT, otherwise it will be considered as dead after the period equal to the data puller timeout (or the default controller timeout) and automatically restarted:
.ping
Additional parameter “event-timeout” asks UC to restart the data puller if there are no events from it.
Data output
Data puller should collect data and output update commands to STDOUT in format, equal to UC UDP API:
<item_id> u <status> <value>
# e.g.
sensor:tests/test1 u 1 777.555
If the status or the value need to be skipped, they must be set to “None”:
sensor:tests/test1 u None 777.555
Data pullers SHOULD:
collect and output all available data on startup
during the execution, send to Universal Controller only modified data
manually handle logic mapping, e.g. converting Modbus registers or Ethernet/IP tags to EVA ICS items states.
Logging
Data puller can send messages to Universal Controller logs. If a stdout line starts with .log, it is considered to be a log command:
.log <level> <message>
# e.g.
.log warning Some warning information
Where level is the level of the message (debug, info, warning, error or critical, first letter is enough, e.g. w for warning).
The data puller can also print messages to STDERR, they will be automatically logged with error level.
State reporting
A data puller can report custom state messages (e.g. “loading”, “working”, “having a problem”) with lines starting with “.state”:
.state <message>
# e.g.
.state almost ready
Environment variables
All cvars and other EVA ICS core variables are available in the datapuller process system environment.
Pulr
Pulr is a free data pulling tool, 100%-compatible with EVA ICS. Refer to Pulr documentation about the tool configuration, here are few remarks how to use it:
In Pulr configuration, set output: eva/datapuller to output data in EVA ICS native format.
Set beacon value to less than Universal Controller default timeout, otherwise the Pulr subprocesses will be killed and continuously restarted.
All fields in process sections must have set-id param, to convert pulled data IDs into EVA ICS item IDs (<unit|sensor>:<group>/<id>)
set-id must end either with .status or with .value to set the proper EVA ICS items state field. If no ending is specified, the item state value is updated.
Pulr supports SNMP (v2), Modbus (TCP/UDP) and Ethernet/IP (Allen-Bradley compatible) field buses.