inputs

An input is where a pipeline's messages come from. A pipeline may have several — they are merged into one stream, each pumped by its own task so a slow or timer-driven input can't be starved by a busy one — and one input failing is reported and survived; the pipeline stops only when the last one is gone.

Three fields are declared by no input kind and accepted by all of them, because they sit on the wrapper rather than on the kind:

• buffer gathers messages before the transforms see them, by count, by time, or by whichever comes first. It never emits an empty batch, and its window opens at the first message rather than at the clock — so what it promises is a latency bound, not a cadence. See buffering an input.
• envelope attaches what the input knows about a message to the message, in band, as ordinary JSON fields. Absent, messages are passed on byte for byte as they arrive. See message metadata.
• ack says when the input tells its broker a message is done with. Only inputs with a broker-side notion of the difference honour it; the rest refuse to build rather than quietly ignoring it. See acknowledging an input.

max_batch, on the inputs that have it, is a third thing again: it never waits. It takes one message and then drains whatever has already arrived, so a quiet topic yields batches of one however high the cap is and only a catch-up ever fills one. That is what makes it the cheapest fix there is for a consumer replaying a backlog.

dummy

Emits one generated message on a fixed interval — a heartbeat for testing a pipeline without a real source attached.

Every message carries a value and the current_time it was emitted at. What the value holds is the payload field's business: a number sampled from a sine wave, so a chart of it has a shape, or a random sentence, so a text transform has something to chew on.

field	type		description
duration	integer	required	seconds between messages
amplitude	number	optional	peak of the sine wave — it swings between -amplitude and +amplitude. Numeric payloads only; defaults to 1.
payload	number | text	optional	what each message's value holds: a number sampled from a sine wave, or a random sentence as text. Defaults to number.
period	number	optional	seconds for one full turn of the sine wave. Numeric payloads only; defaults to 60. Sampling is by wall clock rather than by message count, so the wave keeps its period whatever duration is.
ack	on_receipt | on_delivery	optional	when this input tells its broker a message is done with. Available on every input kind in the schema, but only honoured by ones with a broker-side notion of "received" vs "delivered" of their own (kafka, for now) — an input with nothing to acknowledge refuses to build rather than silently treating this as on_receipt. Defaults to on_receipt, which is what every input has always done. See "acknowledgement modes" in the guide.
buffer	static | tumbling | batch	optional	batch messages from this input before the transforms see them — by count (static), by time (tumbling) or by whichever comes first (batch). Never emits an empty batch. Available on every input kind. Not to be confused with the buffer transform.
envelope	merge | wrap	optional	attach metadata about where each message came from — the subject, topic, partition and so on listed under "metadata" below. Available on every input kind. Omit it and messages are passed on exactly as they arrive.

buffer — type: "static"

field	type		description
size	integer	required	how many messages to gather before the batch is handed on

buffer — type: "tumbling"

field	type		description
window_seconds	integer	required	how long to gather messages for, measured from the first one

buffer — type: "batch"

field	type		description
size	integer	required	how many messages end the batch immediately
window_seconds	integer	required	how long to wait for them, measured from the first message in the batch

envelope — type: "merge"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.

envelope — type: "wrap"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.
payload	string	optional	the field the original payload is written to. Defaults to value.

metadata — what this input attaches to a message when its envelope is set.

field	holds
pipeline	id of the pipeline that read the message
input	kind of input it was read by, e.g. nats
received_at	when kayak read it, RFC 3339. This is an arrival time and not an event time: it says when the message reached this pipeline, not when whatever it describes happened.

http

Accepts messages posted to this pipeline's own endpoint, POST /api/pipelines/{id}/messages — the pipeline is the receiving end of an http API rather than something that reaches out to a broker.

The endpoint is derived from the pipeline's id and appears as soon as the pipeline is running; nothing is configured about it here. The body is one JSON message or an array of them, and an array arrives as one batch. A pipeline can only have one of these — two would share an endpoint, and which of them a request went to would be a coin toss — so a second one fails to build.

field	type		description
auth	bearer | header	optional	what a post must present to be accepted. Absent — the default — means the endpoint takes anything that reaches it, which is what every pipeline with an http input has always done.
capacity	integer	optional	how many posted batches may queue up ahead of the pipeline before it starts refusing them with a 503. Defaults to 1024. The queue is what lets a burst through; refusing past it is deliberate, since the alternative is holding a request open until the pipeline catches up.
ack	on_receipt | on_delivery	optional	when this input tells its broker a message is done with. Available on every input kind in the schema, but only honoured by ones with a broker-side notion of "received" vs "delivered" of their own (kafka, for now) — an input with nothing to acknowledge refuses to build rather than silently treating this as on_receipt. Defaults to on_receipt, which is what every input has always done. See "acknowledgement modes" in the guide.
buffer	static | tumbling | batch	optional	batch messages from this input before the transforms see them — by count (static), by time (tumbling) or by whichever comes first (batch). Never emits an empty batch. Available on every input kind. Not to be confused with the buffer transform.
envelope	merge | wrap	optional	attach metadata about where each message came from — the subject, topic, partition and so on listed under "metadata" below. Available on every input kind. Omit it and messages are passed on exactly as they arrive.

auth — type: "bearer"

field	type		description
token	string	required	the token. A ${NAME} reference, so the config file holds the name and the secret store holds the value.

auth — type: "header"

field	type		description
name	string	required	the header's name, matched case-insensitively on the way in. On an http input it may not be one of the headers an envelope passes through, since that would write the credential into the messages.
value	string	required	the exact value that header must have. A ${NAME} reference, as above.

buffer — type: "static"

field	type		description
size	integer	required	how many messages to gather before the batch is handed on

buffer — type: "tumbling"

field	type		description
window_seconds	integer	required	how long to gather messages for, measured from the first one

buffer — type: "batch"

field	type		description
size	integer	required	how many messages end the batch immediately
window_seconds	integer	required	how long to wait for them, measured from the first message in the batch

envelope — type: "merge"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.

envelope — type: "wrap"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.
payload	string	optional	the field the original payload is written to. Defaults to value.

metadata — what this input attaches to a message when its envelope is set.

field	holds
pipeline	id of the pipeline that read the message
input	kind of input it was read by, e.g. nats
received_at	when kayak read it, RFC 3339. This is an arrival time and not an event time: it says when the message reached this pipeline, not when whatever it describes happened.
method	http method the messages were posted with
remote_addr	address the request came from, when the server can see one
headers	request headers, restricted to a fixed list — content-type, user-agent, x-request-id, x-correlation-id and traceparent. Everything else is dropped rather than passed on: a header carrying a credential (authorization, x-api-key) written into a file or an object store is a leak that outlives the request by years, and no allow-list-by-prefix is safe enough to offer instead.

kafka

Consumes JSON messages from a kafka topic, each emitted as a batch of one.

A payload that isn't JSON is skipped with a warning rather than taking the pipeline down, same as the nats input. The consumer connects on the first read and joins a consumer group, so kafka remembers where this pipeline got to between restarts.

field	type		description
connection	kafka connection	required	name of the kafka connection to consume from — see "connections" in the readme. The brokers are declared once, in the connections file, rather than repeated in every pipeline reading from the same cluster.
topic	string	required	the topic to consume from
group	string	required	consumer group id. Kafka tracks the read position per group, so two pipelines sharing a group split the topic between them, and two with different groups each get every message.
max_batch	integer	optional	most messages to put in one batch. Defaults to 1 — one message per batch, which is what this input has always done. Raising it only ever coalesces records that had already arrived: the input still returns as soon as it has one, so an idle topic is no slower than it was. It is worth raising when a consumer is catching up on a backlog, where one-message batches make the run loop, the transforms and every downstream pipeline do their per-batch work a hundred times over.
start_at	earliest | latest	optional	where to start when the group has no committed position yet: earliest replays the topic from the beginning, latest only sees new messages. Defaults to latest.
ack	on_receipt | on_delivery	optional	when this input tells its broker a message is done with. Available on every input kind in the schema, but only honoured by ones with a broker-side notion of "received" vs "delivered" of their own (kafka, for now) — an input with nothing to acknowledge refuses to build rather than silently treating this as on_receipt. Defaults to on_receipt, which is what every input has always done. See "acknowledgement modes" in the guide.
buffer	static | tumbling | batch	optional	batch messages from this input before the transforms see them — by count (static), by time (tumbling) or by whichever comes first (batch). Never emits an empty batch. Available on every input kind. Not to be confused with the buffer transform.
envelope	merge | wrap	optional	attach metadata about where each message came from — the subject, topic, partition and so on listed under "metadata" below. Available on every input kind. Omit it and messages are passed on exactly as they arrive.

buffer — type: "static"

field	type		description
size	integer	required	how many messages to gather before the batch is handed on

buffer — type: "tumbling"

field	type		description
window_seconds	integer	required	how long to gather messages for, measured from the first one

buffer — type: "batch"

field	type		description
size	integer	required	how many messages end the batch immediately
window_seconds	integer	required	how long to wait for them, measured from the first message in the batch

envelope — type: "merge"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.

envelope — type: "wrap"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.
payload	string	optional	the field the original payload is written to. Defaults to value.

metadata — what this input attaches to a message when its envelope is set.

field	holds
pipeline	id of the pipeline that read the message
input	kind of input it was read by, e.g. nats
received_at	when kayak read it, RFC 3339. This is an arrival time and not an event time: it says when the message reached this pipeline, not when whatever it describes happened.
connection	name of the connection it was consumed through
topic	topic the record came from
partition	partition within that topic
offset	the record's offset in the partition. Together with topic and partition this identifies the record exactly.
key	the record's key as a string, or null when it was sent without one
timestamp	the record's own timestamp, RFC 3339, when kafka reports one

nats

Subscribes to a nats subject. Each message is parsed as JSON and emitted as a batch of one; a payload that isn't JSON is skipped with a warning rather than taking the pipeline down. The connection is opened on the first read.

field	type		description
connection	nats connection	required	name of the nats connection to subscribe on — see "connections" in the readme. The server it points at is declared once, in the connections file, rather than repeated in every pipeline that uses it.
subject	string	required	the subject to subscribe to
max_batch	integer	optional	most messages to put in one batch. Defaults to 1 — one message per batch, which is what this input has always done. Raising it only ever coalesces messages that had already arrived: the input still returns as soon as it has one, so a quiet subject is no slower than it was.
ack	on_receipt | on_delivery	optional	when this input tells its broker a message is done with. Available on every input kind in the schema, but only honoured by ones with a broker-side notion of "received" vs "delivered" of their own (kafka, for now) — an input with nothing to acknowledge refuses to build rather than silently treating this as on_receipt. Defaults to on_receipt, which is what every input has always done. See "acknowledgement modes" in the guide.
buffer	static | tumbling | batch	optional	batch messages from this input before the transforms see them — by count (static), by time (tumbling) or by whichever comes first (batch). Never emits an empty batch. Available on every input kind. Not to be confused with the buffer transform.
envelope	merge | wrap	optional	attach metadata about where each message came from — the subject, topic, partition and so on listed under "metadata" below. Available on every input kind. Omit it and messages are passed on exactly as they arrive.

buffer — type: "static"

field	type		description
size	integer	required	how many messages to gather before the batch is handed on

buffer — type: "tumbling"

field	type		description
window_seconds	integer	required	how long to gather messages for, measured from the first one

buffer — type: "batch"

field	type		description
size	integer	required	how many messages end the batch immediately
window_seconds	integer	required	how long to wait for them, measured from the first message in the batch

envelope — type: "merge"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.

envelope — type: "wrap"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.
payload	string	optional	the field the original payload is written to. Defaults to value.

metadata — what this input attaches to a message when its envelope is set.

field	holds
pipeline	id of the pipeline that read the message
input	kind of input it was read by, e.g. nats
received_at	when kayak read it, RFC 3339. This is an arrival time and not an event time: it says when the message reached this pipeline, not when whatever it describes happened.
connection	name of the connection it was received on
subject	the subject this message was published to. This is the concrete subject rather than the pattern subscribed to, which is what makes a wildcard subscription usable: subscribe to *.temperature and the machine's name is here.
reply	the reply subject, when the publisher set one, else null
headers	nats headers, as an object of arrays

pipeline

Takes another pipeline's output as its input. This is what makes the pipelines a graph: several pipelines can read from the same upstream, and it fans out to all of them. The upstream must already exist when this pipeline is created, so declare it earlier in the config file.

field	type		description
upstream	pipeline id	required	id of the pipeline to read from
ack	on_receipt | on_delivery	optional	when this input tells its broker a message is done with. Available on every input kind in the schema, but only honoured by ones with a broker-side notion of "received" vs "delivered" of their own (kafka, for now) — an input with nothing to acknowledge refuses to build rather than silently treating this as on_receipt. Defaults to on_receipt, which is what every input has always done. See "acknowledgement modes" in the guide.
buffer	static | tumbling | batch	optional	batch messages from this input before the transforms see them — by count (static), by time (tumbling) or by whichever comes first (batch). Never emits an empty batch. Available on every input kind. Not to be confused with the buffer transform.
envelope	merge | wrap	optional	attach metadata about where each message came from — the subject, topic, partition and so on listed under "metadata" below. Available on every input kind. Omit it and messages are passed on exactly as they arrive.

buffer — type: "static"

field	type		description
size	integer	required	how many messages to gather before the batch is handed on

buffer — type: "tumbling"

field	type		description
window_seconds	integer	required	how long to gather messages for, measured from the first one

buffer — type: "batch"

field	type		description
size	integer	required	how many messages end the batch immediately
window_seconds	integer	required	how long to wait for them, measured from the first message in the batch

envelope — type: "merge"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.

envelope — type: "wrap"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.
payload	string	optional	the field the original payload is written to. Defaults to value.

metadata — what this input attaches to a message when its envelope is set.

field	holds
pipeline	id of the pipeline that read the message
input	kind of input it was read by, e.g. nats
received_at	when kayak read it, RFC 3339. This is an arrival time and not an event time: it says when the message reached this pipeline, not when whatever it describes happened.
upstream	id of the pipeline this batch came from. Note that metadata already attached upstream arrives with the message and is not replaced — being in band, it flows through the graph like any other field.

mqtt

Subscribes to an mqtt topic — or a topic filter, since mqtt's + and # wildcards are valid here. Each message is parsed as JSON and emitted as a batch of one; a payload that isn't JSON is skipped with a warning rather than taking the pipeline down, the same rule every other input follows.

The connection is opened on the first read, and a stable client id is derived from the pipeline's id and this topic — not configurable, since nothing about it is a choice this pipeline needs to make and getting it wrong (two inputs sharing one id) silently drops one of them.

field	type		description
connection	mqtt connection	required	name of the mqtt connection to subscribe on — see "connections" in the readme. The broker it points at is declared once, in the connections file, rather than repeated in every pipeline that uses it.
topic	string	required	the topic, or topic filter, to subscribe to
max_batch	integer	optional	most messages to put in one batch. Defaults to 1 — one message per batch, which is what this input has always done. Raising it only ever coalesces messages that had already arrived: the input still returns as soon as it has one, so a quiet topic is no slower than it was.
qos	at_most_once | at_least_once | exactly_once	optional	the quality of service to subscribe with. Defaults to at_most_once. ack: on_delivery needs at least at_least_once here — a QoS-0 subscription has nothing for it to acknowledge.
ack	on_receipt | on_delivery	optional	when this input tells its broker a message is done with. Available on every input kind in the schema, but only honoured by ones with a broker-side notion of "received" vs "delivered" of their own (kafka, for now) — an input with nothing to acknowledge refuses to build rather than silently treating this as on_receipt. Defaults to on_receipt, which is what every input has always done. See "acknowledgement modes" in the guide.
buffer	static | tumbling | batch	optional	batch messages from this input before the transforms see them — by count (static), by time (tumbling) or by whichever comes first (batch). Never emits an empty batch. Available on every input kind. Not to be confused with the buffer transform.
envelope	merge | wrap	optional	attach metadata about where each message came from — the subject, topic, partition and so on listed under "metadata" below. Available on every input kind. Omit it and messages are passed on exactly as they arrive.

buffer — type: "static"

field	type		description
size	integer	required	how many messages to gather before the batch is handed on

buffer — type: "tumbling"

field	type		description
window_seconds	integer	required	how long to gather messages for, measured from the first one

buffer — type: "batch"

field	type		description
size	integer	required	how many messages end the batch immediately
window_seconds	integer	required	how long to wait for them, measured from the first message in the batch

envelope — type: "merge"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.

envelope — type: "wrap"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.
payload	string	optional	the field the original payload is written to. Defaults to value.

metadata — what this input attaches to a message when its envelope is set.

field	holds
pipeline	id of the pipeline that read the message
input	kind of input it was read by, e.g. nats
received_at	when kayak read it, RFC 3339. This is an arrival time and not an event time: it says when the message reached this pipeline, not when whatever it describes happened.
connection	name of the connection it was received on
topic	the concrete topic this message arrived on — useful when the input subscribes to a filter containing + or # wildcards
qos	the quality of service it was delivered at
retain	whether the broker sent this as a topic's retained message rather than a live publish

redis

Subscribes to a redis channel. Each message is parsed as JSON and emitted as a batch of one; a payload that isn't JSON is skipped with a warning rather than taking the pipeline down. The connection is opened on the first read.

Plain SUBSCRIBE, not PSUBSCRIBE — a channel name is exact, the same choice the nats input makes for a subject with no wildcard. Redis pub/sub has no broker-side redelivery of any kind: an unsubscribed client simply misses whatever was published while it was gone, and there is nothing an ack could hold open — the same limitation NatsConfig has, for the same reason.

field	type		description
connection	redis connection	required	name of the redis connection to subscribe on — see "connections" in the readme. The server it points at is declared once, in the connections file, rather than repeated in every pipeline that uses it.
channel	string	required	the channel to subscribe to
max_batch	integer	optional	most messages to put in one batch. Defaults to 1 — one message per batch, which is what this input has always done. Raising it only ever coalesces messages that had already arrived: the input still returns as soon as it has one, so a quiet channel is no slower than it was.
ack	on_receipt | on_delivery	optional	when this input tells its broker a message is done with. Available on every input kind in the schema, but only honoured by ones with a broker-side notion of "received" vs "delivered" of their own (kafka, for now) — an input with nothing to acknowledge refuses to build rather than silently treating this as on_receipt. Defaults to on_receipt, which is what every input has always done. See "acknowledgement modes" in the guide.
buffer	static | tumbling | batch	optional	batch messages from this input before the transforms see them — by count (static), by time (tumbling) or by whichever comes first (batch). Never emits an empty batch. Available on every input kind. Not to be confused with the buffer transform.
envelope	merge | wrap	optional	attach metadata about where each message came from — the subject, topic, partition and so on listed under "metadata" below. Available on every input kind. Omit it and messages are passed on exactly as they arrive.

buffer — type: "static"

field	type		description
size	integer	required	how many messages to gather before the batch is handed on

buffer — type: "tumbling"

field	type		description
window_seconds	integer	required	how long to gather messages for, measured from the first one

buffer — type: "batch"

field	type		description
size	integer	required	how many messages end the batch immediately
window_seconds	integer	required	how long to wait for them, measured from the first message in the batch

envelope — type: "merge"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.

envelope — type: "wrap"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.
payload	string	optional	the field the original payload is written to. Defaults to value.

metadata — what this input attaches to a message when its envelope is set.

field	holds
pipeline	id of the pipeline that read the message
input	kind of input it was read by, e.g. nats
received_at	when kayak read it, RFC 3339. This is an arrival time and not an event time: it says when the message reached this pipeline, not when whatever it describes happened.
connection	name of the connection it was received on
channel	the channel this message was published to

opcua

Subscribes to variables on an OPC UA server, one message per value change.

The server pushes: this creates a subscription with a monitored item per node and is told when a value changes, rather than reading them round-robin on a timer. publish_interval_ms is how often the server may send, not how often it samples — a tag that doesn't move produces no messages at all.

Each message is one reading, and carries the tag as well as the value:

json { "node": "ns=2;s=Machine1.Temperature", "name": "temperature", "value": 21.5, "status": "Good", "source_timestamp": "2026-01-01T12:00:00.123Z", "server_timestamp": "2026-01-01T12:00:00.130Z" }

status is the reading's own quality and is always present — a sensor that has failed reports Bad... with a null value rather than going quiet, and a pipeline that acted on those as if they were readings would be acting on nothing. source_timestamp is when the device says the value was produced, which is the one to reduce or partition by; the envelope's received_at is when kayak read it, and on a slow link those are not the same instant.

The nodes are named by nodes, or found by browse, or both — one of them is required, since an input with nothing to monitor would sit silent forever. A node named twice is subscribed to once.

field	type		description
connection	opcua connection	required	name of the opcua connection to subscribe on — see "connections" in the readme. The server it points at is declared once, in the connections file, rather than repeated in every pipeline that uses it.
browse	object	optional	a node to browse, subscribing to every variable found under it.
deadband	number	optional	how far a value must move before the server reports it, in the value's own units. Absent reports every change, however small — which on an analogue signal is every sample, since the last digit is always moving. This is applied by the server, so it saves the network and this pipeline alike. It only applies to numeric nodes; a string or a boolean is reported on every change whatever this says.
max_batch	integer	optional	most messages to put in one batch. Defaults to 1 — one message per batch, which is what every other input does unless asked otherwise. Worth raising here more than elsewhere: one publish from the server carries every node that changed in the interval, so a subscription to two hundred tags at 1 Hz is two hundred batches a second through the run loop unless they are allowed to travel together. Raising it only ever coalesces changes that had already arrived.
nodes	list of opcua node	optional	the nodes to subscribe to, named one by one.
publish_interval_ms	integer	optional	how often the server may send a batch of changes, in milliseconds. Defaults to 1000. This bounds how long a change waits, not how often anything is measured.
queue_size	integer	optional	how many samples the server may hold for a node between publishes. Defaults to 1, which means a value that changes twice in one interval is reported once — the latest. Raise it, together with sampling_interval_ms, when every sample matters rather than the current value.
sampling_interval_ms	integer	optional	how often the server should look at each node, in milliseconds. Absent asks the server to sample at the publishing interval, which is what it does by default; a smaller value here is what fills a queue with intermediate readings between two publishes.
ack	on_receipt | on_delivery	optional	when this input tells its broker a message is done with. Available on every input kind in the schema, but only honoured by ones with a broker-side notion of "received" vs "delivered" of their own (kafka, for now) — an input with nothing to acknowledge refuses to build rather than silently treating this as on_receipt. Defaults to on_receipt, which is what every input has always done. See "acknowledgement modes" in the guide.
buffer	static | tumbling | batch	optional	batch messages from this input before the transforms see them — by count (static), by time (tumbling) or by whichever comes first (batch). Never emits an empty batch. Available on every input kind. Not to be confused with the buffer transform.
envelope	merge | wrap	optional	attach metadata about where each message came from — the subject, topic, partition and so on listed under "metadata" below. Available on every input kind. Omit it and messages are passed on exactly as they arrive.

browse

field	type		description
root	string	required	id of the node to browse under, in the same notation as node_id — typically a folder, e.g. ns=2;s=Machine1. Every variable found beneath it is subscribed to; folders and objects are followed, not subscribed.
depth	integer	optional	how many levels below the root to follow. Defaults to 3, and there is deliberately no spelling for "all of them": a browse of a plant server's whole address space is thousands of nodes, and the pipeline that asked for it would find that out by subscribing to them.

nodes — each entry

field	type		description
node_id	string	required	the node's id, in OPC UA's own notation — ns=2;s=Machine1.Temperature for a string identifier, ns=2;i=1042 for a numeric one, g= for a guid and b= for an opaque one. A node id with no ns= is in namespace 0, the server's own.
name	string	optional	what the messages from this node call it. Defaults to the node id itself, which is exact and unreadable; naming the tag here is what makes the rest of the pipeline — a group_by, a column mapping — legible.

buffer — type: "static"

field	type		description
size	integer	required	how many messages to gather before the batch is handed on

buffer — type: "tumbling"

field	type		description
window_seconds	integer	required	how long to gather messages for, measured from the first one

buffer — type: "batch"

field	type		description
size	integer	required	how many messages end the batch immediately
window_seconds	integer	required	how long to wait for them, measured from the first message in the batch

envelope — type: "merge"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.

envelope — type: "wrap"

field	type		description
meta	string	optional	the field the metadata object is written to. Defaults to _meta.
payload	string	optional	the field the original payload is written to. Defaults to value.

metadata — what this input attaches to a message when its envelope is set.

field	holds
pipeline	id of the pipeline that read the message
input	kind of input it was read by, e.g. nats
received_at	when kayak read it, RFC 3339. This is an arrival time and not an event time: it says when the message reached this pipeline, not when whatever it describes happened.
connection	name of the connection the session was opened through. Which node the reading came from is deliberately not here: it is on the message itself, as node and name, because a value without its tag is not a reading and metadata is opt-in.