Demo
Demo

Knowledge base

PMTA knowledge base›Mechanism§ 8.6

Accounting sent by the node instead of accounting collected from the node

HTTP accounting webhooks
The direction changes, and that changes a lot

Chapter 8 gives this two lines with a reference to the accounting chapter, and their importance is out of proportion to their length.

The direction changes. Analytics built on files requires collecting them from the node: waiting for a file to close, reading someone else’s file system, polling. Sending removes all of this at once — a node that sends records itself does not need to be polled for them at all.

The price is a receiver that must be available or must tolerate whatever retry behavior the node ends up having. Chapter 8 does not describe it; the description is in § 12.1.1, and the contract there is simple: retry until a 2xx response, accumulate on the node’s disk.

And it is an addition, not a replacement. Both methods work at the same time, so the transition can be made without a jolt: enable the webhook next to the files, reconcile, turn off what is not needed.

The main trap is reading these two lines as “now we can do without files.” Files remain: under the webhook there is intermediate storage at the same path. What changes is who initiates the transfer, not whether records get to disk.
What the Hub does. It uses it — and this is one of the decisions the whole fleet telemetry rests on. The node sends batches once a minute, and the Hub learns about deliveries, bounces and complaints from them, not by polling the node.

Polling remains for summaries. Once a day the “Accounting stats” module runs the pmtastats utility on the node over SSH — it counts over the CSV accounting files, and the webhook file is of no use to it.

This also implies a limitation: if the Hub has no public address, there is nowhere to send, and the telemetry block is not written at all. Then polling through the management interface remains — poorer in data, but it works.

Recommended values

The mechanism has no values of its own: it is enabled by one line, http-webhook-url, and everything else configures the details of sending.

What it decides With what
where to send http-webhook-url
how often http-webhook-max-interval
how to prove it is your node http-webhook-extra-headers or http-webhook-certificate
The capability itself and the fact that it adds to files rather than replacing them come from the manual (§ 8.6, § 12.1.1). The analysis of the gain and the price is ours.
Example configuration
# one line changes the direction of the accounting stream
<acct-file /var/log/pmta/webhook.json>
    records d, b, t
    http-webhook-url https://hub.example.com/api/telemetry/pmta/7    http-webhook-extra-headers /etc/pmta/mtahub-headers.txt
</acct-file>
The file-based accounting block can be kept: the methods are not mutually exclusive.
yellow — — directive names, blue — blocks. They are highlighted the same way in the text of the card.
▲

Common mistakes and problems

what people get wrong and how it ends2 mistakes▶

Expecting sending to get rid of files on the node

§ 12.1.1

What it costsThe intermediate storage remains, and it needs space. What changes is who initiates the transfer, not whether records get to disk.
How to spot itThe disk fills up with “file accounting turned off.”
How to do it rightPlan space the same way as for file accounting, with headroom for the time the receiver is unavailable.

Switching to sending in one jolt

our experience

What it costsBoth methods work at the same time, and dropping files before reconciling takes away the ability to check whether everything arrives.
How to spot itThe numbers diverge after the switch, and there is nothing to reconcile against.
How to do it rightKeep both accounting blocks until the receiver has been broken in.
Compatibility
PowerMTA 6.0
described in the manual of this version · platform: any
The vendor does not state the first version for every directive — there is no note here. The name also appears in the 5.0 edition of the manual.
Inheritance
The mechanism has no inheritance — the directives that enable it are inherited. All of them live inside <acct-file>, and each accounting block decides for itself.
⇄

Related entries

4 relations

Depends on

A file, a program or an endpoint — and a file under each of the threeChapter 8 names the capability itself; the accounting chapter describes its contract. § 12.1.1

Same name in other blocks

The telemetry contractThe same stream of records, only handed over the network rather than left as a file. § 8.6

Can be replaced by

Rotate accounting files nowWith sending, the file switch happens by itself — at the batch interval, and there is no need to call the command. inference

Applies to

http-webhook-url<acct-file>The only line that flips the accounting stream from collection to sending. § 8.6

Referenced from

min-free-space<signals>The same trouble as with the accounting webhook: an unavailable receiver causes accumulation, but the limit here is on disk. inferenceCSV or JSON — chosen by the file extension, not by a directiveThe path extension decides not only the record format but also the content type header in the request sent. § 12.1.1

How to check on the node
A check from the receiver’s side: whether at least one batch arrived and what is in it:
tail -f /var/log/nginx/access.log | grep telemetry
POST requests with code 200 mean the mechanism works. Their absence while the line is configured is investigated from the node side — http-webhook-log-verbose will show the reason.
◇

What else to read

3 articles

◇What the node tells about itself and through which channelsSeven observation surfaces: the web monitor, the console in four formats, the log, accounting, accounting search, metrics and export to external analytics.◇Two inbound report processors and what to do with themBounces by message and complaints are parsed by two identically structured blocks; each has its own address list and its own pair of decisions about the message’s fate.◇Five ways for a queue to hand off a messageA queue does not have to use SMTP: it can write to a file, run a program, call over HTTP or discard.

In the Hub
Cost of a mistake
reversible: harms only while it is set
Scenarios
Collect telemetry