Demo
Demo

Knowledge base

PMTA knowledge base›Mechanism§ 12.1.1

A file, a program or an endpoint — and a file under each of the three

critical

accounting destinations
There is a file under every method

Accounting records can be written to a file, passed to a program through a pipe or sent to an endpoint over HTTP. The third method is the webhook, and it works differently from how it seems.

A webhook is not a stream but batches on top of files. Records accumulate in a file at the path given in the block, the file goes out as a POST request and is deleted only after a successful send. Success is a 2xx response and nothing else; everything else triggers a retry after a short delay.

So an unavailable receiver causes accumulation, not loss. That is good news right up to the moment the lower free disk space threshold trips and the engine starts deleting old accounting files. That is what you need to design against: not loss on failure, but the disk.

The file underneath is common to all three methods, and it is also their common point of failure. On the other hand, all three have the same reliability: the data survives a restart because it sits on disk.

The pipe has a separate danger named by the vendor. If writing to the program fails, the node gives up trying until a configuration reload or a timeout, and the vendor advises keeping a regular file alongside so the data survives a broken pipe.

The main trap is treating the choice of method as “either/or.” All three can be configured at once, with different accounting blocks. This makes a migration safe: the files and the webhook work together, the results are compared, and only then is the redundant one turned off.
What the Hub does. It writes a webhook block — the node sends us the records itself, and the file underneath is intermediate. The direction matters: previously analytics would have been built by pulling files from the node; now the node sends, and it does not need to be polled for this.

What we pay for it: the receiver must be available. Ours responds quickly on purpose — it checks the token, puts the body in the cache, queues a job — because the engine waits for a 2xx and holds the connection.

We keep file accounting next to the webhook — as a separate <acct-file /var/log/pmta/acct.csv> block in the same template. That is exactly the insurance the vendor advises: until the receiver is proven, there is something to compare against.

Recommended values

The mechanism has no values of its own: the method is chosen by which lines are written in the block.

Method Enabled by What is underneath
file the path in the <acct-file> tag the file itself
endpoint http-webhook-url a file at the same path, deleted after sending
program a pipe in the block tag a file alongside — on the vendor’s advice

The path extension decides the format: .json is NDJSON, anything else is CSV.

The three methods, the intermediate files, the 2xx success condition, the retry on any other response and the pipe’s danger come from the manual (§ 12.1.1). The conclusion about the disk as the common point of failure and the advice to keep a file alongside while breaking in are ours.
Example configuration
# two methods at once — this is how a migration is done
<acct-file /var/log/pmta/acct.csv>
    records d, b, t
</acct-file>

<acct-file /var/log/pmta/webhook.json>
    records d, b, t
    http-webhook-url https://hub.example.com/api/telemetry/pmta/7</acct-file>
The first block writes a file, the second sends batches. While they work together, the results can be compared, and only then is the redundant one turned off.
yellow — — directive names, blue — blocks. They are highlighted the same way in the text of the card.
▲

Common mistakes and problems

what people get wrong and how it ends3 mistakes▶

Designing against record loss rather than against the disk

§ 12.1.1

What it costsWhen the receiver fails, records are not lost — they accumulate. The real limit here is the disk, and it approaches quietly and strikes by deleting old accounting files.
How to spot itThe receiver was down for a day, the data “was not lost,” but the node ran out of free space.
How to do it rightMonitor the accounting directory and free space; the alarm should fire on accumulation, not on missing data.

Treating the choice of method as mutually exclusive

§ 12.1.1

What it costsThe migration is done in one jump: files turned off, webhook turned on, and there is nothing left to compare the results with.
How to spot itAfter the migration the numbers diverge, and there is no second source.
How to do it rightTurn both on, compare, then turn off the redundant one.

Passing records to a program without a file alongside

§ 12.1.1

What it costsWhen writing to the program fails, the node gives up trying until a configuration reload or a timeout. The data for that period is saved nowhere.
How to spot itThe consuming program was restarted, and there are no records for that interval.
How to do it rightKeep a file accounting block alongside — the vendor’s direct advice.
Compatibility
PowerMTA 6.0
described in the manual of this version · platform: any
The vendor does not state the first version for every directive — there is no note here. The name also appears in the 5.0 edition of the manual.
Inheritance
The mechanism has no inheritance — the directives that enable it are inherited, and they live inside their <acct-file> blocks. There can be several blocks, and each chooses its method independently.
⇄

Related entries

3 relations

Same name in other blocks

Hand a message to a webhook instead of a receiverThe same contract of retrying until a 2xx response, except that there it hands off messages and here accounting records. inference

Works together with

Rotate accounting files nowManually rotating the accounting file is the same action as a batch going to the endpoint: the file is closed and frees up space. inference

Where you can see it

Accounting sent by the node instead of accounting collected from the nodeChapter 8 names the sending capability itself; its contract is described here. § 12.1.1

Referenced from

http-webhook-url<acct-file>The third way to hand off accounting records; the rules for intermediate files, retries and headers are described there. § 12.1.1

How to check on the node
The sign of a healthy webhook is that intermediate files appear and disappear rather than pile up:
ls -la /var/log/pmta/ | tail -20
A growing number of files means the receiver is not responding with 2xx. It is also worth checking free space: that, rather than record loss, is the real consequence of a long receiver outage.
◇

What else to read

3 articles

◇What the node tells about itself and through which channelsSeven observation surfaces: the web monitor, the console in four formats, the log, accounting, accounting search, metrics and export to external analytics.◇The log and accounting — two files and two different questionsThe engine keeps a separate log and a separate accounting file; the separation is made for the sake of format, performance and the ability to hand over the log without disclosing data.◇The watchdog that restarts the engineKeeps an eye on the engine and brings it back up after a crash by itself, sending a crash report to the postmaster address.

In the Hub
Cost of a mistake
reversible: harms only while it is set
Scenarios
Untangle a stuck queue · Collect telemetry
Priority
critical