Demo
Demo

Knowledge base

PMTA knowledge base›Block <acct-file>›Directive§ 3.3.11.22

http-webhook-transfer-encoding

<acct-file>

Whether to compress a batch before sending
in the configuration this is a linehttp-webhook-transfer-encoding no compressioninside the block <acct-file>
The cheapest traffic saving

The line enables compression of the content before sending. One value is supported — gzip, and by default there is no compression.

Accounting records compress very well. This is text with repeating field names: NDJSON repeats them in every line, CSV — in the header, but the values themselves are uniform. The gain on such material is usually several-fold.

The receiver must be able to decompress. The engine sets the header itself, but if the receiving side does not look at it, it gets an incomprehensible binary stream. A receiver that parses the body before replying will not answer 2xx — and the sending goes into a retry; a receiver that replies at once will accept the batch and lose it while parsing.

This is the first thing worth trying in a conversation about volume — before increasing the interval and before parallelization.

The main trap is enabling compression without checking the receiver. The failure looks not like a compression error but like a pile-up of intermediate files: the engine retries, the receiver refuses, and by default only a brief error entry gets into the log — details come from http-webhook-log-verbose.

It is worse if the receiver replies before parsing the body: it will say 2xx, there will be no retry, and the batches will be lost silently.

What the Hub does. It does not write it — batches go out uncompressed.

🔎 Worth a try, though. Every node of the fleet sends five record types once a minute — with the full default field set plus additions (record-fields d *,timeFirstAttempt,… and so on). This is the most compressible material there is, and compression would noticeably cut traffic from the nodes to the Hub.

🔴 But first — decompression in the receiver. There is none now, and Laravel does not decompress a body with Content-Encoding: gzip by itself. And there will be no retry: our receiver replies 2xx without looking into the body, background parsing will not find a single JSON line in the batch — and the records will be lost, leaving a trace only in the telemetry intake error counter.

Recommended values

One value, the only question is “enable or not”:

Value When
not set (default) the receiver cannot decompress
gzip it can — then always

Check the receiver, not the node: the engine compresses and labels by itself.

The only supported value and no compression by default come from the manual (§ 3.3.11.22). The estimate of how well accounting records compress and the order “compression first, everything else later” are ours.
Example configuration
# the cheapest saving: enabled with one line
<acct-file /var/log/pmta/webhook.json>
    http-webhook-url https://hub.example.com/api/telemetry/pmta/7
    http-webhook-transfer-encoding gzip</acct-file>
Before enabling it, make sure the receiver decompresses: otherwise the sending goes into an endless retry, and batches will pile up on the node.
yellow — — directive names, blue — blocks. They are highlighted the same way in the text of the card.
▲

Common mistakes and problems

what people get wrong and how it ends2 mistakes▶

Enabling compression without checking the receiver

our experience

What it costsThe receiver replies with something other than 2xx, the engine retries endlessly, intermediate files pile up. Without detailed logging the log shows only a brief error, and compression is not the first suspect.
How to spot itRight after the configuration edit data stops arriving, while the files on the node grow.
How to do it rightRemove the line, check the receiver separately and enable it afterwards.

Trying other compression methods

§ 3.3.11.22

What it costsOnly gzip is supported. The engine will not accept another value, and the configuration will fail the check.
How to spot itpmta test config complains about the directive’s value.
How to do it rightWrite gzip.
Compatibility
PowerMTA 6.0
described in the manual of this version · platform: any
The vendor does not state the first version for every directive — there is no note here. The name also appears in the 5.0 edition of the manual.
Reference data
Syntax
http-webhook-transfer-encoding <enumeration>
Values
gzip
Default
no compression § 3.3.11.22
Applying
reload, no restart needed
Queue
will not touch what is already queued
Where to write it
<acct-file>
How many times
once in the block
Everything here comes from the manual, with the section.
Inheritance
No inheritance: the line lives inside its own <acct-file> and applies to its requests. One accounting stream may go out compressed, another not.
⇄

Related directives

1 relation

Applies to

http-webhook-url<acct-file>Compresses the content of the very request the batch goes out with. § 3.3.11.22

How to check on the node
Check from the receiver’s side — whether it accepts a compressed body:
curl -s -o /dev/null -w '%{http_code}\n' -X POST -H 'Content-Encoding: gzip' -H 'Content-Type: application/x-ndjson' --data-binary @batch.json.gz URL
2xx proves decompression only for a receiver that parses the body before replying. A receiver that replies at once and parses in the background — like the Hub’s — will answer 2xx to an unreadable body too: for such a receiver, check the parsing result, not the code. Any reply other than 2xx means that after enabling, the sending goes into a retry.
◇

What else to read

3 articles

◇What the node tells about itself and through which channelsSeven observation surfaces: the web monitor, the console in four formats, the log, accounting, accounting search, metrics and export to external analytics.◇Two inbound report processors and what to do with themBounces by message and complaints are parsed by two identically structured blocks; each has its own address list and its own pair of decisions about the message’s fate.◇Five ways for a queue to hand off a messageA queue does not have to use SMTP: it can write to a file, run a program, call over HTTP or discard.

In the Hub
Cost of a mistake
reversible: harms only while it is set
Scenarios
Collect telemetry
Neighboring settings of the block <acct-file>:http-webhook-urlhttp-webhook-max-intervalhttp-webhook-extra-headershttp-webhook-max-connectionshttp-webhook-ca-fileAll 26 directives of the block →