Three places a message body can be, and what each costs verified
A body travels inside the job through the broker, or sits in an external store addressed by a key, or arrives as a file in a watched directory — and each choice moves the bottleneck somewhere else.
In detail
Inside the job is the default and the simplest: one system holds everything, and a message either exists in the broker or does not. The cost is memory and bandwidth — every megabyte of message is a megabyte the broker holds, replicates and pushes twice. In an external store, the job becomes small and the broker becomes cheap, at the price of a second system in the delivery path: fetch failures become a delivery state of their own, and a body that expires before its message has been delivered leaves a job that cannot be completed. The shipped TTL of one hour against a delivery deadline of a day is exactly that hazard, and anyone turning a store on should reconcile the two. In a spool directory the body starts as a file, which is the most compatible arrangement — any program can write a file — and the slowest, since every message costs disk work. For Hub the first arrangement is right and will be for a long while: our nodes send what we hand them, volumes are modest, and a second system in the delivery path is a second thing to monitor. The catalogue records the other two because the customers who need them arrive with them already in place.