Heirlooms

Heirlooms are four of the currencies in Darkest Dungeon. They consist of Crests, Portraits, Deeds and Busts, and are primarily used to upgrade buildings in town.

In dungeons, Crests stack to 12, Busts and Deeds stack to 6, and Portraits stack to 3.

Trade Heirlooms
In town, heirlooms can be exchanged for heirlooms of other types. Comet Shards from the Color of Madness DLC can also be exchanged:

* The heirloom being traded is on the left hand side; the heirloom being received is on the top. Note that some value is lost in each exchange.

Inventory Space
Or in terms of inventory stack size: * The heirloom on the left exchanges for the heirloom on the top, at the rate of one full stack for the given percentage of a stack.

When trading for Busts or Deeds, full stacks of Portraits are more space-efficient than the other two. When trading for Portraits or Crests, all the heirlooms are equally space-efficient.

Of course, not trading at all and avoiding the trading cost is even more efficient, if you can afford to delay a less critical upgrade.

Upgrade Buildings
Different buildings require different heirlooms for upgrade:

Abbey

Blacksmith

Guild

Nomad Wagon

Sanitarium

Stage Coach

Stage Coach (Experienced Recruits)

Survivalist

Tavern

Total Requirements
According to reddit user, u/VinayssusMan, the maximum amount of heirlooms a player will need to collect to upgrade every building on the highest difficulty is detailed below:

Stage Coach Requires: Abbey Requires: Tavern Requires: Blacksmith Requires: Guild Requires: Sanitarium Requires: Nomad Wagon Requires: Survivalist Requires: Total heirloom cost for town completion(No DLC): IF DLC ---
 * 144 Deeds
 * 54 Busts
 * 222 Crests
 * 240 Busts
 * 282 Crests
 * 123 Portraits
 * 240 Crests
 * 277 Deeds
 * 301 Crests
 * 118 Portraits
 * 302 Crests
 * 240 Busts
 * 154 Crests
 * 273 Crests
 * 268 Crests
 * 421 Deeds
 * 534 Busts
 * 241 Portraits
 * 2042 Crests

Districts Require: Total sum of heirlooms(with DLC):
 * 1925 Crests
 * 155 Portraits
 * 140 Busts
 * 310 Deeds
 * 12 Blueprints
 * 731 Deeds
 * 674 Busts
 * 396 Portraits
 * 3967 Crests
 * 12 Blueprints"