I liken it to the same pleasure one gets from solving a crossword puzzle. There are often little clues. If I find that one font has come from a clone collection like Serials, then that will often give me a shortcut to another font ID in the same sample.
I also get a kick out of reversing the modifications applied in the original setup (or as Bobby H might describe it: reviving a murdered font). Often I can just do it visually, stretching, unslanting, unrotating or condensing the line of type until it looks right. Other techniques include Bezier pen drawing one character in a really poor resolution sample and then make an ID from just the one interpreted character.