Traditionally, chaff referred to the discarded stems of a cereal crop when it was threshed for the grain. This chaff was very low in nutrients for animals, basically just being straw. For this reason the term "separating the wheat from the chaff" referred to finding taking worthwhile objects/things out of less worthy things.

Why is Chaff now fed to animals?
Nowadays, the chaff that is cut for horses is far more worthwhile than the chaff that was threshed from wheat.

The main reason is that the crop for Chaff is cut at a much earlier stage than crop for grain whilst it is still very green - up to three months earlier depending upon conditions. For this reason, the nutrients that will eventually be in the seed are still in the hay stem.