The slightly musky-flavored seed of the annatto tree is available whole or ground in East Indian, Spanish and Latin American markets. Buy whole seeds when they're a rusty red color; brown seeds are old and flavorless. Achiote seed is also called annatto which, in its paste and powder form, is used in the United States to color butter, margarine, cheese and smoked fish.