In the 19th Century, those who purported to know believed that racehorses could get spooked very easily and thus not race well. They also believed that goats had mystical calming capabilities. It seemed completely logical, then, to put a goat in the stall with the horse prior to a race. If someone wanted to secure an advantage against another racer, he would sneak in and steal that racer's goat from the horse's stall.
From that practice, we get the term "get one's goat."