What made that call worse was the fact that no one in baseball would admit it was a bad call. You had them all close ranks and say stuff like "A flyball to the warning track could still be an infield fly." It was insulting. If Holbrook had just come out after the game and said "I kicked that call" and Torre had said "It was the wrong decision but it's a judgment call that can't be reversed" I'd feel better.
Also, Holbrook was a line judge. His only function out there was have another pair of eyes on whether a ball was fair or foul. He shouldn't have been making any other calls.