In theory, I'm not against polygamy/polyandry. I don't view marriage as sacred.
The problem with polygamy in practice is that it's been a safe haven for the mistreatment of women. There are issues of abuse, rape, underage marriages, and the placement of women as subservient to men (of course...