A feminist would rightly observe that the phrase "took my wife" objectifies the woman in question, turns her into an object rather than a fellow human being with whom we have a particular relationship. When the relationship is severed, the woman in question is no longer a "wife," so when a woman severs her relationship with her husband to be with another man, she is no longer a wife for another man to take.
Feminists complain that such phrases as "take her as your wife" or "took my wife" or "stole my wife" are indications of patriarchy, but in fact, as we see with the phrase "took my job," this kind of objectification of relationships is not uncommon. Since a job is a relationship similar to that of a wife or husband, boyfriend or girlfriend, etc., any objections to the objectification of women in their relationships to men should apply equally to other kinds of relationships, including jobs.
So the feminists are probably not right that these phrases are an indication of patriarchy, but they are certainly not wrong in the implied critique of objectifying relationships.
In other words, Patrick Peterson provided us with a feminist critique of the phrase "take your job" that, quite frankly, could use considerably more unpacking. Especially since if we understand that jobs are relationships, the idea of someone taking it away loses its bite and we have to reconsider what it means for a job to be "taken" or "lost." Are there new ways of phrasing these things such that they reflect the fact of their being relationships rather than objects? There certainly needs to be.