Try rereading what Scrimgeor says. When Hermione is arguing with him, acting all lawyerish, he brings up magical law enforcement. Combine that with the knowledge of what we know based on ministry visits and it's safe to assume that it's people who argue the law as opposed to uphold it.