No, issues arising from differences due to sexual dimorphism (not “typically associated with gender”) are not solved by weight or age categories. This discussion already happened in this thread. There's a lot of research on it, which you didn't check and won't be bothered to.
There is a pattern here that I can't unsee, in which even hypothetical issues affecting people who would be excluded from female category by a test saying they're male are instantly taken very, very seriously and deemed insurmountable, while any issues affecting female people are instantly and very, very easily dismissed, to the point of absurdity.
Of course it is unfair to allow ineligible people to play in your category! You know it for any other category. If a 20 year-old entered a U10 category, you wouldn't say: „well, he's better than you and that's a point of sport, isn't it?”
But when it's women, suddenly you don't see any issue.