I think your writing & character development is good and your initial instincts are right. I am in the same boat with my protagonist, Agate. I just workshopped the first chapter of my book and ran into the "reader isn't invested/sympathetic to the character yet" problem: (Redacted for sensitive/mature theme: Agate is suicidal and compulsively self-destructive.) In in my case, making my character more "likable" would remove not only his primary inner conflict (how to value and save his life), but also how it parallels the overarching storyline (which is about a species trying to avoid an ethical crisis that will bring them to extinction.) Sometimes, and this may be in your case too, the negative trait and the inner conflict are so entwined that the inner conflict won't make sense if that character had more virtue. If my Agate were more likable, suddenly I'm writing a completely different story about a completely different character.
For me, Agate's story HAS to open on his negative, (sadly, apparently reader-repellant) trait (his suicide attempt) since it is ensuring his main inner conflict. This lays groundwork for the tone of the story, how his distress will impact his relationships, and then his entire species: one flaw in Agate starts snowballing into the story-wide plot. Plus: The story arc is all about how Agate's kind has to save others and make personal sacrifices, so Agate's inability to save himself while he's saving others feels like a satisfying irony to me. I want to write a story where a suicidal character is responsible for lives while denying the responsibility of his own.
Personally, I think Agate is justified in having his negative personality (I won't go into why), but I can understand why readers (who when picking up the story have no idea what they're getting into) could be immediately repelled by such an arrogant, destructive character on page 1. But it is good to remember that different readers are more empathetic to certain tropes, so maybe cast your reader net wider. Your character being controlling may not be an issue for some readers. For example, as a reader, I love/tolerate narcissist characters. I'm not a narcissist and have relational issues with the narcissists in my real life (it's really more a personality disorder than a personality, just as being controlling or being self-destructive is.) BUT, in fiction, I feel empathy towards these characters because I DO think all people deserve empathy, and fiction is a safe distance to study a problematic character from. I would read your story, personally, with or without a controlling protag. But I also want to agree with your betas/friends that you want the character, if not to be positive altogether, then to have his redeemable traits presented before the "unforgivable" traits.
As an author, the most important thing is getting the reader to turn pages. For me, having readers like the protag is essential, however I also feel like I can't change Agate's dark side without "ruining" my story. But really, when I got to thinking about it, I realized Agate is actually a lot of good things: altruistic, self-sacrificing, intelligent, resilient, conscientious, etc. Sometimes, as authors, we forget to step back and ask ourselves... why do I like my character? or, what part of me do I see in my character? I am not the arrogant, destructive mess that Agate is, however, I relate to his fear of death and how he surprises himself by being an overcomer. I wasn't giving him enough credit, therein was the issue! I'm sure Vaughn has a redeemable quality that simply is not being presented as a redeemable quality. (Meaning, you're not making clear enough that there is a way in the story he can overcome/redeem his controlling). Hence your readers aren't drawing the lines between why he is flawed and why we should root for him anyways. I think you're right to ask for critique and workshop this further, but I disagree with any of your betas who try to argue that a controlling character is unwritable, period.
*Edit: Oh wait, you haven't written the book yet? Then don't worry about it, first drafts can be changed later :) Just write on instinct and get a draft/outline finished before you worry about tweaking the character
MasterShortpants Fantastic advice! I totally agree
Weiland is awesome at this, much better than I am. I just finished this book of hers, you may enjoy it!: https://www.amazon.com/Creating-Character-Arcs-Masterful-Development-ebook/dp/B01M6VC68U