Is Ultra instinct stronger than Broly?
Again, Goku's Ultra Instinct is
a level beyond Broly's Legendary Super Saiyan form and the former's strength rivals that of some Gods of Destruction. It wouldn't be wrong to say that Goku's attacks will hit a lot harder than Broly's and that's a key advantage to have in a fight against a monstrous figure.
Why didn't Goku use ultra instinct against Broly?
Originally Answered: Why doesn't Goku use Ultra Instinct in the Broly movie? It's
because Goku couldn't use it at will at the time. Remember Goku just mastered it at the Tournament Of Power not long before the Broly movie so it would make sense that Goku wouldn't be able to use Ultra Instinct when ever he wanted to.
Why didn't Vegeta use SSBE against Broly?
So to finish off Broly before he could grow too powerful, Goku and Vegeta fused into Gogeta, making the power gap far too large for Broly to adapt to in time. So basically, he didn't use the form
because it wouldn't be powerful enough to finish Broly and would merely allow Broly to become more powerful.