Is there a way where I can restrict how big an image can be in a post?
Only byte size...pixels aren't measured but you can set thumbnail size.
I've seen it work on IPB before though? Is it possible for Mybb?
Add in any css sheet, and change the pixel values:
div.post_body img {max-width: 800px; max-height: 300px;}
It won't stop them from posting it, but it won't stretch your page or be annoyingly huge.
There's also an image resizer plugin available.
can FancyZoom work with it?