Fansub encoders usually use a constant rate factor (constant quality) and not a constant bitrate. This causes file sizes to vary for the reasons explained by Leggy.
Streaming services (CrunchyRoll, Hulu, Daisuki, etc. — files ripped by HorribleSubs) on the other hand use a constant bitrate, leading to very uniform file sizes.
You could always use Hiryuu's version until FFF's is released. I did have some weird subtitle issues in it though (the subs were inconsistent until after the opening).
I hope you don't mind me asking, but I've noticed that you haven't uploaded a banner to your profile. Is it because you overlooked, or is it simply undesirable?