As of v2.1.112, vesafb can't be modularised, although at some point in time, the developer of vesafb may decide to modify the sources for modularising. Note that even if modularising is possible, at boot time you will not be able to see any output on the display until vesafb is 'modprobed'. It's probably a lot wiser to leave it in the kernel, for these cases when there are booting problems.