Ha! that totally works. Just copied over every library to /lib on the vfat partition and booted with that as rootfs. Still having some issues with fonts (everything is monospaced) but I will post a ZIP which conaints all files for a blank sdcard with only a fat partition.