The last days (and weeks) I’ve been working on a Vulkan example implementing physically based rendering with image based lighting (you can find a nice article with lots of details over at Trent Reed’s blog) and just pushed it to my public github repository.
While working on this demo I realized that putting all the binary assets (models and esp. textures) in the repository might not have been a great idea. They quickly bloat the size of the repository and once a binary asset is there you won’t be able to completely remove it (to shrink the repo again) unless you actually rewrite the history. But with the repository being public with almost 400 forks and numerous pull requests this pretty much a no-go.
With physically based rendering relying on HDR materials, textures can get pretty big. While there are compressed texture formats for HDR (e.g. BC6UH) I want my examples to support a wide array of devices and compressed texture format support varies greatly. On mobile e.g. you rarely have block compression support, and for e.g. ASTC only LDR formats are currently supported in Vulkan. So the only viable option is to store these textures in an uncompressed format that’s supported on a wide range of devices.
This makes the smallest common denominator for storing HDR assets R16G16B16A16 (three component formats are often not supported) which will result in pretty big textures like the pisa cube used in the example with 1k by 1k faces and mip maps clocking in at 65 mega bytes.
So aside from working on the actual code I also did spent some time on evaluating different ways of storing binary files for the examples without negatively affecting the actual code repository.
A natural (first) choice was git-lfs which is now available to everyone on github. And while git-lfs may sound like the perfect tool for this, the basic (free) plan for it on github is pretty limited and it would require anyone to actually install git-lfs. I also did some testing with git-lfs on a private repository and cloning the repo after converting all assets to git-lfs would fail half of the time, so I ditched the idea.
In the end I want with the most basic way of just uploading an asset pack to my web space that needs to be downloaded for the newer examples to work. I also added a small python script to the repository that automates this tasks and also extracts the files where they need to be. While this option lacks all the nice features git (or any other cvs) offers and adds an additional layer of work (each time I add a new asset I have to re-upload the file) I still think this is the most convenient way that does not harm the repository (in terms of size and performance) and doesn’t take too much effort from my side. Luckily I also just upgraded to a faster connection with ten times the upload speed I had before.
I’ll also use this asset pack for non-HDR assets in the future to keep binary files out of the repo.
A short manual on how to get the new asset pack if you want to build and run the samples yourself can be found in the repository.