Back in Fall 2018, I took CS 534 (Computational Photography) at UW-Madison. The term project was open-ended, so naturally my team and I decided to see if we could get a neural network to generate video game sprites. You know, the little 32×32 pixel art icons you’d find in a dungeon crawler or fantasy RPG (swords, shields, slimes, floor tiles). The idea was straightforward: feed a GAN a bunch of CC-licensed sprite-sheets from OpenGameArt and see what comes out the other side. In practice, nothing about GANs is straightforward.
Please check out the project website (fair warning, it is rough around the edges and has not been touched since 2018. It was a course project, not a commercial endeavor. Built it, learned from it, moved on).
As for why a GAN project is on a blog about half-built robots: it is a half-built something and I built it, which covers most of the criteria. Also, neural networks keep finding their way into my robots (see Triclops), so it fits in better than it looks.
Summary
We built a data pipeline to extract, normalize, and package sprites from public sprite-sheets into a consistent 32×32 RGBA format. Then we trained two types of generative models: a standard DCGAN, and a hybrid that trained the GAN inside a pre-trained autoencoder’s compressed feature space. The autoencoder learns what a sprite “looks like” first; the generator then produces feature vectors in that space, and the frozen decoder turns them back into sprites. We trained a separate model for each sprite category (weapons, humanoids, environment tiles, and so on).
The results were not game-ready pixel art. They were recognizable, vaguely sprite-shaped blobs with coherent color palettes. For a semester project with limited compute and a few thousand training samples, that felt like a win.
Project Structure
The project is split across a few repos:
- GAN Models: Jupyter notebooks with the DCGAN and autoencoder-GAN implementations (Keras/TensorFlow)
- GAN MNIST: warmup experiments learning GANs on the MNIST digits dataset
- Sprite Data: data pipeline scripts for extraction, normalization, dimension sorting, and NumPy packaging
- Project Website: a Jekyll site with dataset links, results, and the write-up
Everything was trained on Google Colab.
Why Post This Now?
I’ve been cleaning out old project directories and figured this one deserved a reference post rather than just sitting in a folder. GANs have come a long way since 2018 (diffusion models have largely eaten this space), but there’s something satisfying about a project where you can follow raw sprite-sheets all the way to generated output in a handful of notebooks. It’s a good snapshot of where the tooling was, and the autoencoder-GAN comparison still makes a good teaching example.
Thanks for reading. Stay tuned and keep building.