I could never get any of the Android apps for Ampache to work.

The android apps are kind of terrible. I use ampache-provider and just-player and it gets the job done. It's not a great solution though

Appears to be compatible with subsonic apps, too.

I might have a long time ago, but I didn't realize it was so fully-featured. Is there a guide for setting this up on Windows anywhere?

http://mpd.wikia.com/wiki/Windows_Compatibility might be your best bet. I don't really run windows boxes, so I don't know unfortunately.