The chip getting hot, by the way, is the Texas Instruments AIC32I audio codec chip.

http://www.ti.com/lit/ds/slas479c/slas479c.pdf

When I connect up the board with just the 5v rail connected, the chip doesn't get hot, but also the BlueGiga doesn't work and I get no serial output from it.

Whereas if I connect up the board with the 3v rail connected, the BlueGiga works and I see serial output, but the AIC32I audio codec chip gets real hot and I'm afraid to leave it that way for any length of time.

This is with all I2s connections disconnected by the way. Just the power, ground, serial.