Only if you also moved all of the interface between the encrypted audio and the audio output hardware into the encrypted module (including stopping the rest of the kernel from reading any memory buffers that the module uses). At which point you may as not well be using the Linux kernel anyway.