The administrations define the budget. Congress approves it. While you can argue that it's Congress ultimately to blame, it's the administrations that suggest these unwieldy budgets to begin with.

I mean, Congress could argue with the President for ages, but that can easily create a situation where the country's being run willy-nilly without a budget, and that's bad news. It's happened a few times.