If you're on the current version (6.1.x), go to the Language section of the Setup Manager, and just empty it out. It should be the second item from the top.
Of course it would be in the config file as this is where all the changes entered in the Setup Manager are saved. It's just safer to use the Setup Manager itself as there's no chance of buggering it up.