Deleting ignored User/Channel/Category results in exceptions

If a User/Channel/Category that is being ignored gets deleted, it will continue to stay in the DB. This results in the corresponding remove commands and verify_perms throwing an exception due to the channel no longer existing. This then prevents the user from using verify_perms or even removing the offending ID from the DB. I think this is also going to effect log channels as well.
