Patel has publicly defended himself against allegations that he drinks on the job, which interferes with his duties.