This paper discusses teachers’ perceptions about the process and challenges of course design in general, and ESP (English for Specific Purposes) courses in particular. The study involves the process of designing an ESP course for returning students from the work place in the continued education department at the Higher Colleges of Technology. the findings in this paper are based on a qualitative research that employed an interpretivist/ constructivist theory, in which qualitative data was gathered primarily through interviews with a view to clarifying teachers' perceptions and understanding the beliefs and practices behind them. The study attempted to answer mainly the following questions. How do teachers perceive the process of an ESP course design? And what are the challenges they might encounter while designing ESP courses in general? In the findings section, four major themes have emerged from the data analysis process; they consist of influence of 1- Culture, Ideology, and Politics 2- The controversy of Needs Analysis, 3- Challenges in producing adequate course materials, and 4- Theory versus practice. Finally, the study discusses the implications of the findings and presents some recommendations for further use.