The purpose of this study is to investigate the factors leading to speaking difficulties from both the EFL Saudi learners and their teachers' perspectives through answering these questions: 1- What are the factors that affect Saudi students’ speaking performance from their own perspectives? 2-What are the factors that affect that affect Saudi students’ speaking performance from their teachers ' perspectives? 3- What are the difficulties that Saudi students’ encounter when they learn how to speak English from their own perspectives? 4-What are the difficulties that Saudi students’ encounter when they learn how to speak English from their teachers' perspectives? The subjects of the study were three hundred and fifty female EFL students and 20 teachers of English. Three instruments were used to answer the research questions: two questionnaires and a classroom observation sheet. The findings of the research proved that the following factors affect Saudi EFL students' speaking performance: conceptual knowledge, listening ability, motivation to speak, teachers’ feedback during speaking activities ,confidence, anxiety, mother tongue, law participation time allow to speak and time allowed for preparation. The findings also revealed that students face the following speaking difficulties when they try to speak: fear of mistakes, fear of criticism, the disability to think of anything to say, speak very little, their mother tongue and students' feel shy to speak. Data from class observations show that students’ speaking performance is mainly affected by four factors: insufficient input, time for preparation, poor instructions and the unsatisfactory amount of practicing speaking.