Background: This study aimed to investigate the potential contribution of invalid skewed responses to invalid factor solution in the result from exploratory factor analysis. In the present study, “the invalid skewed response” is defined as when majority of the respondents consistently rate only at one side which will eventually change the real or valid pattern of overall responses. Methods: A validation approach was conducted using a secondary data from a questionnaire validation study of an eight Likert scale that has a very stable and strong factor solution. Eight sub samples were retrieved from the data to represent multiple sets of analyses with sample size based on rule of thumbs from 3:1 until 10:1. From each sub sample, proportion of dummy response for the extreme left (scale of 0), the middle scale (scale of 4), and the extreme right (scale of 7) were assigned randomly at 10%, 20%, and 30%, respectively. Results: The invalid consistent responses of a middle scale have very low impact toward the factor solution. The occurrence of the invalid skewed responses affected the factor solution. Majority of the factor solutions were still valid based on consistent responses with 10.0% only. However, the construct that was based on forcing into four-factor solution had helped to produce the valid factor solution though some resulted in cross-loadings. All Cronbach’s alphas and minimum corrected item to total correlation were relatively strong for all factor solutions although some of the solutions were invalid. Conclusions: The skewed responses have the potential to change the ideal factor solution. Therefore, necessary steps need to be taken to avoid invalid skewed responses, especially in self-administered survey. Therefore, the recommended sample size guideline for exploratory factor analysis with justifications is proposed.