Fruit size is one of the most important features for grading Harumanis fruits. However, harvesting the fruit at the correct size is problematic for fruits growers. The aim of this paper is to discuss the use of image processing technique to classify the grade of Harumanis fruits before harvesting. This research adopted a computer vision methodology which include image acquisition, image pre-processing, image segmentation, feature extraction and classification. The statistical analysis which used linear regression model showed that the size has high relationship with the Harumanis’ weights and grades. The results showed that it is possible to estimate the weight of fruits before harvesting using image processing technique thus enabling fruits grower to grade their products efficiently.