International Journal of Image and Graphics, Volume 18, Issue 03, July 2018. Advanced digital capturing technologies have led to the explosive growth of images on the Web. To retrieve the desired image from a huge amount of images, textual query is handier to represent the user's interest than providing a visually similar image as a query. Semantic annotation of images' has been identified as an important step towards more efficient manipulation and retrieval of images. The aim of the semantic annotation of images is to annotate the existing images on the Web so that the images are more easily interpreted by searching programs. To annotate the images effectively, extensive image interpretation techniques have been developed to explore the semantic concept of images. But, due to the complexity and variety of backgrounds, effective image annotation is still a very challenging and open problem. Semantic annotation of Web contents manually is not feasible or scalable too, due to the huge amount and rate of emerging Web content. In this paper, we have surveyed the existing image annotation models and developed a hierarchical classification-based image annotation framework for image categorization, description and annotation. Empirical evaluation of the proposed framework with respect to its annotation accuracy shows high precision and recall compared with other annotation models with significant time and cost. An important feature of the proposed framework is that its specific annotation techniques, suitable for a particular image category, can be easily integrated and developed for other image categories.