The anomaly detection task plays an important role in quality control in many industrial or manufacturing processes. However, in many such processes, anomaly detection is done visually by human experts who have in-depth knowledge and vast experience on a product in order to perform well in the detection task. In this paper, we present an approach that (i) identifies anomalies in an image based on the sparse residuals (or errors) obtained during image reconstruction using sparse representation and (ii) learns the threshold to classify an image pixel based on its residual value. The intuitions for our proposed sparse approximation driven approach are, namely: (i) anomalies are infrequent and (ii) anomalies are unwanted portions of an image reconstruction. Empirical results on a real-world image dataset for an industrial surface defect detection task are used to demonstrate the feasibility of our proposed approach.