Fuzzy Encoding With Hybrid Pooling To Improve Visual Dictionary Construction In Food Recognition



Tremendous volume of food images spread over the social media services can be exploited for fascinating purposes. It can be realized via food recognition engine to analyse the food images content for healthcare benefits and food industry marketing. The main challenges in food recognition are the large variability of food appearance that often generates a highly diverse and ambiguous descriptions of local feature. Ironically, the ambiguous descriptions of local feature have triggered information loss in visual dictionary constructions from the hard assignment practices. The information loss has led to the condition where the visual word assignation has become dramatically uncertain and plausible. This research proposes a combination of soft assignment technique by using fuzzy encoding approach and maximum pooling technique to aggregate the features that in the end produced a highly discriminative and robust visual dictionary across various local features and machine learning classifiers. Food recognition has to deal with extremely large variability of foods due to the nature of appearance of food objects. The local features for describing the foods often generate a highly diverse and yet ambiguous descriptions. Hence, the current method based on hard assignment and Fisher vector approach to construct visual dictionary have unexpectedly cause errors from the uncertainty problem during visual word assignation. Thus in this paper, a soft assignment based on fuzzy encoding approach has been proposed to deal with the uncertainty and plausibility problem in food category recognition. The local features by using MSER detector with SURF descriptor was encoded by using fuzzy encoding approach. Support Vector Machine (SVM) with linear kernel was employed to evaluate the effect of fuzzy encoding approach namely fuzzy c-means (FCM) on UECFOOD-100 dataset. The results of the experiments have demonstrated a noteworthy classification performance of fuzzy encoding approach compared to the traditional approach based on hard assignment and Fisher vector technique. The effects of uncertainty and plausibility were minimized along with more discriminative and compact visual dictionary representation.


food recognition;machine learning



F. Kong, H. He, H. A. Raynor, and J. Tan, “DietCam: Multi-view regular shape food recognition with a camera phone,” Pervasive Mob. Comput., vol. 19, no. C, pp. 108–121, 2015.

Z. Jie, W. F. Lu, S. Sakhavi, Y. Wei, E. H. F. Tay, and S. Yan, “Object Proposal Generation with Fully Convolutional Networks,” IEEE Trans. Circuits Syst. Video Technol., vol. 8215, no. c, pp. 1–1, 2016.

WHO, “Obesity and Overweight,” 2018. [Online]. Available: http://www.who.int/en/news-room/fact-sheets/detail/obesity-and-overweight.

M. Anthimopoulos, J. Dehais, P. Diem, and S. Mougiakakou, “Segmentation and recognition of multi-food meal images for carbohydrate counting,” 13th IEEE Int. Conf. Bioinforma. Bioeng. IEEE BIBE 2013, pp. 1–4, 2013.

F. Zhu, M. Bosch, N. Khanna, C. J. Boushey, and E. J. Delp, “Multiple Hypotheses Image Segmentation and ClassificationWith Application to Dietary Assessment,” IEEE J. Biomed. Heal. Informatics, vol. 19, no. 1, pp. 377–388, 2015.

G. M. Farinella, D. Allegra, M. Moltisanti, F. Stanco, and S. Battiato, “Retrieval and classification of food images,” Comput. Biol. Med., vol. 77, pp. 23–39, 2016.

H. Pooja and P. S. A. Madival, “Food Recognition and Calorie Extraction using Bag-of- SURF and Spatial Pyramid Matching Methods,” Int. J. Comput. Sci. Mob. Comput., vol. 5, no. 5, pp. 387–393, 2016.

H. Chougrad, H. Zouaki, and O. Alheyane, “Soft assignment vs hard assignment coding for bag of visual words,” 2015 10th Int. Conf. Intell. Syst. Theor. Appl. SITA 2015, 2015.

H. Hassannejad, G. Matrella, P. Ciampolini, I. De Munari, M. Mordonini, and S. Cagnoni, “A new approach to image-based estimation of food volume,” Algorithms, vol. 10, no. 2, 2017.

N. Martinel, C. Piciarelli, C. Micheloni, and G. L. Foresti, “A Structured Committee for Food Recognition,” Proc. IEEE Int. Conf. Comput. Vis., vol. 2016–Febru, pp. 484–492, 2016.

P. Pouladzadeh, S. Shirmohammadi, A. Bakirov, A. Bulut, and A. Yassine, “Cloud-based SVM for food categorization,” Multimed. Tools Appl., pp. 5243–5260, 2015.

J. C. Van Gemert, C. J. Veenman, A. W. M. Smeulders, and J. M. Geusebroek, “Visual word ambiguity,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 7, pp. 1271–1283, 2010.

Q. Qiu, Q. Cao, and M. Adachi, “Filtering out background features from BoF representation by generating fuzzy signatures,” in International Conference on Fuzzy Theory and Its Applications (iFUZZY2014), 2014, pp. 14–18.

G. Csurka and F. Perronnin, “Fisher Vectors : Beyond Bag-of-Visual-Words Image Representations,” pp. 28–42, 2011.

Y. Kawano and K. Yanai, “FoodCam: A real-time food recognition system on a smartphone,” Multimed. Tools Appl., vol. 74, no. 14, pp. 5263–5287, 2015.

H. Wang and W. Deng, “Face Recognition via Compact Fisher Vector,” in Chinese Conference on Biometric Recognition, 2015, no. 10, pp. 68–77.

L. Xie, Q. Tian, and B. Zhang, “Simple Techniques Make Sense: Feature Pooling and Normalization for Image Classification,” IEEE Trans. Circuits Syst. Video Technol., vol. 26, no. 7, pp. 1251–1264, 2016.

V. Garg, S. Vempati, and C. V. Jawahar, “Bag of visual words: A soft clustering based exposition,” Proc. - 2011 3rd Natl. Conf. Comput. Vision, Pattern Recognition, Image Process. Graph. NCVPRIPG 2011, pp. 37–40, 2011.

L. Liu, L. Wang, and X. Liu, “In defense of soft-assignment coding,” Proc. IEEE Int. Conf. Comput. Vis., pp. 2486–2493, 2011.

N. I. W. Warfield, S. K, N. I. Weisenfeld, and S. K. Warfield, “Kernel Codebooks for Scene Categorization,” Eur. Conf. Comput. Vis., pp. 696–709, 2008.

D. Dell’Agnello, G. Carneiro, T.-J. Chin, G. Castellano, and A. M. Fanelli, “Fuzzy clustering based encoding for visual object classification,” Proc. 2013 IFSA World Congr. - NAFIPS Annu. Meet., pp. 1439–1444, 2013.

T. Ren, Z. Qiu, Y. Liu, T. Yu, and J. Bei, “Soft-assigned bag of features for object tracking,” Multimed. Syst., vol. 21, no. 2, pp. 189–205, 2014.

U. L. Altintakan and A. Yazici, “A novel fuzzy feature encoding approach for image classification,” 2016 IEEE Int. Conf. Fuzzy Syst., pp. 1134–1139, 2016.

Y. Huang, Z. Wu, L. Wang, and T. Tan, “Feature coding in image classification: A comprehensive study,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 3, pp. 493–506, 2014.

S. Ghosh and S. K. S. Dubey, “Comparative analysis of k-means and fuzzy c-means algorithms,” Int. J. Adv. Comput. Sci. Appl., vol. 4, no. 4, pp. 35–38, 2013.

U. L. Altintakan and A. Yazici, “An improved BOW approach using fuzzy feature encoding and visual-word weighting,” IEEE Int. Conf. Fuzzy Syst., vol. 2015–Novem, no. 114, 2015.

D. Thanh Nguyen, Z. Zong, P. O.Ogunbona, Y. Probst, and W. Li, “Food image classification using local appearance and global structural information,” Neurocomputing, vol. 140, pp. 242–251, 2014.

R. S. Abbirami, A. Abhinaya, P. Kavivarthini, and T. Rupika, “Large Scale Learning for Food Image Classification,” J. Recent Innov. Trends Comput. Commun., vol. 3, no. March, pp. 973–978, 2015.

K. Y. Yoshiyuki Kawano, “FoodCam: A real-time food recognition system on a smartphone,” Multimed. Tools Appl., vol. 74, no. 14, pp. 5263–5287, 2015.

M. Wazumi, X.-H. Han, and Y.-W. Chen, “Food recognition using Codebook-based model with sparse-coding,” Proc. 2013 IEEE/SICE Int. Symp. Syst. Integr., pp. 482–485, 2013.

L. a. Zadeh, “Fuzzy sets,” Inf. Control, vol. 8, no. 3, pp. 338–353, 1965.

J. C. Bezdek, R. Ehrlich, and W. Full, “FCM: The fuzzy c-means clustering algorithm,” Comput. Geosci., vol. 10, no. 2–3, pp. 191–203, 1984.

R. Krishnapuram and J. M. Keller, “A Possibilistic Approach to Clustering,” IEEE Trans. Fuzzy Syst., vol. 1, no. 2, pp. 98–110, 1993.

D. Park, “Image Data Classification Using Fuzzy c-Means Algorithm with Different Distance Measures,” in International Symposium on Neural Networks, 2013, pp. 489–496.

Y. Shinomiya and Y. Hoshino, “Bag of Features Based on Feature Distribution Using Fuzzy C-Means,” pp. 546–550.

Y. Matsuda, H. Hoashi, and K. Yanai, “Recognition of multiple-food images by detecting candidate regions,” in Proceedings - IEEE International Conference on Multimedia and Expo, 2012, pp. 25–30.

W. X. Liu, J. Hou, and H. R. Karimi, “Research on vocabulary sizes and codebook universality,” Abstr. Appl. Anal., vol. 2014, 2014.

E. Salahat and M. Qasaimeh, “Recent Advances in Features Extraction and Description Algorithms : A Comprehensive Survey,” in IEEE International Conference on Industrial Technology (ICIT), 2017.

M. H. Lee and I. K. Park, “Performance evaluation of local descriptors for maximally stable extremal regions,” J. Vis. Commun. Image Represent., vol. 47, pp. 62–72, 2017.

J. Zheng, Z. J. Wang, and C. Zhu, “Food Image Recognition via Superpixel Based Low-Level and Mid-Level Distance Coding for Smart Home Applications,” Sustainability, vol. 9, no. 5, 2017.

Y. Li, S. Wang, Q. Tian, and X. Ding, “Feature representation for statistical-learning-based object detection: A review,” Pattern Recognit., vol. 48, no. 11, pp. 3542–3559, 2015.

S. Jabeen, Z. Mehmood, T. Mahmood, T. Saba, A. Rehman, and M. T. Mahmood, “An effective content-based image retrieval technique for image visuals representation based on the bag-of-visual-words model,” PLoS One, pp. 1–24, 2018.

DOI: http://doi.org/10.11591/ijeecs.v21.i1.pp%25p
Total views : 3 times


  • There are currently no refbacks.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

shopify stats IJEECS visitor statistics