Reading Time: 24 minutes

Fisheye freshness detection using common deep learning algorithms
and machine learning methods with a developed mobile application

Abstract

Fish is commonly ingested as a source of protein and essential nutrients for humans. To fully beneft from the proteins and substances in fsh it is crucial to ensure its freshness. If fsh is stored for an extended period, its freshness deteriorates. Determining the freshness of fsh can be done by examining its eyes, smell, skin, and gills. In this study, artifcial intelligence techniques are employed to assess fsh freshness. The author’s objective is to evaluate the freshness of fsh by analyzing its eye characteristics. To achieve this, we have developed a combination of deep and machine learning models that accurately classify the freshness of fsh. Furthermore, an application that utilizes both deep learning and machine learning, to instantly detect the freshness of any given fsh sample was created. Two deep learning algorithms (SqueezeNet, and VGG19) were implemented to extract features from image data. Additionally, fve machine learning models to classify the freshness levels of fsh samples were applied. Machine learning models include (k-NN, RF, SVM, LR, and ANN). Based on the results, it can be inferred that employing the VGG19 model for feature selection in conjunction with an Artifcial Neural Network (ANN) for classifcation yields the most favorable success rate of 77.3% for the FFE dataset

Introduction

Fish consumption is motivated by its taste, health, quality, and freshness. The demand for high-quality and immunefsh products has increased over the past few years due to recent changes in consumer lifestyles [23, 53, 54]. Thereare several factors associated with fsh quality parameters that range from harvesting to consumption, such as safety,nutrition, availability, and freshness, which are afected by storage and processing methods [49]. A variety of factorsmay negatively afect fsh quality and freshness, including
production, transportation, sale, domestic storage, and fnal
preparation of food [11, 32]. Fish provides essential nutrients, vitamins, and proteins for human wellness [12, 37].
Fish freshness may be determined using a variety of techniques. Furthermore, with greater technical advancements,
there have been attempts to create a way of measuring and
assessing more dependable freshness. The criteria used to
determine freshness include sensory, physical, chemical, and
microbiological. Rapid protein liquid chromatography and
hyperspectral imaging techniques are also considered [37,
53, 54]. It is thought that a fsh’s eye region has a strong
relationship between its coloration and the period during
which they are stored. For fsh to maintain its highest quality
after harvest, it must be kept at a specifc temperature for a
specifc time [11].
A multi-nutrient food that provides proteins, omega-3
fatty acids, and vitamins (health benefts) [14]. The fresher
the fsh, the more nutritious it is. Consequently, it can be
challenging for most consumers to determine whether a fsh
is fresh while they are shopping. By touching and squeezing
the fsh’s body to determine its fexibility, you may quickly
determine how fresh it is [53, 54]. Normally fresh fsh has
higher elasticity. Unfortunately, using this method can contaminate food with germs, harming fsh and leading to foodborne illnesses [17, 38, 39].
Fish spoilage is the process through which the quality
of fsh deteriorates, altering its color, odor, smell, favor,
and fesh texture [4, 14]. There are two primary sources of
decomposition once it starts: biological spoilage and chemical spoiling [16]. Microorganisms invade the fsh’s body via
the gills and commence the process of decomposition. Due
to chemical interactions, chemical spoilage causes a disagreeable odor and also afects the favor [42].
The changeover from fresh to stale is indicated by a
change in the color of the gills from brilliant pink to dark
red or yellowish red. A faded color will be detected if the
discoloration of the skin is overall. It is easy to tell a fresh
fsh by the shine and brightness of its skin, as opposed to a
not-so-fresh fsh by its dullness and faded colors. Depending
on the color of the fesh of the fsh, freshness can be determined by colored fesh ranging from cream to yellow-orange to brown to blue-orange [45, 53, 54]. A fsheye’s appearance was also considered [46]. When a fsh gets spoiled, the areas around its pupils become opaque, which creates the illusion that it is spoiled [48]. The evaluation of fsh freshness and quality is the study’s key objective. This is achieved by utilizing deep learning models for extracting features. Training and testing of freshness are performed with images by cross-validation. The CNN classifcation algorithms InceptionV3, SqueezeNet, VGG16, and VGG19 were all utilized. For each algorithm separately machine learning will be applied for predicting the freshness of the image dataset. The algorithms (k-NN, SVM, ANN, LR, and RF) were selected to be examined on the dataset and discuss the obtained results. This study utilized a combination of deep learning techniques and machine learning methods to classify the freshness of fsh. The most successful classifcation algorithm will be determined. After this stage, a mobile application will be developed based on the most successful models. By using pre-trained deep learning models like SqueezeNet for feature extraction, you can often achieve better results than handcrafted feature extraction techniques. This is because deep learning models have learned to extract relevant features automatically from large amounts of data, which can be difcult to do manually

related works

In this section previously implemented methods for fsh freshness classifcation were reviewed. The reviewed areas were models in deep learning, fsh freshness, machine learning algorithms, and prediction with a mobile application. The corresponding articles are listed below

Deep learning models for fsh freshness
classifcation

In their study, Mohammadi Lalabadi et al. proposed the use
of color degradation in the various color areas of fsheyes
and gills as a basis for the segmentation of fsheyes and gills.
This study utilized digital images to analyze the color characteristics of fsheyes and gills during a 10-day ice storage
period. The left and right eyes and gills were scanned to
determine their condition, with fsh being considered fresh
only if both eyes and gills were in good condition. Artifcial neural networks (ANNs) and support vector machines
(SVMs) were employed for feature extraction from the color
spaces to classify the ice-storage periods. The accuracy of
the ANN for extracting eye features was 84%, while the
accuracy for extracting gill features was 96% [32].
Prasetyo et al., presented a DSC Bottleneck Multiplier
in their frst contribution for detecting freshness features of fsheyes. Using Residual Transition, as suggested in the second contribution, it is possible to bridge current feature maps by using Residual Transition, but by using Residual Transition it is possible to skip connecting the current feature maps to the next convolution block. Thirdly, MobileNetV1 Bottleneck with Expansion (MB-BE) was proposed as a method of classifying fsheyes based on their freshness. At last, a 63.21% accuracy percentage was obtained [38, 39]. Fang et al., developed a method for monitoring fsh freshness using color-coded labels that utilize red cabbage anthocyanin and a back propagation neural network. The labels change color based on the fsh’s pH level, indicating its freshness. A BP neural network trained with red cabbage anthocyanin labels was able to predict fsh freshness with an accuracy of 92.6%. This technology was integrated into a smartphone app, providing an efcient system for identifying fsh freshness in real time and improving food quality control throughout the supply chain [13]. Taheri-Garavand et al., proposed a new and reliable fsh freshness detecting method, and to identify fresh carps, a deep convolutional neural network (CNN) was used. The VGG-16 model’s multiple convolutional layers were utilized to extract various flters from the 672 images, which were used for four classes. 98.21% classifcation accuracy was demonstrated by the outcomes [46]. Additional deep learning techniques, such as the use of an electronic nose and chemometric techniques [29], Electronic noses (E-noses), tongues (E-tongues), and colorimeters paired with several machine learning techniques, and a data fusion approach (artifcial neural network, support vector regression, Random Forests regression, extreme gradient boosting) [28] used to determine the freshness of fsh. Even though they depend on semiconductor gas sensors, and fsh quality detectors [42]. An approach to determine freshness based on deep learning was executed by DF Anas et al., for three marine fsh species, utilizing a small version of the YOLOv2 algorithm called tiny YOLOv2. The assessment of fsh freshness levels was improved using this algorithm, and the levels were classifed as good, medium, or poor quality. The experiment achieved an average precision of 72.9%, an average recall of 57.5%, and an average accuracy of 57.5% [8].

Machine learning algorithms for fsh freshness
classifcation

Yudhana et al., used k-Nearest Neighbor (k-NN) and Naïve Bayes (NB) classification techniques based on fish-eye images to determine fish freshness. The categorization process employed RGB and GLCM features. The study involved fsh collection, image acquisition, classifcation, ROI identifcation, preprocessing, feature extraction, dataset separation, and classifcation. Results showed that the k-NN technique outperformed the NB method, with an average accuracy, precision, recall, specifcity, and AUC of 0.97, 0.97, 0.97, and 0.97 [55]. Banwari et al. used 75 fsh samples in total to create a framework and demonstrate the relationship between freshness and retrieved characteristics using the training data. A framework is created and constructed using the training data to use the relationship between the amount of freshness and the extracted characteristics. An accuracy of 96.67% is achieved in identifying the proper amount of freshness [5]. To identify seafood freshness, Jarmin et al., proposed a study to compare RGB color parameters with fsh freshness meters. Three diferent species of fsh were tested with the Torrymeter sensor in this study, which measures the RGB color of their eyes and gills. Four classes were obtained (Very fresh, Fresh, Average, and Spoiled) from 90 samples of fresh fsh [22]. Huang et al. utilized two diferent methods, computer vision and near-infrared spectroscopy (NIR spectroscopy), for detection. An integrated approach combining both methods was used, and it was found that computer vision technology performed better than NIR spectroscopy for the BP-ANN model. The study demonstrated that computer vision technology outperforms NIR spectroscopy in terms of accuracy when analyzing oil and gas samples. The BP-ANN model achieved a result of 94.17%, with a prediction rate of 90.0%, whereas the accuracy rate for the NIR spectroscopy model was 86.67%, with a prediction rate of 80.0% [19]. In another study which was proposed by Alaimahal et al., fsh freshness was detected by using several image processing flters such as the Gaussian fltering method and reshaping. Segmentation was then carried out utilizing the watershed transformation approach. Feature extraction comes next, and then classifcation is done using a K-NN classifer. The detection rate was recorded as 90% of correctness [3]. Based on retrieved image features, Nguyen et al. suggested new threshold-based with neural network-based fsh freshness classifcation approaches. These properties are determined by the physiological parameters of fsheyes in freshness and stale conditions, nine suggested models were developed from 49 fsheye photos taken from two primary groups of the four Crucian carp fsh. The accuracy results for both the training and testing sets were 100% [48].

Prediction with mobile applications

In this study, Iswari et al. developed a system for categorizing the freshness of fsh using fsh images was created.
The classifcation approach based on the summarization of
the colors in fsh images was K-Nearest Neighbour (kNN).
According to the KNN classifcation fndings, the three fsh
were classifed with an average accuracy of 91.36% [30] Navotas et al. created an Android application that can identify three of the most commonly consumed fsh in the Philippines, namely milkfsh, round scad, and tilapia. The application determines its freshness level using RGB values of the eyes and gills, which are rated from 1 (stale) to 5 (fresh). The software was developed using iterative learning with 30 fsh samples per species, resulting in 800 images of the eyes and gills of each fsh. The test demonstrated that the application correctly identifed Milkfsh, Round Skad, and Tilapia with accuracies of 90%, 93.33%, and 100%, respectively [34]. Previously proposed studies on detecting the freshness of fsh that are mentioned in the study were compiled into a tabular as shown in Table 1. In the table, as noted in the articles the number of images used, categorized into how many classes, the applied methods and the obtained classifcation accuracy ratio was presented

Materials and methods

In this study, deep learning (DL) and machine learning (ML)
algorithms were used to classify the freshness of fsh from
the Freshness of the Fish Eyes (FFE) dataset [38, 39]). Our
model starts with implementing the deep learning models frst, each model of SqueezeNet, and VGG-19 was processed
separately for feature extraction.
In the second step of the model, we apply fve diferent
machine learning algorithms for each deep learning model
mentioned previously to classify the fsh freshness again
separately. The workfow of the study is shown in Fig. 1.
In the next sections, details are presented. The fnal step is
the mobile application we have developed to predict the fsh freshness on time by taking a photo of a fsheye or asking to classify a previously stored image in the mobile gallery. The dataset was used for SqueezeNet frst to detect the fsh’s images, the images were categorized into three classes of freshness level. After that with the cross-validation, fve ML algorithms were tried to compare the prediction outcomes. This implementation was re-implemented again for VGG19.

Freshness of fsh eyes (FFE) dataset

For this research, a dataset comprising 4390 sRGB images was utilized, which were captured using a Xiaomi Redmi Note 4 smartphone [38, 39]) obtained from the Mendeley data website. The images were separated into three classes containing eight species of fsh. The level of freshness was (Highly fresh, Fresh, and Not fresh). One thousand seven hundred sixty-four images were highly fresh fsheyes, 1320 of them were fresh fsheyes, and 1306 images of non-fresh fsheyes [38, 39]. The dataset underwent a modifcation, wherein the mirroring technique was applied prior to any feature extraction and classifcation, resulting in a total of 8780 images in the dataset. We augmented the dataset because the classifers were experiencing misclassifcations between the highly fresh and fresh classes. Table 2 displays the distribution of images in the FFE dataset after modifcation according to classes. In the highly fresh class, the fsh has a shelf life of 1 or 2 days, 3–4 days assigned as fresh, and 5–6 days classifed as not fresh. The staleness of the fsheye starts in 2–3 days after catching and transporting it to storage, and as the days pass, the staleness increases [1]. This can be detected in the eye color and the blurriness appearing in the fsheye. A gray color will cover the fsheye when it is totally staled.

Feature extraction with deep learning

CNNs are a subset of artifcial neural networks that are primarily used for image analysis. Each number represents one
pixel in the image, which is seen by the computer as a matrix
of numbers. Even when the image is sent as an input to the
network, the connection between the pixels (values) should
be preserved. To turn the input into the output, many mathematical procedures are stacked on top of each other layer
[10]. In CNN, the convolution process is utilized to identify
features in the input images. We decided to choose two models for the feature selection process. Namely SqueezeNet,
and VGG19. A brief explanation is given for each model.
SqueezeNet: is a 2016 computer vision network introduced by Iandola et al. [20, 25, 35] SqueezeNet is employed
for efcient feature extraction, providing a compact yet powerful architecture that efectively captures essential information from input data [7, 33]. In this extraction from ea image 1000 features were extracted and then prepared for the ML algorithms. Feature extraction with deep learning models, such as SqueezeNet, involves using pre-trained neural networks to extract relevant features from input data. SqueezeNet is a popular deep-learning model that is often used for image classifcation tasks. In this process, the SqueezeNet model is frst pre-trained on a large dataset such as ImageNet, which contains millions of images and their associated labels [7]. VGG19: The VGG19 technique for deep learning was developed by the Visual Geometry Group at the University of Oxford as an extension of VGG16. It consists of 16 convolutional layers and 3 fully connected layers, making a total of 41 layers [31, 35]. Using the VGG19 feature extraction technique, 4096 features were obtained from each image and then preprocessed for the machine learning algorithms. VGG19 is utilized for feature extraction, leveraging its deep and intricate architecture to extract rich and detailed features from input data. SqueezeNet and VGG19 were chosen as feature extractors in this study due to their respective strengths: SqueezeNet for its efciency and compact architecture, and VGG19 for its deep and intricate structure, both contributing to efective and comprehensive feature extraction from the dataset. The SqueezeNet model is implemented as a small and fast model [20] for image recognition, originally trained on the ImageNet dataset. The model achieves accuracy comparable to AlexNet but with signifcantly fewer parameters (50× fewer). The SqueezeNet model is re-implemented, and weights from the author’s pre-trained model [20] are utilized. The activations from the pre-softmax layer (fatten10) are employed as an embedding. The VGG19 model is implemented as a robust 19-layer image recognition model originally trained on the ImageNet dataset [43]. For feature extraction purposes, the activations from the penultimate layer, specifcally fc7, are employed as an embedding. This allows the model to capture and represent essential information from the input data [43].

Machine learning algorithms

in this study, fve diferent algorithms were chosen and
applied for the classifcation process. To predict the images
as freshness level. This section provides an explanation for
each algorithm and information regarding the techniques
utilized is elucidated.

k‑NN

The K-Nearest Neighbors Classifer (k-NN) categorizes a new sample in a feature area based on the training samples closest to its location. It assigns test data to point to the class with the highest frequency among its k nearest neighbors [2, 36]. The KNN algorithm is that it can work well with both linear and non-linear data, and it can be used for multiclass classifcation tasks. However, it can be sensitive to the choice of K and the distance metric used for calculating the distances between data points. For k-Nearest Neighbors (kNN), the algorithm is confgured with a parameter setting of ten neighbors, Euclidean distance as the metric, and uniform weighting

SVM

SVM is a robust classifer efective in both linear and nonlinear tasks, optimizing a hyperplane to maximize class margin for enhanced accuracy and generalization. The kernel trick facilitates non-linear problem-solving by transforming features into a higher-dimensional space, where support vectors play a crucial role in defning the decision boundary. The regularization parameter (C) balances margin width and misclassifcation tolerance [2, 36]. SVM parameters include an SVM Cost (C) of 1.00, Regression loss epsilon (ɛ) of 0.10, kernel set to RBF with g(auto), and optimization parameters specifying a numerical tolerance of 0.0001, along with an iteration limit of 1000

ANN

Artifcial Neural Networks (ANNs) are computer networks that are designed to replicate the structure and functionality of the human brain. In a supervised artifcial neural network, there are three layers: input layer, hidden layer, and output layer. These networks can be used for a broad range of problems that involve supervised learning techniques. They process information and collaborate to train and then make a prediction [51]. In the Artifcial Neural Network (ANN), there are 100 neurons in the hidden layers with ReLU activation, utilizing the Adam solver. The network incorporates regularization with an α value of 0.0001 and is confgured for a maximum of 500 iterations. The training process is set to be replicable (yes)

Logistic regression

In binary classifcation tasks (when the target is categorical), logistic regression is an additional strong supervised machine learning approach. An efective way to understand logistic regression is to think of it as a classifcation-specifc form of linear regression [18]. Logistic Regression utilizes the sigmoid function, which is characterized by an S-shaped curve that transforms any real-valued input to a probability value within the range of 0 and 1. The logistic function’s parameters, namely the weights, and biases, are acquired by employing the maximum likelihood estimation technique during the training stage. In Logistic Regression (LR), Lasso (L1) regularization with a strength parameter (C) set to 1 is employed

Random forests

In the feld of machine learning and statistical analysis, random forests are used for prediction. The building blocks of RF were formed from the tree-based model. There are several forms of random forests, but all trees in the forest depend on the values of an independently selected random vector that is spread uniformly over all trees [6]. Random Forests (RF) is confgured with 500 trees, considering 5 attributes at each split, and opting not to split subsets smaller than 5. For classifcation tasks, each tree “votes” for a class, and the class with the majority of votes is assigned. For regression tasks, the predictions of individual trees are averaged. Random Forests typically use decision trees as their base learners. Decision trees are simple, non-linear models that recursively split the data based on features to create a tree-like structure

Confusion matrix

Confusion matrixes are used to store information about the
actual and predicted classifcations made by machine learning classifcation models. A matrix usually contains the
data used to evaluate the performance of such a model. This
model was evaluated based on the following criteria: accuracy, precision, recall, and F1-score [53], from the obtained
confusion matrix metrics, were calculated. In Table 3 the
metrics used in this study are presented with their formula
[40, 44, 50].
Classifcation models are evaluated using various performance metrics, and accuracy is one of them. The accuracy is
determined by dividing the number of correct predictions by the total number of predictions made. In the case of binary
classifcation, the accuracy can also be measured by calculating true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) [24, 26], as illustrated
in Table 3.
To assess the performance of classification models,
accuracy alone may not be sufcient. Therefore, precision
and recall are two commonly used performance metrics,
which complement accuracy. These metrics are derived
from true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) that are summarized in
a confusion matrix [21]. Precision is a performance metric
that calculates the proportion of relevant instances in the
retrieved instances. Its purpose is to evaluate the model’s
accuracy when it makes a positive prediction [52]. A recall
is a performance metric that quantifes the proportion of relevant instances retrieved out of the total number of relevant
instances. Its purpose is to evaluate how well the model captures positive cases, measuring the model’s completeness.
Precision and recall are frequently employed in combination to balance the compromise between the quality and
quantity of predictions made by a model. A high precision
implies that the model is highly specifc, but it may overlook
some positive cases. Conversely, a high recall implies that
the model is highly sensitive, but it may include some false
positives. Depending on the problem and the cost of errors,
one may favor either higher precision or higher recall. A
common approach to combine precision and recall is to use
the F1-score, which represents the harmonic mean of both
metrics [15, 27].
Table 4 shows the confusion matrix according to the
dataset. True Highly Fresh (THF), True Fresh (TF), True Not-Fresh (TNF), False Highly fresh (FHF), False Fresh (FF), and False Not-Fresh (FNF) are the long form for the values mentioned in Table 4. THF–HF is correctly identified as Highly Fresh and it is Highly Fresh actually, TF–F is predicted as Fresh and it is Fresh actually, TNF–NF predicted as Not-Fresh and actually it is Not-Fresh. FHF–F refered to the misclassified classes that were Highly Fresh but predicted as Fresh. FHF–NF referee to the misclassified classes which were Highly Fresh but predicted as Not-Fresh. FF–HF is predicted as Highly Fresh but, it is Fresh actually, FNF–HF is predicted as Highly Fresh, and it is Not-Fresh actually. FNF–F referees to the misclassified classes which were Not-Fresh but predicted as Fresh

Cross validation

In cross-validation, the data are split into subsets (called folds) and the training and validation are rotated among these folds. Cross-validation is used for evaluating learning models. When training a model, splitting a dataset into two segments is a conventional practice, a training set and a testing set. A training model gets only examples from its training set, whereas testing instances are simulated by its testing set. Cross-validation and K-fold are interchangeable terms. A K-fold simply describes how many folds you want your dataset to be split into. Using Cross-validation, it is possible to estimate model performance using unseen data that is not used during training [41, 47]. In this study, the dataset is split into ten folds. In Fig. 2, you can see how cross-validation is performed

Experimental results

From the dataset used in this study, the confusion matrix
for each model is given independently. First, a data mining design was created, by importing the FFE dataset each
model of (SqueezeNet, and VGG-19) was implemented. Following the preprocessing phases and feature extraction, the
Training-Test data distribution step has been executed. In
this step, the 10-layer cross-validation approach is utilized.
At this level, classifcation operations have been completed. The classifers were listed as k-nearest neighbors, support vector machines, Random Forests, logistic regression, and Artifcial neural networks. VGG19 was implemented for extracting the features after the feature extraction the k-NN was examined to make the classifcation. According to Fig. 3, the VGG19 model classifed 2265 images as highly fresh fsh and 1263 misclassifed. With the k-NN machine learning method. While was classifed correctly for 1814 images as fresh fsh and 826 were misclassifed. Not fresh fsh were correctly classifed as 1449 images and 1163 were misclassifed. Figure 3 shows the outcomes obtained by utilizing the confusion matrix.
While in the SVM method, the highly fresh fsh were truly
classifed as 1869 with 1659 incorrectly classifed. The most
successful method can be assigned as ANN since the truly
classifed was 2758 highly fresh fsh class and 770 misclassifed. The model classifed 1795 images correctly as fresh
and 845 images by mistake. In addition, the not fresh class
was correctly classifed as 1852 images with 760 images by
mistake. The other results are also presented in Fig. 3 for the
other machine learning as LR, and RF SqueezeNet model was applied, and we got the result of
a total of 2469 images classifed correctly for highly fresh
fsh and 1059 classifeds incorrectly with the K-NN algorithm. Moreover, 1949 images were classifed as fresh and
they were fresh, on the other hand, 691 were incorrectly
classifed. For not fresh class 1531 were predicted correctly
according to the actual, besides that 1081 were classifed
incorrectly. Other tables in Fig. 4 show the outcomes for
each SVM, ANN, LR, and RF model
In the proposed study Orange data mining [9] was used
for the deep learning model designs. A computer with
Windows 11 Enterprise version 22H2 with Intel® core ™
i7-870H, a CPU @ 2.20 GHz, and 16 GB RAM was used,
the memory of the computer was 2TB HDD and 250 GB
SSD and with NVIDIA GeForce GTX 1050 Ti graphic card.
The learning parameters that were used in this study are
shown in (Table 5). Orange data mining desktop software
was used for creating the classifcation model. For each ML
algorithm diferent parameters were used, all the values are
presented in the table. The machine learning parameters
utilized in the research were selected based on the default
parameters available in the Orange Data Mining tool.
As mentioned in Table 6, VGG19 applied, with the ANN
model achieved the highest success ratio compared to the
other model. The precision obtained in ANN was also higher
than others as well as with the recall and F1 score. The ANN
gave a higher classifcation ratio. While the results achieved
by other ML algorithms (k-NN, SVM, RF, ANN, and LR)
were lower. The results indicate that VGG19 performs optimally when applied to the FFE dataset using an ANN.
An overview of the results obtained when using deep
learning techniques to train the convolutional neural network
architectures VGG-19 (Table 6), and SqueezeNet (Table 7).
Additionally, to see which performance gives the highest
accurate percentage on the FFE dataset, the accuracy results
have been presented in Table 8 in percentage value.
The ability to automatically determine fsh freshness
using images has several real-world implications and applications. First, it can improve quality control throughout the
supply chain, which has signifcant economic benefts. In
addition to preventing the sale of spoiled fsh, suppliers can
reduce losses caused by customer complaints and improve
overall consumer satisfaction by quickly identifying fish freshness. Disposing of spoiled fsh can also reduce food waste and minimize environmental impacts. By preventing contaminated or spoiled fsh from reaching the consumer, this technology can improve food safety.

Mobile application

Mobile apps can serve a variety of purposes, we suggest that having an application for this purpose will help fsh consumers fnd fresh fsh and be sure about the healthiness of the fsh. Within the study context, a mobile application is developed to detect the freshness of fsh. This application is meant to detect the freshness of fsh on fsheye. If a consumer wants to verify the freshness of fsh at any time and from any place, he/she may use his/her smartphone and take a photo of the fsh, and the application will determine whether the level of freshness (highly fresh, fresh, or not fresh). This program incorporated artifcial intelligence to predict based on deep learning model algorithms and machine learning algorithms. The application is currently only available for Android devices. Here are the interfaces for the mobile program: In Fig. 5 the frst screen is shown while opening the application which is named as splash screen, this screen is followed by a screen after 3 s which is the main page of the application. For starting the fsh freshness prediction, the user should tap the “Is it fresh” button. Then it displays the prediction ways, either the user will take a photo of the fsh’s eye and see the prediction result or press the “Select Image” button to upload an image from the mobile gallery. After uploading the image, the result of the freshness will be shown to the user. We conducted thorough assessments of the mobile application’s functionality to guarantee its strong performance. The outcomes of these examinations are illustrated in Fig. 6

Conclusion

From the prepared study, a fsh freshness dataset was used,
and the dataset was classifed into three classes of fsheyes.
Through the achieved results it is obtained that the freshness of the fsh couldn’t be determined only by detecting
the fsheye with this dataset images. The results were not
particularly outstanding since the image’s resolution quality
was not very precise and because we were unable to locate
our dataset or other suitable images. A total of 8780 images
of 3 based classes were used in the proposed model. These
images were of three classes of freshness highly fresh, fresh,
and not fresh fsh. A mixed model of deep learning and
machine learning models was used to classify the freshness of the fsh. The machine learning models were operated as (k-NN, SVM, LR, RF, and ANN) for each algorithm of (VGG19, and SqueezeNet) and the highest outcomes in ANN were 77.3% and 72.9% for each deep learning method. Prasetyo et al. used the same FFE dataset and implemented MobileNetV1 combined with DSC-BE, RT, and MB-BE, and 63.21% accuracy were obtained [38, 39]. The comparison point of the study is using the same dataset could achieve a better success ratio while using diferent methods other than MobileNetV1, which was used for this dataset. From the outcomes, we can conclude that using the VGG19 model for feature selection and ANN for classifcation can give the highest success percentage of 77.3% for the FFE dataset. There are many proposed studies on the detection of fsh and classifcation of its freshness either using chemical or biological methods or through diverse kinds of sensors. Moreover, studies about detecting freshness through using deep learning algorithms were present. Acknowledgements We would like to thank the Scientifc Research Coordinatorship of Selcuk University for their support with the project titled “Data-Intensive and Computer Vision Research Laboratory Infrastructure Project” numbered 20301027. Authors contributions Muslume Beyza Yildiz: software development, validation, and reviewing the document. Elham Tahsin Yasin: contributed to the conceptualization, methodology, software development, validation, formal analysis, as well as writing and reviewing the manuscript. Murat Koklu: participated in conceptualization, methodology, validation, formal analysis, and reviewing the manuscript. Funding Open access funding provided by the Scientifc and Technological Research Council of Türkiye (TÜBİTAK). Data availability Contacting the corresponding authors Prasetyo et al. [38, 39] or accessing the study’s dataset through this link https://data. mendeley.com/datasets/xzyx7pbr3w

Declarations

Conflict of interest The authors declare that they have no known competing fnancial interests or personal relationships that couldhave appeared to infuence the work reported in this paper. Compliance with ethics requirements This article does not contain any studies with human or animal subjects. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

References

1. Abu Rayan M, Rahim A, Rahman MA, Abu Marjan M, Ehsan Ali
UAM (2021) Fish freshness classifcation using combined deep
learning model. In: 2021 International conference on automation,
control and mechatronics for industry 4.0 (ACMI), pp 1–5. https://
doi.org/10.1109/ACMI53878.2021.9528138
2. Acharya UR, Molinari F, Sree SV, Chattopadhyay S, Ng K-H, Suri
JS (2012) Automated diagnosis of epileptic EEG using entropies.
Biomed Signal Process Control 7(4):401–408
3. Alaimahal A, Shruthi S, Vijayalakshmi M, Vimala P (2017)
Detection of fsh freshness using image processing. Int J Eng Res
Technol (IJERT) 5(9). https://www.ijert.org/research/detectionof-fsh-freshness-using-image-processing-IJERTCONV5IS090
42.pdf
4. Alasalvar C, Shahidi F, Miyashita K, Wanasundara U (2010) Seafood quality, safety, and health applications: an overview. Handbook of seafood quality, safety and health applications. Wiley,
Hoboken, pp 1–10
5. Banwari A, Chandra Joshi R, Sengar N, Kishore Dutta M (2022)
Computer vision technique for freshness estimation from segmented eye of fsh image. Ecol Inf. https://doi.org/10.1016/j.ecoinf.2022.101602
6. Breiman L (2001) Random forests. Machine Learn 45:5–32.
https://doi.org/10.1023/A:1010933404324
7. Butuner R, Cinar I, Taspinar YS, Kursun R, Calp MH, Koklu M
(2023) Classifcation of deep image features of lentil varieties with
machine learning techniques. Eur Food Res Technol. https://doi.
org/10.1007/s00217-023-04214-z
8. Anas DF, Jaya I, Nurjanah (2021) Design and implementation of
fsh freshness detection algorithm using deep learning. IOP Conf
Ser Earth Environ Sci. https://doi.org/10.1088/1755-1315/944/1/
012007
9. Demsar J, Curk T, Erjavec A, Gorup C, Hocevar T, Milutinovic M,
Mozina M, Polajnar M, Toplak M, Staric A, Stajdohar M, Umek
L, Zagar L, Zbontar J, Zitnik M, Zupan B (2013) Orange: data
mining toolbox in Python. J Mach Learn Res 14:2349–2353
10. Dogan M, Taspinar YS, Cinar I, Kursun R, Ozkan IA, Koklu M
(2022) Dry bean cultivars classifcation using deep cnn features
and salp swarm algorithm based extreme learning machine. Comput Electron Agric 204:1–13. https://doi.org/10.1016/j.compag.
2022.107575
11. Dutta MK, Issac A, Minhas N, Sarkar B (2016) Image processing
based method to assess fsh quality and freshness. J Food Eng
177:50–58. https://doi.org/10.1016/j.jfoodeng.2015.12.018
12. Erasmus VN, Kadhila T, Thyberg K, Kamara EN, BaulethD’Almeida G (2021) Public perceptions and factors afecting
domestic marine fsh consumption in Namibia, southwestern
Africa. Region Stud Mar Sci. https://doi.org/10.1016/j.rsma.2021.
101921
13. Fang S, Guan Z, Su C, Zhang W, Zhu J, Zheng Y, Li H, Zhao
P, Liu X (2022) Accurate fsh-freshness prediction label based
on red cabbage anthocyanins. Food Control. https://doi.org/10.
1016/j.foodcont.2022.109018
14. Franceschelli L, Berardinelli A, Dabbou S, Ragni L, Tartagni M
(2021) Sensing technology for fsh freshness and safety: a review.
Sensors 21(4):1373. https://doi.org/10.3390/s21041373
15. Gencturk B, Arsoy S, Taspinar YS, Cinar I, Kursun R, Yasin ET,
Koklu M (2023) Detection of hazelnut varieties and development
of mobile application with CNN data fusion feature reductionbased models. Eur Food Res Technol. https://doi.org/10.1007/
s00217-023-04369-9
16. Ghaly AE, Dave D, Budge S, Brooks MS (2010) Fish spoilage
mechanisms and preservation techniques: review. Am J Appl Sci
7(7):859–877 17. Hashanuzzaman M, Bhowmik S, Rahman MS, Zakaria M, Voumik LC, Mamun AA (2020) Assessment of food safety knowledge, attitudes and practices of fsh farmers and restaurants food
handlers in Bangladesh. Heliyon 6(11):e05485. https://doi.org/
10.1016/j.heliyon.2020.e05485
18. Hoss B, Alireza H (2021) A step-by-step breakdown with data,
algorithms, codes, and applications. Machine learning guide for
oil and gas using Python. Elsevier, Amsterdam, pp 169–295.
https://doi.org/10.1016/C2019-0-03617-5
19. Huang X, Xu H, Wu L, Dai H, Yao L, Han F (2016) A data
fusion detection method for fish freshness based on computer vision and near-infrared spectroscopy. Anal Methods
8(16):2929–2935. https://doi.org/10.1039/C5AY03005F
20. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ,
Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with
50× fewer parameters and<0.5 MB model size. arXiv preprint.
https://arxiv.org/abs/1602.07360. https://doi.org/10.48550/
arXiv.1602.07360
21. Isik H, Tasdemir S, Taspinar YS, Kursun R, Cinar I, Yasar A,
Yasin ET, Koklu M (2023) Maize seeds forecasting with hybrid
directional and bi-directional long short-term memory models.
Food Sci Nutr. https://doi.org/10.1002/fsn3.3783
22. Jarmin R, Yoot Khuan L, Hashim H, Hidayah Abdul Rahman N
(2012) A comparison on fsh freshness determination method. In:
2012 International conference on system engineering and technology (ICSET). https://doi.org/10.1109/ICSEngT.2012.6339329
23. Jose JA, Kumar CS, Sureshkumar S (2022) Tuna classifcation
using super learner ensemble of region-based CNN-grouped
2D-LBP models. Inf Process Agric 9(1):68–79. https://doi.org/
10.1016/j.inpa.2021.01.001
24. Koklu M, Ozkan IA (2020) Multiclass classifcation of dry beans
using computer vision and machine learning techniques. Comput
Electron Agric 174:105507
25. Koonce B (2021) SqueezeNet. Convolutional neural networks
with swift for tensorfow: image recognition and dataset categorization. Apress, Berkeley, pp 73–85. https://doi.org/10.1007/
978-1-4842-6168-2_7
26. Kursun R, Yasin ET, Koklu M (2023) The efectiveness of deep
learning methods on groundnut disease detection. In: 11th International conference on advanced technologies (ICAT’23), Istanbul,
Turkiye
27. Kursun R, Yasin ET, Koklu M (2023) Machine learning-based
classifcation of infected date palm leaves caused by dubas insects:
a comparative analysis of feature extraction methods and classifcation algorithms. In: Innovations in intelligent systems and
applications conference (ASYU), Sivas, Turkiye
28. Li H, Wang Y, Zhang J, Li X, Wang J, Yi S, Zhu W, Xu Y, Li J
(2022) Prediction of the freshness of horse mackerel (Trachurus
japonicus) using E-nose, E-tongue, and colorimeter based on biochemical indexes analyzed during frozen storage of whole fsh.
Food Chem 402:1–10. https://doi.org/10.1016/j.foodchem.2022.
134325
29. Li P, Niu Z, Shao K, Wu Z (2021) Quantitative analysis of fsh
meal freshness using an electronic nose combined with chemometric methods. Measurement. https://doi.org/10.1016/j.measu
rement.2021.109484
30. Made Satvika Iswari N, Wella, Ranny (2017) Fish freshness classifcation method based on fsh image using k-nearest neighbor.
In: 4th International conference on new media studies (CONMEDIA), pp 87–91. https://doi.org/10.1109/CONMEDIA.2017.
8266036
31. Mateen M, Wen J, Song S, Song Z (2018) Fundus image classifcation using VGG-19 architecture with PCA and SVD. Symmetry.
https://doi.org/10.3390/sym11010001
32. Mohammadi Lalabadi H, Sadeghi M, Mireei SA (2020) Fish
freshness categorization from eyes and gills color features using multi-class artifcial neural network and support vector machines.
Aquacult Eng. https://doi.org/10.1016/j.aquaeng.2020.102076
33. Hamid N, Sumait BS, Bakri BI, Al-Qershi O (2021) Enhancing
visual quality of spatial image steganography using SqueezeNet
deep learning network. Multimedia Tools Appl. https://doi.org/
10.1007/s11042-021-11315-y
34. Navotas IC, Santos CNV, Balderrama EJM, Candido FEB, Villacanas AJE, Velasco JS (2018) Fish identifcation and freshness
classifcation through image processing using artifcial neural
network. J Eng Appl Sci 13(18):4912–4922
35. Ozyurt F (2019) Efcient deep feature selection for remote sensing image recognition with fused deep learning architectures.
J Supercomput 76(11):8413–8431. https://doi.org/10.1007/
s11227-019-03106-y
36. Peker M, Özkaraca O, Şaşar A (2018) Use of orange data mining toolbox for data analysis in clinical decision making. Expert
system techniques in biomedical science practice. IGI Global,
Hershey, pp 143–167. https://doi.org/10.4018/978-1-5225-5149-
2.ch007
37. Prabhakar PK, Vatsa S, Srivastav PP, Pathak SS (2020) A comprehensive review on freshness of fsh and assessment: analytical
methods and recent innovations. Food Res Int 133:109157. https://
doi.org/10.1016/j.foodres.2020.109157
38. Prasetyo E, Purbaningtyas R, Adityo RD, Suciati N, Fatichah C
(2022) Combining MobileNetV1 and depthwise separable convolution bottleneck with expansion for classifying the freshness of
fsh eyes. Inf Process Agric. https://doi.org/10.1016/j.inpa.2022.
01.002
39. Prasetyo E, Suciati N, Fatichah C (2022) Yolov4-tiny with wing
convolution layer for detecting fsh body part. Comput Electron
Agric. https://doi.org/10.1016/j.compag.2022.107023
40. Ropelewska E, Sabanci K, Slavova V, Genova S (2023) The classifcation of leek seeds based on fuorescence spectroscopic data
using machine learning. Eur Food Res Technol 249(12):3217–
3226. https://doi.org/10.3390/agriculture13020498
41. Sabanci K, Aslan MF, Slavova V, Genova S (2022) The use of
fuorescence spectroscopic data and machine-learning algorithms
to discriminate red onion cultivar and breeding line. Agriculture
12(10):1652. https://doi.org/10.1007/s00217-023-04392-w
42. Saeed R, Feng H, Wang X, Zhang X, Fu Z (2022) Fish quality
evaluation by sensor and machine learning: a mechanistic review.
Food Control. https://doi.org/10.1016/j.foodcont.2022.108902
43. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:
1409.1556. https://doi.org/10.48550/arXiv.1409.1556
44. Singh D, Yavuz Selim T, Kursun R, Cinar I, Koklu M, Ozkan IA,
Lee H-N (2022) Classifcation and analysis of pistachio species
with pre-trained deep learning models. Electronics 11(7):981.
https://doi.org/10.3390/electronics11070981
45. Suresh A, Vinayachandran A, Philip C, Velloor JG, Pratap A
(2021) Fresko pisces: fsh freshness identifcation using deep learning. In: Raj JS, Iliyasu AM, Bestak R, Baig ZA (eds) Innovative data communication technologies and application. Springer,
Singapore
46. Taheri-Garavand A, Nasiri A, Banan A, Zhang Y-D (2020) Smart
deep learning-based approach for non-destructive freshness diagnosis of common carp fsh. J Food Eng. https://doi.org/10.1016/j.
jfoodeng.2020.109930
47. Taspinar YS, Dogan M, Cinar I, Kursun R, Ozkan IA, Koklu
M (2022) Computer vision classifcation of dry beans (Phaseolus vulgaris L.) based on deep transfer learning techniques.
Eur Food Res Technol 248:2707–2725. https://doi.org/10.1007/
s00217-022-04080-1
48. Nguyen TTA, Le M, Vo HN, Nguyen DTH, Phuoc TTA, Pham TV
(2020) A proposed rapid image processing-based method for fsh
freshness determination on mobile application. In: International
conference on green technology and sustainable development, pp
60–71. https://yahootechpulse.easychair.org/publications/prepr
int_download/LLjF
49. Tomic M, Matulic D, Jelic M (2016) What determines fresh fsh
consumption in Croatia? Appetite 106:13–22. https://doi.org/10.
1016/j.appet.2015.12.019
50. Unal Y, Taspinar YS, Cinar I, Kursun R, Koklu M (2022) Application of pre-trained deep convolutional neural networks for cofee
beans species detection. Food Anal Methods 15(12):3232–3243.
https://doi.org/10.1007/s12161-022-02362-8
51. Park Y-S, Lek S (2016) Artifcial neural networks: multilayer perceptron for ecological modeling. Developments in environmental
modelling, vol 28. Elsevier, Amsterdam, pp 123–140. https://doi.
org/10.1016/B978-0-444-63623-2.00007-4
52. Yasin ET, Koklu M (2023) Classifcation of organic and recyclable waste based on feature extraction and machine learning
algorithms. In: International conference on intelligent systems
and new applications (ICISNA’23), Liverpool, United Kingdom.
https://scholar.google.com.tr/scholar?oi=bibs&cluster=14683
431963100619745&btnI=1&hl=tr
53. Yasin ET, Kursun R, Koklu M (2023) Deep learning-based classifcation of black gram plant leaf diseases: a comparative study.
In: 11th International conference on advanced technologies
(ICAT’23), Istanbul-Turkiye
54. Yasin ET, Ozkan IA, Koklu M (2023) Detection of fsh freshness using artifcial intelligence methods. Eur Food Res Technol.
https://doi.org/10.1007/s00217-023-04271-4
55. Yudhana A, Umar R, Saputra S (2022) Fish freshness identifcation using machine learning: performance comparison of k-NN
and Naïve Bayes classifer. J Comput Sci Eng 16(3):153–164.
https://doi.org/10.5626/JCSE.2022.16.3.153
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional afliations.