Introduction
Researchers have long known that smoke quitting protocols very often determine strict correlations with dietary habits’ changes of individuals involved in the smoking cessation process,1 which is frequently linked to weight gain, eating and mood disorders, especially in the initial period of abstinence.2,3 Conversely, counseled changes of dietary habits result crucial to make the smoke quitting protocols successful.4
To support a clinical smoke quitting protocol application, emerging technologies can be exploited, especially the ones that results pervasive in the daily life of the involved patients, i.e., personal mobile apps by means of which monitoring-related data,5,6 under an automatic or user-driven paradigm, can be inputted. Collected data can indeed represent a consistent database both for centralized automated tools, as well as for clinicians, that will make decisions and follow-ups based on the accrued data.
FoodRec7 is a food recognition mobile app which relies on state-of-art image recognition algorithms boosted by ultimate Artificial Intelligence techniques, aiming at providing a multidisciplinary approach to support people who are trying to quit smoking and health personnel who help them do it. Among all these cutting-edge technologies, food recognition from digital images is one of the most promising to automatically monitor food intake and dietary changes8–10 and to draw correlations in the smoking cessation process.6,7
The FoodRec app aims to favor automatic extraction of inferred information about users’ eating habits and emerging trends from advanced analysis of images of desired and/or eaten meals in any monitored health process, besides the smoking cessation one, which initially started the investigation that drove the design and prototyping of the technological solution detailed in this paper.
The investigation results presented hereafter refer to users’ appreciation of the proto-typed app, based both on standard statistical analysis of specific data required to the test App’ users as better detailed in the next section, and on interview specifically tailored to be filled out after the test period.
Material and Methods
This study was an uncontrolled pre-test post-test open pilot study in which participants tested a food recognition app (FoodRec), created by an interdisciplinary expert group, for two consecutive weeks to test its usability and suitability. The pilot used a mixed methods design comprising quantitative and qualitative study de-signing the app and identifying the needs of participants. We executed usability testing of the app with a purposive sample.11 The joint effort of the multi-disciplinary group focuses both on the replies to a semi-structured interview from three different testing groups, aiming at highlighting strengths, open issues, and desirable feature improvements, and on quantitative collected data analysis, involving a testing group of 149 people, whose evaluation strategy leverages on well-known statistical methodologies to confirm known behavior patterns and determine unforeseen ones.
The app satisfies the software inspection criteria and security requirements of the Apple App Store and the Google Play App Store.
Ethical approval
The study was approved by the IERB of the Department of Education Sciences University of Catania and was conducted in agreement with the ethical norms set by the Italian National Psychological Association. The study was performed in accordance with the ethical recommendations of the Helsinki Declaration.
App design and user interaction
The development of the app is the result of the collaboration between computer scientists and clinician researchers experienced in addiction, clinical psychology and dietary changes.
The addiction cessation process could lead to severe withdrawals, which can be associated with various negative effects, such as irritability, weight loss or gain, eating disorders. The main idea is to monitor the user’s dietary habits and anomalies at different times during the quitting process to assess correlations between the observed data and known information about the cessation treatment.
All the extracted data were exploited to infer the dietary habits, the kind and amount of taken food and how many and at what times a user has a meal. Furthermore, some inferences on user’s behavior could be carried out, i.e., mood changes over time.
Testing the impact of the app usage
The FoodRec app has been tested on a group of 149 people, aged between 19 and 80. Users had a time window of two weeks (14-days) in which they used the app with their smartphones, monitoring his/her dietary habits. The tests were performed for a period of 4 months, starting on 12 August 2020, and ending on 02 December 2020. At the end of the test, users were asked to fill in a survey to give feedback of their experience to improve the app features for the future.
The extracted data were mainly used to collect user-related characteristics, such as birthdate, sex, enrollment period, nationality, perception of weight in everyday life, and their dietary habits, such as quantity and time of meals, drinks intake. Special focus also on personal mood before eating has been drawn.
The “Think-Aloud” method allows end-users to say aloud their views, emotional state, and the comments they make when they are testing the app.12 Task analyses were employed to evaluate features and the user interface of the app.12
For the duration of the study, participants were required to complete the tracking app daily and to attend a final onsite visit to complete a qualitative semi-structured interview with the participants to fully understand feasibility, acceptability and how they experienced the use of the app. The following questions were asked: “Does the app work on your device? What challenges have you experienced using the app? What needs to be changed before a new version? Did the app help you? How did the app help you?”.
The interviews were conducted by two researchers and the participants’ responses to the semi-structured interview were noted during the interviews. Data was analyzed according to the principles of qualitative description.13 Notes were summarized into key units, arranged according to their shared content, and finally identified. A third researcher reviewed the sum-ups and classes, and discrepancies were fixed by dialogue.
Users features
Table 1 shows the distribution of the 149 active smoking users enrolled in the tests subdivided by gender and three birthdate ranges.
In Table 2, the number of users is classified by three levels of importance (“low”, “medium”, high") given by themselves to their weight with respect to the study and further divided by birthdate ranges. The percentage of users has been calculated with respect to the users’ percentage within a certain birthdate range to provide a normalized view of the information being examined. For each birthdate group, the most testers are counted in the intermediate (“medium”) range as to their perception of weight importance in every day’s life.
Results
Dietary habits
This subsection presents a collection of stats based on the data automatically accrued by the app during the two-week usage period.
During the testing phase, the involved users have permanently uploaded 1529 meals whose distribution is shown in Table 3. Morning and evening snack have been introduced in addition to the three traditional main meals: breakfast, lunch, and dinner. An estimate of the average upload daytime of meals has also been calculated as for the five different meal categories. The average upload time is skewed toward the three main meals a bit later than expected. This may suggest a certain degree of eating disorder among the app testers, but it must be taken into account their eating habits and some confusion in identifying the right meal type on uploading.
The input of the user’s mood condition while uploading the meal is another feature provided by the app. In that respect, a couple of significant plots have been drawn showing the distribution of the five possible conditions whose labels have been defined as “very sad”, “sad”, “so so”, “happy”, “very happy” in Figure 1, corresponding to the five mood icons in the app’s GUI.
The plot reveals that largest slice of users seems to exhibit very good mood conditions while approaching food intake, once again confirming that food is not only fuel for the body but also for the soul.14
Another correlation shown in Figure 2 highlights that the most favorable mood states happen at lunch time, where the expectations from food are the highest ones, a result that adds a new measurement to food-related emotions in consumer research.15 Conversely, the bad mood conditions are equally distributed throughout the four meals, revealing that those conditions belong more to a state-of-mind rather than to the meal type.
The last set of statistics are about the intakes of drinks that the app is able to record. The histogram in Figure 3 plots the numbers of recorded intakes with respect to their volume expressed in cl.
The histogram in Figure 4 plots the distribution of drinks intakes throughout thirteen timeslots, where the highest number is related to the timeslot corresponding to the lunch’s average upload time (see Table 3).
User evaluation and usage with Feasibility/Acceptability Testing
The two-week (14 days) feasibility study evaluated the use of the app in real-life settings. Participants’ attitudes towards the app were examined, revealing practical and technological challenges. Adherence data, defined as at least ≥75% of days (9/12) of usage, exceeded 75% of all study participants.
149 participants were interviewed in person after completing the post-assessment with semi-structured questions. Participants found the app useful and declared that it provided them with a tool of self-management support, considering the opportunity to daily self-monitor food consumption. They all reported that they would recommend the app to family and friends. In addition, participants considered the app easy to use. However, some technical difficulties have been reported below, considering that the feasibility testing will inform and facilitate successful app implementation.
The detailed qualitative results of the interviews are summarized in the following descriptive categories: App working, App helps, Task analysis of navigating in Food Rec App, Critical points, Changes for the new version.
App working
All participants reported to have had a good approach with the app because it was extremely easy to use and met their expectations. Sosme participants began to reflect on daily nutrition, calling it “monotonous” and/or “unhealthy”, “the app allows you to monitor the food that is consumed every day, something often forgotten or neglected: this could lead to a balanced and varied diet”.
By some participants, the absence of the daily calories calculation has been appreciated for any users suffering from eating disorders, who may feel uncomfortable and pay special attention to kcal. Common opinion concerns the graphics, very pretty, with warm and bright colors that favor positive emotions.
The advice before meals and the relationship between the emotions felt before a meal and the latter are also appreciated. The reminder is also useful, although it would prefer to be editable according to the habits of the users.
App helps
Some qualities of the app were appreciated such as the attractive, streamlined and intuitive graphics of the “home”, which allows an approach to it by an audience with a lower mastery of the technological context. It was appreciated the small weight that makes it easily usable on their device and the statistical area within the app that allows easy monitoring of the diet.
All participants declared that the app helped them monitor their eating behavior and 86 (58%) participants reported that the app helped them reduce their food consumption and understand their eating behavior in a better way. 95 (64%) participants also reported that knowing they had the app gave a sense of support and served as a reminder to regulate food intake daily.
Task analysis of navigating in FoodRec App
50 participants performed four task assignments to assess their ability to navigate in the App. The task assignments were performed on a smartphone used by all participants.
Each task assignment was scored by the interviewer using the codes:
3. Carries out the assignment independently,
2. Carry out the assignment with oral guidance,
1. Carry out the assignment with physical guidance,
0. Cannot do the assignment.
The results from the task analysis are printed in Table 4.
The first three task assignments: enter “which type of meal will be inserted”, choose “moods that can be associated with the meal”, carry out the image upload of the user’s meal photo, were performed independently by all participants. For the last task assignment: select ingredients and add them in case the algorithm fails, ten participant needed oral guidance.
Critical points
Many participants expressed the following concerns: the function of the app is not well understood, no feedback is provided, neither positive nor negative on the meals that are consumed. Most think that the photograph of each meal is time-consuming and could lead to not using the app and often the reading of the dish does not correspond to the dish itself, some ingredients are added, and in the dish, instead, they are not present, and vice versa: Sometimes the ingredients are not read. Moreover, with the absence of some foods, it is not possible to measure the subjectivity of users, who could consume the same meal, but with variables. Participants expressed difficulty in entering the list of ingredients, excessively cumbersome and lacking in terms and suggested that it would be much easier to allow the user to enter the ingredients and food in general manually.
As further critical points they suggested: the inability to change the times at which it would be more appropriate to receive notifications for meals, such a lack can lead the subject to forget the use at the appropriate time of the app; the initial questionnaire presents a lack of useful data such as: height, pathologies (mental and/or physical) that affect the weight (hormonal decompensations etc.), any treatments, any diet followed, intolerances and / or allergies, any addictions; Impossibility in entering the quantities relating to the ingredients used.
Changes for the new version
Smoking participants, consider potentially important to mark the count of daily cigarettes smoked to understand how mood, food and smoking interact with each other. Therefore, it would be useful to count the number of cigarettes smoked daily, and correlate this information with all the other data recorded by the FoodRec app. It is also important to be able to state a daily target for water consumption, also creating a weekly/monthly chart. In this way, it would be possible to trigger a scheduled notification that reminds the user to drink according to a specific time interval and target.
It would be useful to keep the mood data so that you can look at them again later and receive feedback or updates related to the food/mood association, that is, a graph that perhaps shows the mood trend in relation to the foods consumed. It would be interesting for women to be able to enter data relating to menstruation, as they greatly affect mood and choice of foods.
It is desirable to insert a graph that calculates the prevalence of positive or negative emotions corresponding to the meals consumed. You could insert a recipe book, with healthy recipes and suitable for the user.
Discussion
Although the final aim of the FoodRec app is to infer correlations among dietary habit changes and the smoking quitting process, the presented study is focused on the degree of appreciation of the app functionalities from a selected group of test users. Mood status and dietary habits during the test period were extracted by means of statistical methodologies applied to a variety of information that the App asked the user to input while using it and processed in relation to test users’ features and meal types.
At the end of the study, participants’ feedback has been collected. The majority expressed positive comments, declaring that it is easy to use the app and it helped them on their daily self-monitored food consumption. Additionally, they affirmed that they would recommend the app to family and friends.
According to interviews, participants found the app very useful for monitoring their daily food consumption, in particular they appreciate the absence of daily calorie kilo calculation, the relation between the emotion felt before the meal and the meal itself, the design of the app with bright colors that convey positive emotions and the graphics attraction and intuitiveness of the app’s “home”.
Efforts have been made to support and help people to monitor their dietary habits. Lee, Ji-Eun, et al.10 developed an app, Diet-A, that monitors dietary intake among adolescents. They used a voice or text mode input to insert each meal. Ahn, Jeong Sun, et al.16 presents a smartphone application for dietary self-monitoring, where participants could select food and recipe from a database and insert their meal; nutrient intakes are evaluated, and daily statistics are calculated to help the user to monitor his food consumption. Both applications use a voice/text input to insert a meal, while we chose to use the photo of the meal because it makes the upload of it faster and more intuitive, increasing the effectiveness and usability of our application.
The present study revealed some limitations. First, participants have expressed some suggestions related to the app usage. The app could be enriched with new functionalities like the possibility to change the meal notification time, improve the precision of the image recognition algorithm, add the possibility to specific food, ingredients, and increase the number of personal information that the user can insert during the registration. In addition, it could be possible to implement new artificial intelligence algorithms in order to evaluate new inferences that could be useful both for the user because he can better monitor his habits and for us because we will be able to extract more precise data.
Conclusions
In summary, we have preliminary tested the FoodRec App specifically developed with the aim to visually recognize ingredients and related calories of eaten meals, based on pictures shot and uploaded into the app by a selected group of 149 test users. App users are smokers included in a smoke quitting process, which tested the impact of using the app in two weeks’ time, in order to assess whether the usage of the app would help change the dietary and smoking habits of the participants.
The feedback suggests that the FoodRec App was perceived as extremely user-friendly and lightweight. It also turned out to be useful in the perception of users’ own dietary habits and helpful in relieving the stress of a food intake reduction process that, in a considerable measure, took place as a consequence of the app usage.
Issues related to some missing correlation between the smoke quitting process and the food recognition approach are to be solved with the next versions of the app, according to the upcoming project’s phases,7 which will also make use of enhanced Artificial Intelligence techniques. Indeed, several studies proposed to exploit AI on user taken pictures and other sources for lifelogging and health applications,17–19 as well as for developing advanced tools for large scale population survey.20
This work represents a first step towards investigating the role and impact of the FoodRec App in a large international and multicultural context. The experience gained in the current study will be used to modify and refine the large international RCT protocol version of the app. Moreover, the collected data will be exploited to further investigate the relations between food images and mood,21 with the aim to develop advanced inference tools able to monitor and analyze the patient mood during the cessation program. New data and new inferences will support our future efforts that will be aimed to demonstrate how effective the use of FoodRec can be in alleviating stress, irritability, and worsening dietary eating, consequences of smoking cessation.
Acknowledgements
Written consent was obtained from participants and confidentiality and anonymity were assured.
Contributorship
Conceptualization, S.B. and P.C.; methodology, P.C.; software, R.L.; vali-dation, S.B., A.O. and R.P.; writing—original draft preparation, G.M and A.M; writing—review and editing, P.C., A.O.; visualization, S.B.; supervision, R.P.; project administration, S.B., A.O. All authors have read and agreed to the published version of the manuscript.
Conflicting interests
Riccardo Polosa is full tenured professor of Internal Medicine at the University of Catania (Italy) and Medical Director of the Institute for Internal Medicine and Clinical Immunology at the same University. In relation to his recent work in the area of respiratory diseases, clinical immunology, and tobacco control, RP has received lecture fees and research funding from Pfizer, Glax-oSmithKline, CV Therapeutics, NeuroSearch A/S, Sandoz, MSD, Boehringer Ingelheim, Novartis, Duska Therapeutics, and Forest Laboratories. Lecture fees from a number of European EC industry and trade associations (including FIVAPE in France and FIESEL in Italy) were directly donated to vaper advocacy no-profit organizations. RP has also received grants from European Commission initiatives (U-BIOPRED and AIRPROM) and from the Integral Rheumatology & Immunology Specialists Network (IRIS) initiative. He has also served as a consultant for Pfizer, Global Health Alliance for treatment of tobacco dependence, CV Therapeutics, Boehringer Ingelheim, Novartis, Duska Therapeutics, ECITA (Electronic Cigarette Industry Trade Association, in the UK), Arbi Group Srl., Health Diplomats, and Sermo Inc. RP has served on the Medical and Scientific Advisory Board of Cordex Pharma, Inc., CV Therapeutics, Duska Therapeutics Inc, Pfizer, and PharmaCielo. RP is also founder of the Center for Tobacco prevention and treatment (CPCT) at the University of Catania and of the Center of Excellence for the acceleration of Harm Reduction (CoEHAR) at the same University, which has received support from Foundation for a Smoke Free World to conduct 8 independent investigator-initiated research projects on harm reduction. RP is currently in-volved in a patent application concerning an app tracker for smoking behavior developed for ECLAT Srl. RP is also currently involved in the following pro bono activities: scientific advisor for LIAF, Lega Italiana Anti Fumo (Italian acronym for Italian Anti-Smoking League), the Consumer Advocates for Smoke-free Alternatives (CASAA) and the International Network of Nicotine Consumers Organizations (INNCO); Chair of the European Technical Committee for standardization on “Requirements and test methods for emissions of electronic cigarettes” (CEN/TC 437; WG4).
Pasquale Caponnetto initiated research supported by GRAND (Global Research Award for Nico-tine Dependence) an independently reviewed competitive grants program funded by Pfizer Inc (US). The other authors declare no conflict of interest.
Funding
This investigator-initiated study was sponsored by ECLAT srl, a spin-off of the University of Catania, with the help of a grant from the Foundation for a Smoke-Free World Inc., a US nonprofit 501(c)(3) private foundation with a mission to end smoking in this generation. The contents, selection, and presentation of facts, as well as any opinions expressed herein are the sole responsibility of the authors and under no circumstances shall be regarded as reflecting the positions of the Foundation for a Smoke-Free World, Inc. ECLAT srl. is a research-based company from the University of Catania that delivers solutions to global health problems with special emphasis on harm minimization and technological innovation.