Ali Al Essa's, Miad Faezipour's, Jeongkyu Lee's, and Gopala Duggina's poster using Hadoop and MapReduce programming to analyze Twitter data about influenza to analyze flu trends

Al Essa, Ali

Duggina, Gopala Krishna

Faezipour, Miad

Lee, Jeongkyu

UB ScholarWorks

IntroductionFlu Trend Prediction Using Social Media Network DataProblem DefinitionFlu and Seasonal Influenza are seriousproblems that may cause fatalities. About250,000 to 500,000 deaths worldwide each yearare because of flu [1]. Public health providersare in favour of knowing about the seasonal fluas soon as possible in order to take the requiredactions for their communities. Getting an earlywarning will help to prevent the spread of flu inthe population.Nowadays a very large number ofpeople use social media networks on a dailybasis to share their news, events, and even theirhealth status [2]. This leads to the idea of usingcommonly used social media networks for fludetection and providing early warnings topublic health providers to take the right actionat the perfect time. Users of social medianetwork can be used as sensors to predict theflu trend in a specific area and time. The socialmedia network used for this research project isTwitter. It is one of the most widely used socialnetwork. It has over 271 million monthly activeusers [3]. The retrieved data from Twitter isenormous and contains large number ofattributes. However very few attributes willonly be required in this project analysis. Sincethe Twitter data is very huge, Big Data Hadoopsystems and MapReduce programming can beused for analysis and to create a goodprediction model for flu trend and then toprovide health care providers with timelywarnings. This can help health care industry toprovide high quality services at the right time.Data SetThe data set is collected from Twitterweb site using a developed crawler whichworks together with the Twitter API to streamtweets. The crawler is designed to filter thetweets based on flu-related keywords such asflu and influenza. Also the developed crawlercleans the data set to include only the followingattributes: Created_at: tweet posting date  Text: the posted tweet Loc: location of the user. Time Zone: the time zone used by the userThe data set consists of 135,160 tweetscollected during the month of November 2015.Implementation, Analysis,  and ResultsThe prediction of flu spread could be muchmore efficient by taking advantage of Hadoop,MapReduce programming and Hadoop EcoSystems. These tools and techniques can use theavailable twitter data and predict the flu trend at theearliest time. In the MapReduce programming weused one mapper and one reducer to do all theanalysis scenarios. The mapper’s input is just thecollection of tweets about flu. Mapper takes theinput and then does the required mapping processas a key and value pairs. Finally the reducer takesthe output from the mappers and sums up thevalues part.In this study we implemented six different fluprediction analysis scenarios.1.FLU TREND AND SEASONS RELATIONSHIP:Figure 1 describes our first analysis basedon flu trend and seasons relationships.We picked four areas with different seasonsduring November. This analysis shows that theeastern side is the most affected area in November.We conclude that the flu virus became more activeduring fall season.2. FLU TREND BASED ON LOCATION:This analysis shows the flu trend in differentlocations. Figure 2 shows users’ flu-tweets aroundthe world.From this analysis we can say that people inthe USA and EUROPE are most affected whencompared to people in the other locations.3. FLU TREND BASED ON TIME:This analysis shows the number of flu-tweets in different dates. Figure 3 demonstrates theresults of this analysisFrom the figure we can observe that themaximum flu trend was on Thursday November12th and least flu trend on Tuesday November 10th.From this analysis we can say that people areaffected by flu after long working days.4.TOP 5 AFFECTED AREAS BASED ON TIME ZONES:From Figure 4 we can say that people in theareas using the Eastern Time zone are the mostaffected by flu. Based on this analysis people in thisarea have to take necessary actions to prevent theflu.5. FLU AND FLU SHOT:This analysis provides the information onhow many tweets have been posted about flu andgives the number of users who took or are planningto take a flu shot or vaccine. Figure 5 showssurprisingly that 32,370 people out of 135,160tweet about flu shot. This constitutes only 24% ofwhole dataset.Conclusion and Future WorkFigure 2: Flu trend based on locationMost of health care provides need to get anearly warning of the flu season in order to takeright actions. Our results show that using the dataof social media network together with BigDatatools and techniques can provide an early warningabout flu trend, which might help to prevent thespread of flu in the population.For the evaluation, we are planning to useCDC reports and Google Flu Trend as a groundtruth. This will help measure the quality of ourproposed solution output.Most of health care providers take therequired actions only after getting the reports offlu from the Center for Disease Control andPrevention (CDC). This center collects datafrom health care providers to monitor Influenza–Like illnesses. It then publishes the reports.This generally takes one to two weeks delayresulting in the fact that the required warningscome late to the provider’s attention [1]. This iswhile health-care providers need to be warnedearly enough in order to do the right actions toprevent the spread of flu.Ali Al Essa, Dr. Miad Faezipour, Dr. Jeongkyu Lee, and Gopala DugginaDepartment of Computer Science and EngineeringUniversity of Bridgeport, Bridgeport, CT Figure 1: Flu trend and seasons relationshipFigure 3: Flu trend based on timeFigure 4: Top 5 affected areas based on time zonesFigure 5: Tweets on flu and flu shot[1].Achrekar, H., Gandhe, A., Lazarus, R., Yu, S. H., & Liu, B. (2011, April). Predicting flu trends using twitter data. In Computer Communications Workshops (INFOCOM WKSHPS), 2011 IEEE Conference on (pp. 702-707). IEEE.[2].Murthy, D., Gross, A., & Longwell, S. (2011, June). Twitter and e-health: a case study of visualizing cancer networks on twitter. In Information Society (i-Society), 2011 International Conference on (pp. 110-113). IEEE.[3].Nambisan, P., Luo, Z., Kapoor, A., Patrick, T. B., & Cisler, R. (2015, January). Social Media, Big Data, and Public Health Informatics: Ruminating Behavior of Depression Revealed through Twitter. In System Sciences (HICSS), 2015 48th Hawaii International Conference on (pp. 2906-2913). IEEE.[4].Corley, C. D., Cook, D. J., Mikler, A. R., & Singh, K. P. (2010). Text and structural data mining of influenza mentions in web and social media.International journal of environmental research and public health, 7(2), 596-615.References

Flu Trend Prediction Using Social Media Network Data

https://scholarworks.bridgeport.edu/xmlui/bitstream/handle/123456789/1555/156-edited_poster_flu_trend.pdf?sequence=1&isAllowed=y

Flu Trend Prediction Using Social Media Network Data

Abstract

Similar works

Full text

Available Versions

UB ScholarWorks

UB ScholarWorks