Candan, Selcuk K.

Casagranda, Paolo

Sapino, Maria Luisa

English

Institutional Research Information System University of Turin

Context-Aware Proactive Personalization of Linear AudioContentPaolo CasagrandaRai Radiotelevisione italianaand University of TorinoTorino, Italypaolo.casagranda@rai.itMaria Luisa SapinoUniversity of TorinoTorino, Italymlsapino@di.unito.itK. Selçuk CandanArizona State UniversityTempe, USAcandan@asu.eduABSTRACTHow many times did you wish the radio programming wasmore aligned with your interests or current situation? Howmany times did you feel the need to change the channel be-cause of a non-interesting content on your favorite station?Did you ever feel distracted by the audio programming inyour car at a busy intersection? We present a platform forproactive personalization of linear audio content within ahybrid content radio framework. Hybrid content radio pro-gramming aims at enhancing the traditional broadcast radioexperience and augmenting it with audio content related tothe listener’s context. It allows enrichment of the broad-caster’s program schedule with context-aware, personalizedaudio content, with the goal of improving the users’ listeningexperience, decreasing their propensity to channel-surf, andgiving them more targeted content, such as local news, en-tertainment, music and also relevant advertisements. Differ-ently from most of the popular commercial recommendation-based streaming music services, hybrid content radio sys-tematically and automatically adds audio content to an ex-isting, linear audio structure. More specifically, part of thelinear content is replaced, in a proactive way, with contentrelevant to the user’s current context – i.e., profile, emotionalstate, activity, geographical position, weather, or other fac-tors contributing to the state of the listener. In additionto enabling functional enhancement of the radio experience,the presented framework also supports network resource op-timization, allowing effective use of the broadcast channeland the Internet.CCS Concepts•Information systems→ Recommender systems; Lo-cation based services; Multimedia streaming; Speech/ audio search;c©2017, Copyright is with the authors. Published in Proc. 20th Inter-national Conference on Extending Database Technology (EDBT), March21-24, 2017 - Venice, Italy: ISBN 978-3-89318-073-8, on OpenProceed-ings.org. Distribution of this paper is permitted under the terms of the Cre-ative Commons license CC-by-nc-nd 4.0Figure 1: The hybrid content radio audio replace-ment concept, and a view of the prototype app play-ing the clip.KeywordsPersonalization; Recommender Systems; Context-based rec-ommendations; Location-based services; Radio.1. INTRODUCTIONWe present a novel platform for context-aware proactivepersonalization of linear audio content within a hybrid con-tent radio framework, so we specifically focus on audio con-tent delivery. The Proactive Personalyzed Hybrid ContentRadio (or PPHCR) system we present is part of the evo-lution of traditional linear radio, as described at the In-ternational Broadcasting Convention in 2015 [6] and in [7,10]. PPHCR is based on the live radio streams and associ-ated metadata from Rai, the Italian Public Service MediaCompany. Beside dealing with all the intermediate scenarioswhere the broadcaster provides a linear programming, hy-brid content radio recommends enriching and context-basedonline content.1.1 Context-Driven Content-Delivery in Hy-brid Content RadioHybrid content radio aims at making linear broadcast ra-dio more flexible, allowing seamless replacement of partsof the broadcast audio content with relevant audio contentmined from recent podcasts and content archives. The basicmetadata descriptions enabling this service come from theETSI Standards created by the RadioDNS Project, see [9].Figure 1 illustrates the content replacement concept, withDemonstration  Series ISSN: 2367-2005 574 10.5441/002/edbt.2017.73Figure 2: Audio content is recommended followingcontextual information, such as listener’s positionand route. When the user’s car starts moving, thesystem predicts a travel duration ∆T , and tries toallocate the most relevant content for the availabletime ∆T , recommending media items A, B, C, D.Item B is also relevant to location LB the user willreach.the user interface of the prototype client app.Context-aware recommender systems have been subject toincreasing attention in the last few years, see [2]. Some stud-ies specifically focus on location awareness, specially thoserelated to the mobile context, see for example [14] and [11].The proposed PPHCR prototype takes advantage of anovel proactive recommender system (PRS), see [5, 13],capable of deciding the time of the recommendation delivery,as described in Section 1.2. The recommender system pro-vides contextually-relevant alternative audio content thatwill replace part of the broadcast content thus increasingthe user’s satisfaction and decreasing her tendency to switchchannels. Using linear radio as the basic building block forpersonalized radio has two main advantages:• the relevance of the content for the listeners increases,enriching their experience, while they keep on listeningtheir favorite radio station• the efficiency of content delivery can be optimized, ifthe device allows using a broadcast technology to re-ceive the audio from the broadcast channelThe core novelty of the proposed service, compared to otherexisting approaches, consists in the joint usage of linear radiopersonalization and in the type of proactive context-basedrecommendations, based on the listener’s location, move-ment and preferences. PPHCR creates personalized audiocontent suggestions accounting for uncertainties in the mo-bile user’s future path as well as driving conditions duringthe scheduling and delivery of a highly relevant and enjoy-able, and yet non-distracting hybrid-audio content.Figure 2 illustrates an example of proactive recommenda-tion for a driver: the system predicts the route and travelduration and maximizes the relevance of the recommendedmedia items, based on a combination of learned user prefer-ences and geographic relevance of the content. As illustratedin this example, effective delivery of personalized linear ra-dio, with context-based content recommendations, requiresto solve the problem of integrating linear schedule timings,spatial information and listeners’ preferences, making thesolution innovative with respect to existing proposals [4].1.2 System DescriptionThe functionality of the system relies on the server archi-tecture shown in Figure 3 and on the client PPCHR appINTRANETINTERNETUsers	TrackingProfilesFeedbacksTracking		DataU s e r 	 D a t aMetadata Content	RepositoryA u d i o 	 D a t aFTP	Audio	ClipPublic	Rest	API	ServerRabbitMQ	CommunicationClientsRequest ResponseAutomatic	Speech	Recognition	and	ClassificationClip	Data	ManagementRecommender	SystemUser	ManagementLive	Radio	StreamsLive	Radio	Streams	and	MetadataFigure 3: Simplified architecture of the context-aware personalized radio systemthat provides the user interface and collects data relevantto discover the context. The content server is the integra-tion of several components cooperating to offer a real-time,personalized radio service. Radio Rai, the radio division ofRai Public Service Media Company, directly provides 10 live96kbps audio streams [1], the editorial version of more than100 podcasts created every day and the associated sched-ule metadata are used to populate the content repositoryand the metadata DB. The podcasts are classified by theclip data management component according to their cat-egories. News programs, including large parts of speech, areanalyzed using an automatic speech recognizer trained withthe Italian language. The extracted text is then classifiedwith a Bayesian classifier trained with a set of news, ac-cording to a set of 30 categories spacing from art to culture,music, economics.User data are organized by the user management com-ponent. The user’s demographic details are stored in theprofiles DB. The feedbacks DB hosts content navigationlogs sent by the listener’s app together with the implicit orexplicit rating given by the user. The tracking data DB isa PostGIS based spatial DB with the listener’s geographicalinformation. The amount of GPS data arriving to the track-ing data DB requires to periodically process and simplifythem, extracting a compact, discrete model which describesdestination, trajectory, speed, frequency, time of the day andcomplexity. Major staying points on the driving paths arecalculated using a density based location clustering [8] andcomplexity is calculated analysing the trajectory simplifiedusing the Ramer-Douglas-Peucker algorithm (RDP).Contextually relevant linear content recommendations areprovided by the recommender system component usingboth user and audio data clusters. For each user the recom-mender filters a candidate set of media items using content-based relevance based on past listener’s feedbacks. Thena compound relevance score is calculated through weightedcombination of the content-based relevance and the context-based relevance (location, trajectory, speed and time infor-mation). The recommender system then uses the this scoreto identify the recommendation set of content to be deliv-ered to the listener according to a relevance objective func-tion and temporal scheduling and presentation constraints,taking into account driving conditions as well as driver’s pro-575jected distraction levels at intersections and roundabouts atuser’s projected driving path.1.3 Client Android AppThe client PPHCR app, whose interface is shown on theright side of Figure 1, has been implemented on Androidmobile OS. The listener can choose one of the live radio ser-vices, change service, pause, or skip content. While the useris listening to the service, a positive implicit feedback is pe-riodically sent for that audio content. In contrast, each skipaction generates a negative feedback. The app synchronizesmetadata and implements buffering and synchronization toensure that the selected live audio is seamlessly replaced bythe recommended clips.The controls are shown in Figure 1: the user can manu-ally give a feedback, skip current program or navigate thefavorite media items.2. DEMONSTRATION OUTLINEThe demonstration is centered on the personalization oflive radio for a listener on the move, in her car. The move-ment can be real or simulated using an Android third partyapp providing fake locations (see for example [12]). Thepersonalization works in a proactive way, using a proac-tive recommender system as described in [13, 3, 5]. ThePPHCR App collects and sends to the user managementmodule user’s preference data and the GPS locations of themoving listener, allowing to predict the trajectory she’s fol-lowing. Using the listener’s feedback and the routes collectedwhen she moves, the system learns to suggest and play con-tent independently from an explicit user action. The keytasks enabled by the prototype are the following:implicit and explicit user feedback the user can give im-plicit feedback to the content skipping it, or explicitfeedback with the like/dislike buttonsmanual skip users can skip live programs and, thanks tobuffering synchronized with program schedule meta-data, seamlessly replace them with recommended con-tentproactive recommendations destination, trajectory, speedand available time predictions allow to proactively sug-gest a list of media items; geographic information isalso used to refine recommendationseditorial recommendations injection the editor can se-lectively choose and inject recommended audio contentto specific users2.1 Demonstration ScenariosWe present two demonstration scenarios: in the first one,the content change is manually triggered by the listener; inthe second, the enrichment of the audio content is triggeredby a real-time change in the listener’s context – more specif-ically, listener’s movement in space.2.1.1 Manual Program ChangeWhile listening to linear radio, from a broadcast channelsuch as analog FM or digital DAB+, or from the Internet,the user can sometimes wish to listen different content. Gregis passionate about technology and economy, often listeningto programs on this topic during the day. This morningthere is an endless discussion about football results on hisProgram	1AUDIO	BUFFERINGLILLY	LISTENSLIVE	RADIOProgram	2 Program	310:42:30 10:55:00 11:10:00 11:20:0011:00:00 11:15:00 11:25:00Program	1 Program	2 Program	3RECOMMENDED	AUDIO	CLIP CLIP	CLIP	Figure 4: The app recommends an audio clip whileLilly is driving to work (timeline).Figure 5: Control Dashboard: map with the lastlistener’s movementsfavorite radio channel. He is about to zap channel, but nowhis radio app allows him to skip the live program and surfa list of suggested audio clips. After two skips Greg reachesone of his favorite programs: “Wikiradio”.2.1.2 Contextual Proactive RecommendationLilly is a young researcher and an appreciated amateurchef. She always tries new recipes and lets her colleaguestaste the results. Lilly also likes listening radio while shedrives from home to work in the morning and from work tohome in the evening. Sometimes music, more often radiotalks and discussions on several subjects, especially to thoserelated to food, recipes and cookery. In the past it was diffi-cult to find an interesting program. She tried to record audioprograms in the evening for the morning, but it was tricky.This morning is different: her radio app had an update someweeks ago. After she has been driving for some minutes, thePPHCR App automatically plays the last news and, afterthis, an audio clip from ”Decanter” program, discussing thedifferences between French Champagne, Spanish Cava andItalian Prosecco. After that she is pleased to hear the jokesof the time shifted live ”The rabbit’s roar”: the program be-gan 20 minutes ago, but the app can still smoothly presentit after ”Decanter”. She listens, pleased and interested, for-getting the skip button. Figure 4 shows the time-line ofthe personalization process for Lilly. The suggestions arebased on her previous skip history and the knowledge of thecontext.2.2 Control Dashboard576Figure 6: Control Dashboard: list of recommenda-tions to send to a specified userDuring the demonstration a web-based control dashboardwill be used to visualize the users’ behavior during the ex-perimentation. The website visualizes the user’s past trajec-tories, content preference, and the details of the recommen-dation process (see Figure 5). The dashboard also allows tomanual injection of recommendations help test recommen-dations (Figure 6).3. CONCLUSIONS AND FUTUREWORKWe presented a Proactive Personalized Hybrid Radio sys-tem, capable of enhancing the linear radio stream, proac-tively proposing targeted audio content. The recommendedaudio items list and the time to show it is generated usingthe listener’s past preferences and the prediction of her cur-rent movement. The key contributions are the location andmovement information awareness to decide both the timeand items to recommend, and the integration of live broad-cast and personalized audio content. The system allows alistener-centric radio experience, with the possibility of ex-plicit content skips and proactive audio content recommen-dations, while preserving the appeal and sense of connectionbetween listeners of traditional broadcast radio. During thedemonstration, the audience will be able to observe both theuser experience (through the client mobile app) as well as thedata-flow and recommendation generation process (throughthe web based control dashboard).For the future, we are planning to estimate the geographicrelevance of audio items available in the archives. This oper-ation involves the analysis of informative and entertainmentcontent as well as advertisements, validated by a user trial.Furthermore, we plan to create recommendations list takinginto account richer contexts: time, activity, weather, andthe ensemble effect of the recommendations list.4. ACKNOWLEDGMENTSThe authors would like to thank Francesco Russo and Raf-faele Teraoni Prioletti of Rai Research Centre for the discus-sions and support during the system implementation.5. REFERENCES[1] Radio rai. http://www.radio.rai.it. Accessed:2016-11-15.[2] G. Adomavicius and A. Tuzhilin. Context-awarerecommender systems. In Recommender systemshandbook, pages 191–226. Springer, 2015.[3] J. I. A´rnason, J. Jepsen, A. Koudal, M. R. Schmidt,and S. Serafin. Volvo intelligent news: A contextaware multi modal proactive recommender system forin-vehicle use. Pervasive and Mobile Computing, 14:95– 111, 2014. Special Issue on The Social Car:Socially-inspired Mechanisms for Future MobilityServices.[4] M. Braunhofer, M. Kaminskas, and F. Ricci.Location-aware music recommendation. InternationalJournal of Multimedia Information Retrieval,2(1):31–44, 2013.[5] M. Braunhofer, F. Ricci, B. Lamche, and W. Wo¨rndl.A context-aware model for proactive recommendersystems in the tourism domain. In Proceedings of the17th International Conference on Human-ComputerInteraction with Mobile Devices and Services Adjunct,MobileHCI ’15, pages 1070–1075, New York, NY,USA, 2015. ACM.[6] P. Casagranda, A. Erk, S. O’Halpin, D. Born, andW. Huijten. A framework for a context-based hybridcontent radio. In International BroadcastingConvention (IBC), 2015.[7] P. Casagranda, M. L. Sapino, and K. S. Candan.Audio assisted group detection using smartphones. InMultimedia & Expo Workshops (ICMEW), 2015 IEEEInternational Conference on, pages 1–6. IEEE, 2015.[8] M. Ester, H.-P. Kriegel, J. Sander, and X. Xu. Adensity-based algorithm for discovering clusters inlarge spatial databases with noise. In Proceedings ofthe Second International Conference on KnowledgeDiscovery and Data Mining, pages 226–231, 1996.[9] ETSI. Ts 103 270, RadioDNS hybrid radio; hybridlookup for radio services, January 2015.http://www.etsi.org/deliver/etsi ts/103200 103299/103270/01.01.01 60/ts 103270v010101p.pdf.[10] S. Metta, P. Casagranda, A. Messina,M. Montagnuolo, and F. Russo. Leveraging MPEG-21user description for interoperable recommendersystems. In Proceedings of the 31st Annual ACMSymposium on Applied Computing, pages 1072–1074.ACM, April 2016.[11] M. Sarwat, J. J. Levandoski, A. Eldawy, and M. F.Mokbel. LARS*: An efficient and scalablelocation-aware recommender system. IEEETransactions on Knowledge and Data Engineering,26(6):1384–1399, 2014.[12] D. Villeneuve. Lockito. https://play.google.com/store/apps/details?id=fr.dvilleneuve.lockito. Accessed:2016-11-15.[13] W. Woerndl, J. Huebner, R. Bader, andD. Gallego-Vico. A model for proactivity in mobile,context-aware recommender systems. In Proceedings ofthe fifth ACM conference on Recommender systems,pages 273–276. ACM, 2011.[14] H. Yin, Y. Sun, B. Cui, Z. Hu, and L. Chen. LCARS:a location-content-aware recommender system. InProceedings of the 19th ACM SIGKDD internationalconference on Knowledge discovery and data mining,pages 221–229. ACM, 2013.577

Context-Aware Proactive Personalization of Linear Audio Content

https://iris.unito.it/bitstream/2318/1632008/1/paper-433-openproceedings.pdf

Context-Aware Proactive Personalization of Linear Audio Content

Abstract

Similar works

Full text

Available Versions

Institutional Research Information System University of Turin