第20回画像の認識・理解シンポジウム（MIRU2017): ポスター予

Oagata, Hiroaki

Shimada, Atsushi

Suehiro, Daiki

Taniguchi, Yuta

Kyoto University Research Information Repository

Title Extracting Teaching Activities from E-book Logs Using Time-Series ShapeletsAuthor(s)Suehiro, Daiki; Taniguchi, Yuta; Shimada, Atsushi; Oagata,HiroakiCitationMIRU2017 Extended Abstract集 (2017)Issue Date2017-08URL http://hdl.handle.net/2433/232847Right発表概要の著作権は著者に帰属します．Type OthersTextversionpublisherKyoto UniversityCONFIDENTIAL EXTENDED ABSTRACT.DO NOT DISTRIBUTE ANYWHERE.The 20th Meeting on Image Recognition and UnderstandingExtracting Teaching Activities from E-book LogsUsing Time-Series ShapeletsDaiki Suehiro1,a) Yuta Taniguchi1,b) Atsushi Shimada1,c) Hiroaki Ogata2,d)1. IntroductionTeaching analytics for face-to-face classes have been re-searched for many years. In particular, determining theteaching activity from data is highly important. In manyof the related works, the teaching activities or hypothesesare manually designed to be easily understandable. How-ever, the results depend on the knowledge of domain ex-perts and the preparations for considering useful variablesof teaching activities are very time-consuming. The data-driven approaches using digital devices such as wearablesensors achieve moderate success [4]. However, teachinganalytics using IT and digital tools are still at an earlystage, and the cost of using such digital tools for large-scale verication is high.In this paper, we take a novel approach for practic-ing teaching analytics in face-to-face classes as following:First, to collect the data eciently (without optional de-vices or tools) in a face-to-face class, we use e-books withlogs. Second, to extract the teaching activity, we con-sider the page transition as time-series data, and pro-vide a novel analysis scheme using time-series shapelets [7](we call shapelets for short) and machine learning tech-niques [6]. Shapelets-based time-series analysis can pro-vide us with data-driven and interpretive variables fromtime-series data. Finally, the results show that our anal-ysis succeeded in extracting the teaching activities accu-rately.2. Data CollectionIn some classes at Kyushu university in Japan, an e-book system is employed. The teachers show slides and1 Kyushu University2 Kyoto Universitya) suehiro@ait.kyushu-u.ac.jpb) taniguchi@artsci.kyushu-u.ac.jpc) atsushi@ait.kyushu-u.ac.jpd) hiroaki.ogata@gmail.comTable. 1. Example of e-book logsUser Slides Name Page Time   Teacher 1 Statistics 1 3 16/04/30 08:50:09   Student 1 Statistics 1 4 16/04/30 08:50:15   :::::::::::::::Student 9 English 2 22 16/05/23 13:34:30   Teacher 7 English 2 25 16/05/23 13:34:32   Table. 2. Details of seven \Information science" coursesCourse ID Teacher # students # lecturesc1A2615c2 46c3 B 109 15c4C1658c5 239c6D11115c7 43students read the slides on their terminals. The e-booksystem allows us to eciently collect various logs on learn-ers or teachers, such as the page numbers users are view-ing, memos, and other learning actions and along with thetime (e.g., Table 1). In this study, we collected e-booklogs from seven \Information science" courses that weretaught by four teachers during rst semester of 2016. Allof the materials for teaching are shared in the courses. Weprovide the details of the seven courses in Table 2. Thematerials contain 32 sets of slides based on the course con-tents. Each teacher may choose several sets of slides forclassroom use. The number of students in each course arealso described in Table 2.This paper focuses on the teachers' and the students'page-view logs. We believe that we can use these logsto observe some teaching activities that relate to control-ling the class (e.g., teaching speed that considers students'learning capabilities). First, using time-stamp data as inFigure 1, we created a time-series data of page-view num-1The 20th Meeting on Image Recognition and Understandingbers for each student and teacher. To create contiguoustime-series data, we simply ll the last-viewed page num-ber for each time of non-stamped time. Second, in one ofthe main points of this paper, we create secondary datathat is dened by the dierences between the pages thatthe teacher is showing and those that each student is view-ing, which we call DPTS. Moreover, to observe teachingactivities that can provide information on the expertnessof each teacher, a DPTS is created for 32 content, yield-ing 32 DPTSs in total. The value of each data point inthe time series in a DPTS is dened as the page numberof a student minus that of the teacher at a given time inseconds. Hence, the value of DPTS at each time indicatesfollowing properties: If the value is positive, then the student is ahead ofthe teacher. If the value is negative, then the student is behindthe teacher or reviewing. If the value is \0", then student is following theteacher.We think that the page number itself does not have ameaning, but the value of the dierence might indicatewhether students are able to follow the teacher.2.1 Problem SettingFor the rDPTS of each content, we set a (multi-class)time-series classication problem. More precisely, giventime-series data rDPTS = ¹t1; y1º; : : : ; ¹tN; yN º, we wish tond a rule that detects a teacher of unlabeled data, whereti is a time-series example and yi is the label in corre-sponding to the teacher, N is the number of the studentsfor each teacher. It is important to note that detectingteachers is not our true objective but rather is a proceduretowards extracting the teaching activities of the experts.Our true goal is to nd teaching activities that allow usto accurately discriminate between the teachers.2.2 Our Data Mining MethodThere are many methods of resolving the time-seriesclassication problem [5, 8]. In practice, some statisticalvalues of a time window (e.g., mean, max) are used asthe variables because they are easy for domain expertsto understand their meaning. However, such statisticalvalues sometimes omit several important features, suchas small changes of value over a short-period. Therefore,in order to realize our objective, we employ shapelets-based time-series classication methods [8]. Informallyspeaking, shapelets are a set of \short" time series, whichindicates characteristic patterns of the partial transitions,used to detect the label of a time series by whether a givenshapelet partly matches or mismatches for a time series(illustrated in Figure 1). Moreover, as many algorithmsFigure 1. The black line is an original time series, the blue, green,and red line are the shapelets. The feature values of the time se-ries are given by minimum distances between any sub-sequenceof the time series and each shapelet.to nd shapelets retain the full information of the originaltime series, we may nd more precise features (character-istic transition-patterns of each teacher) using this processthan with other methods. With such a property, shapeletsmay nd discriminative and interpretive patterns in timeseries.Here we show more a formal denition. Let ¹t1; : : : ; tN ºbe the sample data of a time series, where N is thenumber ofe xamples. Given a set of K shapelets S =fs1; s2; : : : ; sK g, the feature values ¹xi;1; : : : ; xi;K º of a time-series ti are dened as the minimum Euclidean distancebetween shapelet sj and any sub-sequences of ti [2];xi; j = minl¹ti»l : l + jsj j   1¼   sjº2; (1)where t»a : b¼ sub-sequence ta; ta+1; : : : ; tb. In this paper,we nd suitable shapelets using the method proposed bySuehiro et al. [7], which nds eciently and accuratelynds good shapelets. The method uses sparse SVM (Sup-port Vector Machines), which is one of the most popu-lar machine-learning algorithms for binary classicationproblems*1. However, in this paper, we use sparse AUC-maximizing SVM [6], which we abbreviate to SASVM.SASVM nds the linear hypothesis f that maximizesAUC (Area Under the ROC Curve) [1]. Informally speak-ing, AUC, as known as c-statistic, is a validation measurethat is often used for heavily biased data. More formally,the AUC of any hypothesis h is dened as follows;*1 \Sparse" means that the method induces good feature selec-tion. Hence, we can obtain well-selected teaching activities.2The 20th Meeting on Image Recognition and UnderstandingAUC¹hº = 1pnpÕi=1nÕj=1Ih¹xiº   h¹x 0jº  0; (2)where ¹x1; : : : ; xpº is an example of target label and¹x 01; : : : ; x 0nº are examples of the other label, I¹º is theindicator function that returns \1" if ¹º is true and \0"otherwise. In all pairs of the target examples and the otherexamples, the more closely the h values match the targetexamples, the higher the AUC score (close to 1:0). Ta-ble 2 shows that the number of students in each course isbiased. Furthermore, a multi-class classication problemis usually decomposed into one-vs-all binary class classi-cation sub-problems (e.g., the examples of teacher A arelabeled as positive and the examples of the other teachersare labeled as negative). However, for example, if we tryto solve a one-vs-all sub-problem in which the examplesof teacher A are labeled as positive, the number of posi-tive examples is 72 and the number of negative examplesis 667, we encounter highly biased data. For this reason,we want to maximize AUC but not simple classicationaccuracy. SASVM inputs training examples that are rep-resented as feature vectors and returns a linear functionthat gives higher values to positive examples than nega-tive examples:f ¹xº = w  x = w1x1 +    + wK xK; (3)where w is a weight vector that holdsÍKj=1 jwj j = 1 and indicates the inner product. Each element of w representsthe contributing rate for each shapelet because the featurevalue xj is given by shapelet sj .After we solve the multi-class classication problem foreach content, we obtain the useful shapelets S and thecontribution w that discriminates between the teachers.Moreover, f has a property that ranks the time series ofthe target teacher higher than the other teachers. In otherwords, the time series with a higher value might be thetypical time series of the target teachers.3. Results3.1 Teacher-prediction AccuracyFor the seven \information science" courses of the rstsemester of the 2016 school year, we prepared a set ofDPTS by content. In this paper, we show the result ofprediction accuracy measured by AUC, for three contentsof all 32 contents. Table 3 shows the AUCs for each con-tent evaluated using 5-fold cross-validation. We can seethat our teacher-predicting method achieves high AUCscores*2. Therefore, it is clear that the shapelets extracted*2 AUC is one of the standard measure for classication ac-Table. 3. AUCs for DPTS by each contentContents IDAUCsTeacher A Teacher B Teacher C Teacher DM1 0.872 0.906 0.900 0.889M2 0.894 0.916 0.834 0.856M3 0.890 0.871 0.834 0.860by our method provide some discriminative power for pre-dicting each target teacher. In other words, the extractedshapelets represent some of the teaching activities or thestyles of each teacher.3.2 Observing Teaching Activities via ShapeletAnalysisTo observe good teaching activities in extractedshapelets, in this paper, we focus on the specic con-tent \M2" and teacher C. In fact, the content of \M2"is \image recognition" and teacher C is an expert in im-age recognition. Then, we consider obtaining some goodteaching activities by observing the shapelets and time se-ries of teacher C for \M2." Figure 2 exemplies the threerepresentative results for teacher C. The legend describesthe importance-score of each shapelet (scored by [6]), andthe shapelets shown are the most and second most im-portant characeristic pattern. Note that the shapeletswith negative importance (e.g., the red shapelet as seenin Figure 2) represent the teaching activities of teacher C,but the shapelets with positive weight (e.g., the light-blueshapelet) represent the teaching activities of the otherteachers. Thus, when we want to know the teaching ac-tivities of a targeted teacher, we only have to focus on theshapelets with negative weight. So, by observing Figure 2,Figure 2. Three representative time-series of DPTS for teacherC (black line). The colored lines are highly important shapeletsof teacher C. The legend means the weight of each shapelet. 1time-unit is 10 seconds.we discern the following things:( 1 ) The three typical students seem to follow the teachercuracy. It is said that a hypothesis is considered excellentdiscrimination if the AUC is more than 0:8 [3].3The 20th Meeting on Image Recognition and UnderstandingC because the page-dierences basically remain atzero.( 2 ) The shapelet described by the red line seems to con-stitute a teaching activity by teacher C that controlsthe students to return to the current slide.( 3 ) The activity should be performed at approximately90{110 or 280{300 (the times the red shapeletmatches).We can discern the details of the activities by checkingthe slides or videos used during the \activated" time. In-deed, we checked the slides used during the time, thenwe found that the slides explain a mathematical way ofthinking and the students tend to be behind the teacherduring the slides*3.4. Conclusion and Future WorkIn this paper, we proposed a novel teaching analyticsfor use in face-to-face classes that involve the use of e-books. We focused on the dierence between the pagethat the teacher was showing in the class at a given timeand the pages that a given student was viewing at thattime on their terminal. Second, we set a time-series clas-sication problem, and we solve the problem using time-series shapelets and machine-learning techniques. Theresults show that some extracted shapelets are discrim-inative and interpretable, and suggests the possibility ofeciently discovering teaching activities.References[1] Bradley, A. P.: The use of the area under the ROC curvein the evaluation of machine learning algorithms, PatternRecognition, Vol. 30, pp. 1145{1159 (1997).[2] Hills, J., Lines, J., Baranauskas, E., Mapp, J. and Bagnall,A.: Classication of Time Series by Shapelet Transfor-mation, Data Mining and Knowlege Discovery, Vol. 28,No. 4, pp. 851{881 (2014).[3] Hosmer, D. W. and Lemeshow, S.: Applied logistic re-gression, Wiley series in probability and statistics (2000).[4] Prieto, L. P., Sharma, K., Dillenbourg, P. and Jesus, M.:Teaching analytics: towards automatic extraction of or-chestration graphs using wearable sensors, Proceedings ofLAK '16, pp. 148{157 (2016).[5] Senin, P. and Malinchik, S.: SAX-VSM: InterpretableTime Series Classication Using SAX and Vector SpaceModel, 2013 IEEE 13th International Conference onData Mining, pp. 1175{1180 (2013).[6] Suehiro, D., Hatano, K. and Takimoto, E.: ApproximateReduction from AUC Maximization to 1-norm Soft Mar-gin Optimization, Proceedings of ALT '11, pp. 324{337(2011).*3 Unfortunately, we did not record the videos of the classes,however, teacher C told us to the teaching activity that\slowed down the teaching during the slides and instructedthe students to lay weight on intuitive understanding."[7] Suehiro, D., Kuwahara, K., Hatano, K. and Takimoto, E.:Time Series Classication Based on Random Shapelets,NIPS 2016 Time Series Workshop.[8] Ye, L. and Keogh, E.: Time Series Shapelets: A NewPrimitive for Data Mining, Proceedings of KDD '09,ACM, pp. 947{956 (2009).4

Extracting Teaching Activities from E-book Logs Using Time-Series Shapelets

https://repository.kulib.kyoto-u.ac.jp/dspace/bitstream/2433/232847/1/MIRU.2017_PS2-39.pdf

Extracting Teaching Activities from E-book Logs Using Time-Series Shapelets

Abstract

Similar works

Full text

Available Versions

Kyoto University Research Information Repository