大数据技术基础

本教程由厦门大学计算机科学系教师林子雨对大量网络资料和相关书籍内容进行阅读整理后编写而成，可以作为计算机专业研究生课程《大数据技术基础》的辅助教材。林子雨的E-mail是：[email protected] 林子雨的个人主页是：http://www.cs.xmu.edu.cn/linziyu介绍大数据技术相关基础知识，包括大数据概述、大数据处理模型、大数据关键技术、大数据时代面临的新挑战、NoSQL数据库、云数据库、Google Spanner、Hadoop、HDFS、HBase、MapReduce、Zookeeper、流计算、图计算和Google Dremel等

PipelineJoin：A New MapReduce-based Multi-table Join Algorithm

MapReduce是一种并行分布式计算模型，用于大规模数据集的并行运算。它具有良好的可扩展性、容错性、可用性，现在，无论在工业界还是在学术界都得到了广泛的应用。MapReduce比较热门的一个应用领域是处理大型表的连接操作，其中连接操作广泛应用于日志分析、数据分析处理以及联机分析处理等方面。应用MapReduce模型来处理连接操作，可以大幅度提高连接操作的速度，进而提高了数据分析效率和用户满意度。 现有的基于MapReduce的连接算法，按连接表的数量可以分为两表连接和多表连接两种。目前，针对两表连接的研究优化已经相当成熟，而针对多表连接的算法及其优化仍有很大的改进空间。特别是在处理链式多表...MapReduce, a parallel and distributed computing model, has been used to process parallel computing for large-scale data sets. It has a good scalability, fault tolerance and availability. Now it has been widely used in industry and academia. And processing join operations for large tables is one of the most popular aspects in MapReduce applications. Join operations are widely used in log analysis, ...学位：工学硕士院系专业：信息科学与技术学院_计算机科学与技术学号：2302013115316

JD.com (A): New Chief Human Resources Officer

自1998年诞生于北京中关村以来，京东集团始终处于高速奔跑状态，从传统企业起步到成为电子商务领域的佼佼者，京东以迅猛的增长速度出现在公众视野。截至2012年，京东集团已经发展成为中国最大的自营B2C电商。2009—2012年间，公司员工数量迅速从几千人增长至3万人，但依然面临巨大的人才缺口。为了保持后续发展速度，创始人刘强东于2012年前后陆续引入外部高管人才，空降CXO，其中包括新任CHO隆雨。 案例A主要描述了2012年8月隆雨上任后面临的一系列人力资源管理挑战，启发学员思考在应对这些挑战时应优先考虑什么；案例B主要描述了隆雨上任第一年着手做了什么，以及在执行改革过程中的具体措施——包括企业文化梳理和人才盘点，并启发学员思考在未来几年应如何继续应对人力资源管理方面的挑战。本系列案例较全面地反映了京东集团自开创以来在人力资源工作方面所遇到的种种挑战。In 2012, JD.com (hereafter “the company”) had emerged to be the biggest B2C (Business to Consumer) e-commerce retailer in China. From 2009 to 2012, the company’s headcount quickly grew from several thousand to nearly thirty thousand. Still, it was short of talents; and the human resources (HR) department’s role and business impact were perceived as insignificant. Its founder and CEO, Richard Liu (hereafter Richard), realized that he had to strengthen the company’s internal management in order to sustain rapid growth. In 2011 and 2012, he recruited several chief officers (CxOs), including Rain Long (hereafter Rain)—chief human resources officer (CHO) and general counsel. Case A describes the challenges she faced when joining the company in August 2012, and invites students to think about what her priorities should be when tackling these challenges. Case B describes what Rain decided to do and how she executed her priorities in the first year—primarily, two improvement projects—culture consolidation and talent review. It asks students to set priorities on tackling the remaining HR problems in the coming few years

JD.com (B): Culture Consolidation and Talent Review

自1998年诞生于北京中关村以来，京东集团始终处于高速奔跑状态，从传统企业起步到成为电子商务领域的佼佼者，京东以迅猛的增长速度出现在公众视野。截至2012年，京东集团已经发展成为中国最大的自营B2C电商。2009—2012年间，公司员工数量迅速从几千人增长至3万人，但依然面临巨大的人才缺口。为了保持后续发展速度，创始人刘强东于2012年前后陆续引入外部高管人才，空降CXO，其中包括新任CHO隆雨。 案例A主要描述了2012年8月隆雨上任后面临的一系列人力资源管理挑战，启发学员思考在应对这些挑战时应优先考虑什么；案例B主要描述了隆雨上任第一年着手做了什么，以及在执行改革过程中的具体措施——包括企业文化梳理和人才盘点，并启发学员思考在未来几年应如何继续应对人力资源管理方面的挑战。本系列案例较全面地反映了京东集团自开创以来在人力资源工作方面所遇到的种种挑战。In 2012, JD.com (hereafter “the company”) had emerged to be the biggest B2C (Business to Consumer) e-commerce retailer in China. From 2009 to 2012, the company’s headcount quickly grew from several thousand to nearly thirty thousand. Still, it was short of talents; and the human resources (HR) department’s role and business impact were perceived as insignificant. Its founder and CEO, Richard Liu (hereafter Richard), realized that he had to strengthen the company’s internal management in order to sustain rapid growth. In 2011 and 2012, he recruited several chief officers (CxOs), including Rain Long (hereafter Rain)—chief human resources officer (CHO) and general counsel. Case A describes the challenges she faced when joining the company in August 2012, and invites students to think about what her priorities should be when tackling these challenges. Case B describes what Rain decided to do and how she executed her priorities in the first year—primarily, two improvement projects—culture consolidation and talent review. It asks students to set priorities on tackling the remaining HR problems in the coming few years

SURFACE DISTRIBUTION of RADIUM ISOTOPES IN THE PRYDZ BAY AND ITS ADJACENT SEA AREAS

中国第27次南极科学考察期间(2010年12月30日至2011年1月16日),对普里兹湾及其邻近海域表层海水进行了226rA和228rA的分析,结果表明:226rA和228rA比活度的变化范围分别为1.47—2.43bQ/M3和0.17—0.45bQ/M3,平均值分别为2.13bQ/M3和0.29bQ/M3,228rA/226rA)A.r.(228rA与226rA的活度比)的变化范围为0.08—0.20,平均值为0.14。根据盐度和226rA的质量平衡方程,计算出研究海域表层水中冰融水、南极夏季表层水和普里兹湾中深层水的份额。研究海域表层水中温度、盐度、226rA、228rA、228rA/226rA)A.r.和冰融水份额的空间分布显示,在埃默里冰架前沿海域,西侧海域较东侧海域具有低温、高盐、高226rA、低228rA、低228rA/226rA)A.r.、低冰融水份额的特征,证实埃默里冰架下水体东进西出的运动规律。根据埃默里冰架前沿东、西侧水体228rA/226rA)A.r.的差异,估算出埃默里冰架下表层水体东进西出所经历的时间为1.85A。此外,在普里兹湾湾口中部海域(66.5—67.5°S,72°—74°E),观察到次表层水的上升通风作用,该区域较高的228rA含量和228rA/226rA)A.r.证明这些表层水体并非来自湾外绕极深层水的上涌,而可能来自湾内埃默里冰架输出水体。226Ra and 228 Ra in surface water in the Prydz Bay Antarctica and its adjacent sea areas were measured during the 27 th China Antarctic Research Expedition(from December 30, 2010 to January 16, 2011).Our results show that 226 Ra and 228 Ra activity concentrations ranged from 1.47 to 2.43Bq/m3 and from 0.17 to 0.45Bq/m3, in average of 2.13Bq/m3 and 0.29Bq/m3, respectively.228Ra/226Ra)A.R.varied from 0.08 to 0.20 in average of 0.14.Based on mass balance of salinity and 226 Ra, fractions of three components, i.e.ice-melted water, Antarctic summer surface water, and the Prydz Bay deep water, were calculated.Along the front of the Amery ice shelf, characteristics of low temperature, high salinity, high 226 Ra, low 228 Ra, low 228Ra/226Ra)A.R.and low fraction of the ice-melted water were observed in the western region compared to the eastern region, indicating that waters flow into the ice shelf in the eastern region and exit in the western region.The elapsed time for surface water transport from the eastern to the western was estimated as 1.85 a according to the change of surface 228Ra/226Ra)A.R..In addition, ventilation of sub-surface water was observed in the central mouth of the Prydz Bay(66.5°—67.5°S, 72—74°E).Relatively high 228 Ra and 228Ra/226Ra)A.R.in the central mouth of the Prydz Bay suggested that these surface waters came from the exported waters from the western Amery ice shelf, not from the upwelling of circumpolar deep water.南北极环境综合考察与评估专项;CHINARE2014-01-04-03号;CHINARE2014-04-01-06号; 国家自然科学基金杰出青年基金项目;41125020号; 厦门大学海洋科学国家人才基地项目;2013C01

川西周公山柳杉人工林群落的边缘效应

为研究人工林群落的边缘效应特征,本文以川西周公山森林公园的柳杉(Cryptomeria fortunei)人工林破碎化大斑块为对象,以植株平均胸径、平均高度、平均密度、丰富度指数(D)和Shannon-Wiener指数(H)来综合衡量边缘效应深度。在实地踏查的基础上,从林缘向林内(梯度1至梯度5)设置5条样带(宽度为10 m),在每条样带中设置4个10 m×10 m的小样方进行调查。结果表明:(1)从物种组成上看,在总面积为2,000 m~2的20个小样方中共记录到111个物种,隶属于54科96属,物种数从林缘至林内递减。(2)从群落结构上看,乔木层的平均胸径从林缘至林内呈减小趋势,平均密度则相反,平均高度无显著变化;灌木层的平均密度从林缘向林内减小,平均高度无显著变化;草本层的平均密度和平均高度均呈减小趋势。(3)从物种多样性上看,总体上各层次的丰富度指数和Shannon-Wiener指数均从林缘向林内呈减小趋势,其中灌木层和草本层的变化趋势最明显;同时,林内各梯度与梯度1(林缘)的共有种和相似性系数从林缘向林内递减。(4)分析各项群落特征发现,平均高度、平均密度和相似性系数的数值在梯度2向梯度3过渡时的起伏变化最明显,推断本研究中柳杉人工林斑块的边缘深度可达林内20 m

油橄榄果实经济性状随成熟度的变化Changes of economic characters of Olea europaea fruit with maturity

为确定油橄榄的最佳采收期，测定了4个油橄榄品种（‘佛奥’‘科拉蒂’‘小苹果’和‘米扎’）不同成熟度果实的表型性状和内在品质，观察了果实经济性状随成熟度的变化，最后采用主成分分析法对其品质进行评价，并探讨了不同品种的最佳成熟度。结果表明：随着成熟度增加，4个品种油橄榄果实单果质量总体呈上升趋势，而果形指数呈相反的变化趋势，除‘科拉蒂’的果肉率在第五成熟度达到最大值外，其余品种的果肉率均在第三成熟度达到最大值；不同成熟度的4个品种油橄榄果实鲜果含水率为42.07%~68.48%，干基含油率为40.60%~55.37%,‘佛奥’‘小苹果’和‘米扎’果实在第四成熟度达到最大干基含油率，分别为54.06%、49.10%和45.66%，‘科拉蒂’的干基含油率在第五成熟度达到最大值（55.37%）。随着成熟度增加，4个品种果实的蛋白质含量总体较稳定；还原糖含量变化不一，呈现波浪式的变化；总多酚和总黄酮含量总体呈下降的趋势。不同品种不同成熟度的油橄榄果实中油脂的脂肪酸组成与含量存在差异，具体表现为随着成熟度增加，亚油酸含量增加，油酸含量降低，MUFA/PUFA和油酸/亚油酸呈下降趋势。‘科拉蒂’橄榄油的UFA含量最高，为85.01%，而SFA含量最低，为15.28%。对4个品种果实的表型性状、干基含油率、主要化合物含量、主要脂肪酸组成及含量等共14个指标进行主成分分析并依据综合得分进行排序，发现不同品种有不同的最佳成熟度，其中‘科拉蒂’和‘佛奥’为第一成熟度，‘小苹果’为第五成熟度，‘米扎’为第四成熟度；品质最好的品种为‘科拉蒂’，‘佛奥’和‘小苹果’表现相当，其次为‘米扎’。 In order to determine the best harvest time of Olea europaea, phenotypic traits and intrinsic quality of the fruits of four Olea europaea varieties (Frantoio, Coratina, Manzanila and Mixaj) with different maturity levels were determined, and the changes of economic characters of the fruits with maturity were observed. Finally the quality of the fruits was evaluated by principal component analysis to discuss the best maturity of different varieties. The results showed that with the increase of maturity, the fruit weight of the four varieties of olives showed an overall upward trend, while the fruit shape index showed an opposite trend. Except that the pulp rate of Coratina reached the maximum at the fifth maturity, the pulp rate of others reached the maximum at the third maturity. The water content of fresh fruit was 42.07%-68.48%, and the oil content of dry basis was 40.60%-55.37%. Frantoio, Manzanila and Mixaj reached the maximum oil contents of dry basis at the fourth maturity, which were 54.06%, 49.10% and 45.66% respectively. Coratina reached the maximum oil contents of dry basis at the fifth maturity (55.37%). With the increase of maturity, the protein contents of each variety was generally stable; the content of reducing sugar varied, showing a wave-like change; the contents of total polyphenols and total flavonoids generally showed a downward trend. The fatty acid composition and content of oils from different varieties and maturity levels fruits were different. As the maturity increased, the content of linoleic acid increased and the content of oleic acid decreased, and MUFA/PUFA and oleic acid/linoleic acid showed a downward trend. Coratina had the highest UFA content at 85.01%, while the SFA content was the lowest at 15.28%. Principal component analysis was performed on 14 indicators such as phenotypic traits of fruit, oil content of dry basis, main compounds content, main fatty acid composition and content, and comprehensively scores were calculated according to principal component scores, and it was found that different varieties had different optimal maturity. The optimal maturity for Coratina and Frantoio fruits were the first maturity. The optimal maturity for Manzanila was the fifth maturity, and for Mixaj it was the fourth maturity. The variety with best quality was Coratina, with Frantoio and Manzanila performed similarly, followed by Mixaj

JUNO Sensitivity on Proton Decay p→νˉK+p\to \bar\nu K^+ Searches

The Jiangmen Underground Neutrino Observatory (JUNO) is a large liquid scintillator detector designed to explore many topics in fundamental physics. In this paper, the potential on searching for proton decay in $p\to \bar\nu K^+$ mode with JUNO is investigated.The kaon and its decay particles feature a clear three-fold coincidence signature that results in a high efficiency for identification. Moreover, the excellent energy resolution of JUNO permits to suppress the sizable background caused by other delayed signals. Based on these advantages, the detection efficiency for the proton decay via $p\to \bar\nu K^+$ is 36.9% with a background level of 0.2 events after 10 years of data taking. The estimated sensitivity based on 200 kton-years exposure is $9.6 \times 10^{33}$ years, competitive with the current best limits on the proton lifetime in this channel