GPU-based Private Information Retrieval for On-Device Machine Learning Inference

On-device machine learning (ML) inference can enable the use of private user data on user devices without revealing them to remote servers. However, a pure on-device solution to private ML inference is impractical for many applications that rely on embedding tables that are too large to be stored on-device. In particular, recommendation models typically use multiple embedding tables each on the order of 1-10 GBs of data, making them impractical to store on-device. To overcome this barrier, we propose the use of private information retrieval (PIR) to efficiently and privately retrieve embeddings from servers without sharing any private information. As off-the-shelf PIR algorithms are usually too computationally intensive to directly use for latency-sensitive inference tasks, we 1) propose novel GPU-based acceleration of PIR, and 2) co-design PIR with the downstream ML application to obtain further speedup. Our GPU acceleration strategy improves system throughput by more than $20 \times$ over an optimized CPU PIR implementation, and our PIR-ML co-design provides an over $5 \times$ additional throughput improvement at fixed model quality. Together, for various on-device ML applications such as recommendation and language modeling, our system on a single V100 GPU can serve up to $100,000$ queries per second -- a $>100 \times$ throughput improvement over a CPU-based baseline -- while maintaining model accuracy

Widening Access to Applied Machine Learning with TinyML

Broadening access to both computational and educational resources is critical to diffusing machine-learning (ML) innovation. However, today, most ML resources and experts are siloed in a few countries and organizations. In this paper, we describe our pedagogical approach to increasing access to applied ML through a massive open online course (MOOC) on Tiny Machine Learning (TinyML). We suggest that TinyML, ML on resource-constrained embedded devices, is an attractive means to widen access because TinyML both leverages low-cost and globally accessible hardware, and encourages the development of complete, self-contained applications, from data collection to deployment. To this end, a collaboration between academia (Harvard University) and industry (Google) produced a four-part MOOC that provides application-oriented instruction on how to develop solutions using TinyML. The series is openly available on the edX MOOC platform, has no prerequisites beyond basic programming, and is designed for learners from a global variety of backgrounds. It introduces pupils to real-world applications, ML algorithms, data-set engineering, and the ethical considerations of these technologies via hands-on programming and deployment of TinyML applications in both the cloud and their own microcontrollers. To facilitate continued learning, community building, and collaboration beyond the courses, we launched a standalone website, a forum, a chat, and an optional course-project competition. We also released the course materials publicly, hoping they will inspire the next generation of ML practitioners and educators and further broaden access to cutting-edge ML technologies.Comment: Understanding the underpinnings of the TinyML edX course series: https://www.edx.org/professional-certificate/harvardx-tiny-machine-learnin

Widening Access to Applied Machine Learning With TinyML

Broadening access to both computational and educational resources is crit- ical to diffusing machine learning (ML) innovation. However, today, most ML resources and experts are siloed in a few countries and organizations. In this article, we describe our pedagogical approach to increasing access to applied ML through a massive open online course (MOOC) on Tiny Machine Learning (TinyML). We suggest that TinyML, applied ML on resource-constrained embedded devices, is an attractive means to widen access because TinyML leverages low-cost and globally accessible hardware and encourages the development of complete, self-contained applications, from data collection to deployment. To this end, a collaboration between academia and industry produced a four part MOOC that provides application-oriented instruction on how to develop solutions using TinyML. The series is openly available on the edX MOOC platform, has no prerequisites beyond basic programming, and is designed for global learners from a variety of backgrounds. It introduces real-world applications, ML algorithms, data-set engineering, and the ethi- cal considerations of these technologies through hands-on programming and deployment of TinyML applications in both the cloud and on their own microcontrollers. To facili- tate continued learning, community building, and collaboration beyond the courses, we launched a standalone website, a forum, a chat, and an optional course-project com- petition. We also open-sourced the course materials, hoping they will inspire the next generation of ML practitioners and educators and further broaden access to cutting-edge ML technologies

What Stimulates Researchers to Make Their Research Usable? Towards an Openness Approach

Ambiguity surrounding the effect of external engagement on academic research has raised questions about what motivates researchers to collaborate with third parties. We argue that what matters for society is research that can be absorbed by users. We define openness as a willingness by researchers to make research more usable by external partners by responding to external influences in their own research practices. We ask what kinds of characteristics define those researchers who are more open to creating usable knowledge. Our empirical study analyses a sample of 1583 researchers working at the Spanish Council for Scientific Research (CSIC). Results demonstrate that it is personal factors (academic identity and past experience) that determine which researchers have open behaviours. The paper concludes that policies to encourage external engagement should focus on experiences which legitimate and validate knowledge produced through user encounters, both at the academic formation career stage as well as through providing ongoing opportunities to engage with third parties.The data used for this study comes from the IMPACTO project funded by the Spanish Council for Scientific Research - CSIC (Ref. 200410E639). The work also benefited from a mobility grant awarded by Eu-Spri Forum to Julia Olmos Penuela & Paul Benneworth for her visiting research to the Center of Higher Education Policy Studies. Finally, Julia Olmos Penuela also benefited from a post-doctoral grant funded by the Generalitat Valenciana (APOSTD-2014-A-006).Olmos-Peñuela, J.; Benneworth, P.; Castro-Martínez, E. (2015). What Stimulates Researchers to Make Their Research Usable? Towards an Openness Approach. Minerva. 53(4):381-410. https://doi.org/10.1007/s11024-015-9283-4S381410534Abreu, Maria, Vadim Grinevich, Alan Hughes, and Michael Kitson. 2009. Knowledge exchange between academics and the business, public and third sectors. Cambridge: Centre for Business Research and UK-IRC.Aghion, Philippe, Mathias Dewatripont, and Jeremy C. Stein. 2008. Academic freedom, private-sector focus, and the process of innovation. RAND Journal of Economics 39: 617–635.Ajzen, Icek. 2001. Nature and operation of attitudes. Annual Review of Psychology 52(1): 27–58.Alrøe, Hugo Fjelsted, and Erik Steen Kristensen. 2002. Towards a systemic research methodology in agriculture: Rethinking the role of values in science. Agriculture and Human Values 19(1): 3–23.Audretsch, David B., Werner Bönte, and Stefan Krabel. 2010. Why do scientists in public research institutions cooperate with private firms. In DRUID Working Paper, 10–27.Baldini, Nicola, Rosa Grimaldi, and Maurizio Sobrero. 2007. To patent or not to patent? A survey of Italian inventors on motivations, incentives, and obstacles to university patenting. Scientometrics 70(2): 333–354.Bandura, Albert. 1977. Social learning theory. Englewood Cliffs, NJ: Prentice-Hall.Barnett, R. 2009. Knowing and becoming in the higher education curriculum. Studies in Higher Education 34(4): 429–440.Becher, Tony. 1994. The significance of disciplinary differences. Studies in Higher Education 19(2): 151–161.Becher, Tony, and Paul Trowler. 2001. Academic tribes and territories: Intellectual enquiry and the culture of disciplines. McGraw-Hill International.Bekkers, Rudi, and Isabel Maria Bodas Freitas. 2008. Analysing knowledge transfer channels between universities and industry: To what degree do sectors also matter? Research Policy 37(10): 1837–1853.Belderbos, René, Martin Carree, Bert Diederen, Boris Lokshin, and Reinhilde Veugelers. 2004. Heterogeneity in R&D cooperation strategies. International Journal of Industrial Organization 22(8): 1237–1263.Benner, Mats, and Ulf Sandström. 2000. Institutionalizing the triple helix: Research funding and norms in the academic system. Research Policy 29(2): 291–301.Bercovitz, Janet, and Maryann Feldman. 2008. Academic entrepreneurs: Organizational change at the individual level. Organization Science 19(1): 69–89.Berman, Elizabeth Popp. 2011. Creating the market university: How academic science became an economic engine. Princeton University Press.Bleiklie, Ivar, and Roar Høstaker. 2004. Modernizing research training-education and science policy between profession, discipline and academic institution. Higher Education Policy 17(2): 221–236.Bozeman, Barry, Daniel Fay, and Catherine P. Slade. 2013. Research collaboration in universities and academic entrepreneurship: The-state-of-the-art. The Journal of Technology Transfer 38(1): 1–67.Collini, Stefan. 2009. Impact on humanities: Researchers must take a stand now or be judged and rewarded as salesmen. The Times Literary Supplement 5563: 18–19.D’Este, Pablo, and Markus Perkmann. 2011. Why do academics engage with industry? The entrepreneurial university and individual motivations. The Journal of Technology Transfer 36(3): 316–339.D’Este, Pablo, Oscar Llopis, and Alfredo Yegros. 2013. Conducting pro-social research: Cognitive diversity, research excellence and awareness about the social impact of research: INGENIO (CSIC-UPV) Working Paper Series.Deem, Rosemary, and Lisa Lucas. 2007. Research and teaching cultures in two contrasting UK policy contexts: Academic life in education departments in five English and Scottish universities. Higher Education 54(1): 115–133.DiMaggio, Paul J., and Walter W. Powell. 1983. The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review 48(2): 147–160.Downing, David B. 2005. The knowledge contract: Politics and paradigms in the academic workplace. Lincoln: Nebraska University of Nebraska Press.Donovan, Claire. 2007. The qualitative future of research evaluation. Science and Public Policy 34(8): 585–597.Durning, Bridget. 2004. Planning academics and planning practitioners: Two tribes or a community of practice? Planning Practice and Research 19(4): 435–446.Edquist, Charles. 1997. System of innovation approaches: Their emergence and characteristics. In Systems of innovation: Technologies, institutions and organizations, ed. C. Edquist, 1–35. London: Pinter.Etzkowitz, Henry, and Loet Leydesdorff. 2000. The dynamics of innovation: from National Systems and “Mode 2” to a Triple Helix of university–industry–government relations. Research Policy 29(2): 109–123.Fromhold-Eisebith, Martina, Claudia Werker, and Marcel Vojnic. 2014. Tracing the social dimension in innovation networks. In The social dynamics of innovation networks, eds. Roel Rutten, Paul Benneworth, Frans Boekema, and Dessy Irawati. London: Routledge (in press).Geuna, Aldo, and Alessandro Muscio. 2009. The governance of university knowledge transfer: A critical review of the literature. Minerva 47(1): 93–114.Gibbons, Michael, Camille Limoges, Helga Nowotny, Simon Schwartzman, Peter Scott, and Martin Trow. 1994. The new production of knowledge: The dynamics of science and research in contemporary societies. London: Sage.Gläser, Jochen. 2012. How does Governance change research content? On the possibility of a sociological middle-range theory linking science policy studies to the sociology of scientific knowledge. Technical University Berlin. Technology Studies Working Papers. http://www.ts.tu-berlin.de/fileadmin/fg226/TUTS/TUTS-WP-1-2012.pdf . Accessed 16 Feb 2015.Goethner, Maximilian, Martin Obschonka, Rainer K. Silbereisen, and Uwe Cantner. 2012. Scientists’ transition to academic entrepreneurship: Economic and psychological determinants. Journal of Economic Psychology 33(3): 628–641.Gulbrandsen, Magnus, and Jens-Christian Smeby. 2005. Industry funding and university professors’ research performance. Research Policy 34(6): 932–950.Haeussler, Carolin, and Jeannette Colyvas. 2011. Breaking the ivory tower: Academic entrepreneurship in the life sciences in UK and Germany. Research Policy 40(1): 41–54.Hessels, Laurens K., Harro van Lente, John Grin, and Ruud E.H.M. Smits. 2011. Changing struggles for relevance in eight fields of natural science. Industry and Higher Education 25(5): 347–357.Hessels, Laurens K., and Harro Van Lente. 2008. Re-thinking new knowledge production: A literature review and a research agenda. Research Policy 37(4): 740–760.Hoye, Kate, and Fred Pries. 2009. ‘Repeat commercializers’, the ‘habitual entrepreneurs’ of university–industry technology transfer. Technovation 29(10): 682–689.Jacobson, Nora, Dale Butterill, and Paula Goering. 2004. Organizational factors that influence university-based researchers’ engagement in knowledge transfer activities. Science Communication 25(3): 246–259.Jain, Sanjay, Gerard George, and Mark Maltarich. 2009. Academics or entrepreneurs? Investigating role identity modification of university scientists involved in commercialization activity. Research Policy 38(6): 922–935.Jasanoff, Sheila, and Sang-Hyun Kim. 2013. Sociotechnical imaginaries and national energy policies. Science as Culture 22(2): 189–196.Jensen, Pablo. 2011. A statistical picture of popularization activities and their evolutions in France. Public Understanding of Science 20(1): 26–36.Kitcher, Philip. 2001. Science, truth, and democracy. Oxford: Oxford University Press.Knorr-Cetina, Karin. 1981. The manufacture of knowledge: An essay on the constructivist and contextual nature of science. Oxford: Pergamon Press.Kronenberg, Kristin, and Marjolein Caniëls. 2014. Professional proximity in research collaborations. In The social dynamics of innovation networks, eds. Roel Rutten, Paul Benneworth, Frans Boekema, and Dessy Irawati. London: Routledge (in press).Krueger, Rob, and David Gibbs. 2010. Competitive global city regions and sustainable development’: An interpretive institutionalist account in the South East of England. Environment and planning A 42: 821–837.Lam, Alice. 2011. What motivates academic scientists to engage in research commercialization: ‘Gold’, ‘ribbon’ or ‘puzzle’? Research Policy 40(10): 1354–1368.Landry, Réjean, Malek Saïhi, Nabil Amara, and Mathieu Ouimet. 2010. Evidence on how academics manage their portfolio of knowledge transfer activities. Research Policy 39(10): 1387–1403.Lee, Alison, and David Boud. 2003. Writing groups, change and academic identity: Research development as local practice. Studies in Higher Education 28(2): 187–200.Lee, Yong S. 1996. ‘Technology transfer’ and the research university: A search for the boundaries of university–industry collaboration. Research Policy 25(6): 843–863.Lee, Yong S. 2000. The sustainability of university–industry research collaboration: An empirical assessment. The Journal of Technology Transfer 25(2): 111–133.Leisyte, Liudvika, Jürgen Enders, and Harry De Boer. 2008. The freedom to set research agendas—illusion and reality of the research units in the Dutch Universities. Higher Education Policy 21(3): 377–391.Louis, Karen Seashore, David Blumenthal, Michael E. Gluck, and Michael A. Stoto. 1989. Entrepreneurs in academe: An exploration of behaviors among life scientists. Administrative Science Quarterly 34(1): 110–131.Lowe, Philip, Jeremy Phillipson, and Katy Wilkinson. 2013. Why social scientists should engage with natural scientists. Contemporary Social Science 8(3): 207–222.Martín-Sempere, María José, Belén Garzón-García, and Jesús Rey-Rocha. 2008. Scientists’ motivation to communicate science and technology to the public: Surveying participants at the Madrid Science Fair. Public Understanding of Science 17(3): 349–367.Martin, Ben. 2003. The changing social contract for science and the evolution of the university. In Science and innovation: Rethinking the rationales for funding and governance, eds. A. Geuna, A.J. Salter, and W.E. Steinmueller, 7–29. Cheltenhan: Edward Elgar.Merton, Robert K. 1973. The sociology of science: Theoretical and empirical investigations. Chicago: University of Chicago Press.Miller, Thaddeus R., and Mark W. Neff. 2013. De-facto science policy in the making: how scientists shape science policy and why it matters (or, why STS and STP scholars should socialize). Minerva 51(3): 295–315.Muthén, Bengt O. 1998–2004. Mplus Technical Appendices. Muthén & Muthén. Los Angeles, CA.: Muthén & Muthén.Nedeva, Maria. 2013. Between the global and the national: Organising European science. Research Policy 42(1): 220–230.Neff, Mark William. 2014. Research prioritization and the potential pitfall of path dependencies in coral reef science. Minerva 52(2): 213–235.Nelson, Richard R. 2001. Observations on the post-Bayh-Dole rise of patenting at American universities. The Journal of Technology Transfer 26(1): 13–19.Nowotny, Helga, Peter Scott, and Michael Gibbons. 2001. Re-thinking science: Knowledge and the public in an age of uncertainty. Cambridge: Polity Press.Olmos-Peñuela, Julia, Paul Benneworth, and Elena Castro-Martínez. 2014a. Are ‘STEM from Mars and SSH from Venus’? Challenging disciplinary stereotypes of research’s social value. Science and Public Policy 41: 384–400.Olmos-Peñuela, Julia, Elena Castro-Martínez, and Manuel Fernández-Esquinas. 2014b. Diferencias entre áreas científicas en las prácticas de divulgación de la investigación: un estudio empírico en el CSIC. Revista Española de Documentación Científica. doi: 10.3989/redc.2014.2.1096 .Ouimet, Mathieu, Nabil Amara, Réjean Landry, and John Lavis. 2007. Direct interactions medical school faculty members have with professionals and managers working in public and private sector organizations: A cross-sectional study. Scientometrics 72(2): 307–323.Perkmann, Markus, Valentina Tartari, Maureen McKelvey, Erkko Autio, Anders Brostrom, Pablo D’Este, Riccardo Fini, et al. 2013. Academic engagement and commercialisation: A review of the literature on university-industry relations. Research Policy 42(2): 423–442.Philpott, Kevin, Lawrence Dooley, Caroline O’Reilly, and Gary Lupton. 2011. The entrepreneurial university: Examining the underlying academic tensions. Technovation 31(4): 161–170.Rutten, Roel, and Frans Boekema. 2012. From learning region to learning in a socio-spatial context. Regional Studies 46(8): 981–992.Sarewitz, Daniel, and Roger A. Pielke. 2007. The neglected heart of science policy: reconciling supply of and demand for science. Environmental Science & Policy 10(1): 5–16.Sauermann, Henry, and Paula Stephan. 2013. Conflicting logics? A multidimensional view of industrial and academic science. Organization Science 24(3): 889–909.Schein, Edgar H. 1985. Organizational culture and leadership: A dynamic view. San Francisco, CA: Jossey-Bass.Shane, Scott. 2000. Prior knowledge and the discovery of entrepreneurial opportunities. Organization Science 11(4): 448–469.Spaapen, Jack, and Leonie van Drooge. 2011. Introducing ‘productive interactions’ in social impact assessment. Research Evaluation 20(3): 211–218.Stokes, Donald E. 1997. Pasteur’s quadrant: Basic science and technological innovation. Washington, DC: Brookings Institution Press.Tartari, Valentina, and Stefano Breschi. 2012. Set them free: scientists’ evaluations of the benefits and costs of university–industry research collaboration. Industrial and Corporate Change 21(5): 1117–1147.Tinker, Tony, and Rob Gray. 2003. Beyond a critique of pure reason: From policy to politics to praxis in environmental and social research. Accounting, Auditing & Accountability Journal 16(5): 727–761.van Rijnsoever, Frank J., Laurens K. Hessels, and Rens L.J. Vandeberg. 2008. A resource-based view on the interactions of university researchers. Research Policy 37(8): 1255–1266.Venkataraman, Sankaran. 1997. The distinctive domain of entrepreneurship research: An editor’s perspective. Advances in Entrepreneurship, Firm Emergence, and Growth 3: 119–138.Verspagen, Bart. 2006. University research, intellectual property rights and European innovation systems. Journal of Economic Surveys 20(4): 607–632.Villanueva-Felez, Africa, Jordi Molas-Gallart, and Alejandro Escribá-Esteve. 2013. Measuring personal networks and their relationship with scientific production. Minerva 51(4): 465–483.Watermeyer, Richard. 2015. Lost in the ‘third space’: the impact of public engagement in higher education on academic identity, research practice and career progression. European Journal of Higher Education (online first, doi: 10.1080/21568235.2015.1044546 ).Weingart, Peter. 2009. Editorial for Issue 47/3. Minerva 47(3): 237–239.Ziman, John. 1996. ‘Postacademic science’: Constructing knowledge with networks and norms. Science Studies 1: 67–80.Zomer, Arend H., Ben W.A. Jongbloed, and Jürgen Enders. 2010. Do spin-offs make the academics’ heads spin? The impacts of spin-off companies on their parent research organisation. Minerva 48(3): 331–353

Impact of Cerebral Microbleeds in Stroke Patients with Atrial Fibrillation

OBJECTIVES: Cerebral microbleeds are associated with the risks of ischemic stroke and intracranial hemorrhage, causing clinical dilemmas for antithrombotic treatment decisions. We aimed to evaluate the risks of intracranial hemorrhage and ischemic stroke associated with microbleeds in patients with atrial fibrillation treated with Vitamin K antagonists, direct oral anticoagulants, antiplatelets, and combination therapy (i.e. concurrent oral anticoagulant and antiplatelet) METHODS: We included patients with documented atrial fibrillation from the pooled individual patient data analysis by the Microbleeds International Collaborative Network. Risks of subsequent intracranial hemorrhage and ischemic stroke were compared between patients with and without microbleeds, stratified by antithrombotic use. RESULTS: A total of 7,839 patients were included. The presence of microbleeds was associated with an increased relative risk of intracranial hemorrhage (aHR 2.74, 95% confidence interval 1.76 - 4.26) and ischemic stroke (aHR 1.29, 95% confidence interval 1.04 - 1.59). For the entire cohort, the absolute incidence of ischemic stroke was higher than intracranial hemorrhage regardless of microbleeds burden. However, for the subgroup of patients taking combination of anticoagulant and antiplatelet therapy, the absolute risk of intracranial hemorrhage exceeded that of ischemic stroke in those with 2-4 microbleeds (25 vs 12 per 1,000 patient-years) and ≥11 microbleeds (94 vs 48 per 1,000 patient-years). INTERPRETATION: Patients with atrial fibrillation and high burden of microbleeds receiving combination therapy have a tendency of higher rate of intracranial hemorrhage than ischemic stroke, with potential for net harm. Further studies are needed to help optimize stroke preventive strategies in this high-risk group. This article is protected by copyright. All rights reserved