Facilitated by modern networked ICT, people around the world organize in self-sustaining communities of practice (CoP) across professional domains and organizational boundaries. Communities thereby pursue the main goal to learn how to do better. With today’s trend for mass individualization, we find a vast number and diversity of small professional niche communities in the long tail, apart from few large communities in the mainstream of the Web. As prerequisite for sustained success, communities must constantly stay aware of quality and impact of their community information systems (CIS). With fast-paced emergent technological and social progress, communities must develop agency to decide which tools best support them and why. While data-driven enterprises spend considerable technical and human resources in sophisticated business intelligence and analytics (BI&A) solutions for such decisions, communities face a real challenge, given their inherent resource scarcity. In meritocratic communities, a shared notion of CIS success must be negotiated from multiple specific, sometimes conflicting and idiosyncratic stakeholder perceptions of CIS success existing in a community. Standard positivist BI&A approaches with myopic focus on financial outcomes lack support for such inherently anti-positivist negotiation processes. Communities require respective support for CIS success awareness in terms of methodology, formalization, and technical infrastructure.As solution to this problem, this dissertation presents MobSOS, a framework for CIS success awareness, tailored to the ontological properties of professional long tail CoP. From a methodological point of view, MobSOS extends a community-oriented design science research methodology by an explicit notion for CIS success modeling as essential part of evaluation. CIS success models play the central role of fluid digital media operationalizing negotiation. From a technical point of view, MobSOS pursues an integrative approach to extend contemporary Web and P2P-based CIS platforms by CIS success awareness with the help of a CIS success modeling toolkit. Based on a comprehensive formalization framework, MobSOS supports operations such as monitoring, assessing, exploring, modeling, measuring, visualizing, validating, sharing, and negotiating different stakeholder notions of CIS success in a community-wide discourse. In the context of several national and international research projects, we successfully applied MobSOS in communities with varying scopes in domains such as health care, multimedia management, and technology-enhanced learning. Typical advanced CIS success models include few, yet highly relevant and effective success factors. They are well-balanced combinations of universal vs. community-tailored success factors and metrics. In particular, we note that CIS success models with full coverage for both quality and impact and focused scope are more useful than models with myopic single-dimension focus and a vague scope. Awareness on CIS quality and impact brings communities agency in form of a better informed decision on CIS tool selection, use or active development. We could demonstrate that CIS success models contributed to determine ideal software configurations, to estimate development efforts, to identify and eliminate bottlenecks, to elicit hidden requirements, or to judge concrete impact on the community. MobSOS also proved as effective learning analytics framework to identify learning patterns, to detect usage anomalies, or to measure learning progress. Our technical evaluations finally show, that MobSOS is an effective, performant, and low-cost open-source framework for CIS success awareness
