What broke where for distributed and parallel applications — a whodunit story

Detection, diagnosis and mitigation of performance problems in today\u27s large-scale distributed and parallel systems is a difficult task. These large distributed and parallel systems are composed of various complex software and hardware components. When the system experiences some performance or correctness problem, developers struggle to understand the root cause of the problem and fix in a timely manner. In my thesis, I address these three components of the performance problems in computer systems. First, we focus on diagnosing performance problems in large-scale parallel applications running on supercomputers. We developed techniques to localize the performance problem for root-cause analysis. Parallel applications, most of which are complex scientific simulations running in supercomputers, can create up to millions of parallel tasks that run on different machines and communicate using the message passing paradigm. We developed a highly scalable and accurate automated debugging tool called PRODOMETER, which uses sophisticated algorithms to first, create a logical progress dependency graph of the tasks to highlight how the problem spread through the system manifesting as a system-wide performance issue. Second, uses this logical progress dependence graph to identify the task where the problem originated. Finally, PRODOMETER pinpoints the code region corresponding to the origin of the bug. Second, we developed a tool-chain that can detect performance anomaly using machine-learning techniques and can achieve very low false positive rate. Our input-aware performance anomaly detection system consists of a scalable data collection framework to collect performance related metrics from different granularity of code regions, an offline model creation and prediction-error characterization technique, and a threshold based anomaly-detection-engine for production runs. Our system requires few training runs and can handle unknown inputs and parameter combinations by dynamically calibrating the anomaly detection threshold according to the characteristics of the input data and the characteristics of the prediction-error of the models. Third, we developed performance problem mitigation scheme for erasure-coded distributed storage systems. Repair operations of the failed blocks in erasure-coded distributed storage system take really long time in networked constrained data-centers. The reason being, during the repair operation for erasure-coded distributed storage, a lot of data from multiple nodes are gathered into a single node and then a mathematical operation is performed to reconstruct the missing part. This process severely congests the links toward the destination where newly recreated data is to be hosted. We proposed a novel distributed repair technique, called Partial-Parallel-Repair (PPR) that performs this reconstruction in parallel on multiple nodes and eliminates network bottlenecks, and as a result, greatly speeds up the repair process. Fourth, we study how for a class of applications, performance can be improved (or performance problems can be mitigated) by selectively approximating some of the computations. For many applications, the main computation happens inside a loop that can be logically divided into a few temporal segments, we call phases. We found that while approximating the initial phases might severely degrade the quality of the results, approximating the computation for the later phases have very small impact on the final quality of the result. Based on this observation, we developed an optimization framework that for a given budget of quality-loss, would find the best approximation settings for each phase in the execution

Constitutional Design, Democratic Vote Counting, and India's Fortuitous Multiculturalism

Following independence, the Indian state, with fresh memories of the communal violence that marked the partition of the subcontinent, committed itself to an unprecedented experiment of actualising the ideal of multiculturalism as a cornerstone of the nation and the most important basis of its legitimacy. The legitimacy of the state structure was based on the twin principle of individual rights and protection of minorities. This entailed a constitutional design committed to denying hegemony to any religion. Subsequently, as the message of democracy spread, this gave rise to many new problematic issues. Ethnic and national minorities challenged the state and its capacity to accommodate conflicting identities by demanding neutrality as well as genuine recognition and active support for their culture and religion. The essay examines this contested character of India's constitutionally guaranteed multiculturalism on the basis of the history of state formation, the freedom movement, the uncertainty of the ultimate nature of divinity in Hinduism, and thereby, illustrates how post-colonial India was able to devise a series of concrete institutions and policies in order to work her way towards new conceptions of the rights and status of minorities. Thus, the specific case of India’s theoretically fuzzy multiculturalism and the abstract issue of accommodation are juxtaposed to some existing measures of the Constitution of India as well as some survey data of about ten thousand men and women shortly after the parliamentary elections of 1996. Drawing on aspects of India's political culture and the debate on Hindu theology, the essay suggests that contrary to the spectre of the rise of Hindu 'fundamentalism', India presents a relatively successful case of the growth of a multicultural nation, ensconced within of a post-colonial, democratic state

The Novelty of Europe as seen from the Periphery: Indian Perception of the 'New Europe' in a Multi-polar world

Europa ist, so verstanden, dass eine Stimme den Anspruch erhebt, für alle fünfundzwanzig zu sprechen, selbst für den vermeintlichen Europäer etwas Neues. Verständlicherweise herrscht bei den Außenstehenden mit dem Blick nach Europa hinein eher Unverständnis und Konfusion. Folglich bleibt das Verständnis des neuen Europas bei Nicht-Europäern vage, widersprüchlich und fragwürdig, begleitet von einem besorgten Unterton. In diesem Artikel werden, schwerpunktmäßig aus der Sicht Indiens, die Ursprünge der Diskrepanz zwischen der Wahrnehmung Europas von innen heraus und der Wahrnehmung von außen her von den im Zeitalter des Kolonialismus geprägten Kategorien des „rationalen Ich“ und des „irrationalen Anderen“ abgeleitet. Ausgehend von den englischen, französischen und deutschen Genealogien dieser grundlegenden Dichotomie wird in diesem Beitrag aufgezeigt, wie diese Kategorien von Hegemonie und Unterwerfung im Kontext ihrer post-koloniale Praxis Stück für Stück gestutzt wurden. Die Analyse weist auf das Potential für einen reflexiven Diskurs-Modus hin, der auf die natürliche Affinität zwischen Indien und dem neuen Europa zurückgreifen und auf diese Weise die ontologische Basis für einen geeigneten Diskurs und geeignete Institutionen in einer multipolaren Welt schaffen könnte

Politics and the room to manoeuvre: democracy, social opportunity and poverty in India

Ownership is the essence of economic citizenship. Beyond actual possession, a sense of personal welfare and proprietorship, or at least the hope of achieving them, constitute a necessary and important complement to being stakeholders in a society. Together with efficacy and legitimacy, these are necessary attributes of political agency. In this chapter we examine the interplay between democra- cy, social opportunity and economic security, drawing upon survey data to ex- plore popular perceptions of India’s new economic policy. The article first situ- ates itself within the context of new theoretical literature on the multi- dimensional nature of poverty and how to measure it, then identifies aggregate indicators of the performance of Indian states and the overall achievements and failings of India in terms of poverty alleviation. In the second half, the article identifies what seems to be the lack of a ‘politics of poverty’ in India and the various cultural, historical, political explanations that have been proffered for this apparent anomaly. Finally, the impact of democracy on poverty reduction is examined through the programmes that have been launched and which aim at creating a level-playing field, but which nevertheless have the potential to de- generate into highly populist measures

Collective Violence and the Making of Civil Society : India in European Perspective

Are riots, risings and revolutions acts of collective madness, or are they political events, offering a rare glimpse into the inner workings of a system, laying bare the legitimacy deficit that moves rational men and women to kill and die? Fun, profit, peer pressure, or moral outrage - which of these is the main motive of rioters? Is collective violence a form of violent participation, which, in the final analysis, acts as the midwife of civil society? The full investigation of these general and comparative questions is beyond the scope of my paper. Still, they provide the main inspiration for the empirical analysis undertaken here. The paper develops a model of conflict resolution based on countervailing powers, the symbolic recognition of memories of violence, and new institutional arrangements. This framework is used for the analysis of three identity-related issues from post-independence Indian politics. Two of these have been successfully resolved where as attempts to resolve the third have been less successful. Drawing on the contributions of Natalie Davis (1973), Ian Gilmour (1992) Pierre Nora (1989) and Simon Schama (1989) to collective violence and the foundation of civil societies in the west, the paper characterises the outbreak of pogroms, riots, and other forms of collective violence as political phenomena that indicate deeply seated conflicts over the core values of a society. How these conflicts are solved has important implications for the establishment of an institutional framework that promotes a society based on interpersonal trust, respect of individuals and groups, orderly rule and the rights of expression and association. Scholarly interest in the role of violence in accelerating social change has gone out of fashion since the general diffusion of utopian ideas like democracy, social capital and world governance in western liberal democracies. The insistence of donors in the North and their clients in the South on these canons as the only modes of correct political behaviour has consequence for transitional societies of the South that are far from benign. Newcomers to the high table of states and nations, these candidate-members must earn this privilege by subjecting their political conduct to the rules laid down by the members of the club who conveniently overlook the tortuous path they themselves have had to take to reach their current institutional forms. By the same logic, scholarly inquiries into riots, pogroms, insurgencies and other forms of political unrest in non-western societies must conform to a prescribed code of conduct by first condemning their subject before engaging in an analysis of the social process that has led to its outbreak. The liberal bias, often accompanied by the failure to situate the collective violence in its context, results in the blatant characterisation of these political acts as bizarre, perverse or simply as the proof of the cultural incapacity of the societies where they occur to sustain civilised norms in public life. Such a failure of imagination and empathy would, in an academic debate, be risible if its costs in terms of avoidable suffering were not so immense. It is not my intention in this paper to engage in cultural one-upmanship, nor to exonerate human suffering in terms of cultural idiosyncrasy. Instead, the paper focuses on the origin and demise of collective violence, meted out by one group against another for the sheer reason of their difference. I maintain in this paper that the best chance for the creation of a civil society out of the wreckage of collective violence consists in grounding one's analysis of its origin firmly in the social and historical context, and keeping the scholarly inquiry as close to the actors as possible. Though India is the main empirical context for this paper, the analysis of Indian data undertaken here draws on European examples of collective violence partly to establish parameters for historical comparison, but also to generate analytical space for institutional arrangements that have led to the creation of civil societies in the West where much blood has been shed on account of religious differences

Jagannatha Compared:The politics of appropriation, re-use and regional traditions in India

Many in Orissa, experts and lay devotees alike, think of the ‘cult of Jagannatha’ as unique, and specific to Orissa. This is not unusual because space has a special connotation in Hinduism and Jagannatha, though, technically, the Lord of the Universe, is seen by Oriyas as ‘their’ god. Similar identification of particular gods with particular spaces and people can be seen all over India. However, looked at more closely and comparatively, the political and cultural dynamic that goes into the making of the cult is more general than the cult itself. Available evidence shows that the partial process that underpins India’s regional traditions is based on mutual accommodation of rival sacred beliefs, in other regions of India as well. Based on the comparative accounts and the history of Jagannatha of Orissa, and of temples from north and south India which have been converted either by the Jaina or the Lingayat communities, the paper examines the processes of the re-use of sacred sites and material by the holders of political power and the use of such hybrid sacred objects in the making of regional state traditions

Emerging Major Powers and the International System: Significance of the Indian View

India?s new and contested status as a nuclear power, the scale of her arms purchases, her investment in missile technology and the huge deployment of ground troops on the western front against Pakistan are issues of immediate concern to her South Asian neighbours. Since tension feeds on tension, war in Afghanistan, terrorist attacks in Kolkata, Delhi, Jammu and Srinagar, mounting tension between India and Pakistan over the issue of cross-border terrorism in Kashmir and the recent threat by General Pervez Musharraf to consider the first strike option as part of Pakistan?s strategic response to Indian mobilisation have contributed to the seriousness of the situation. The probability of the regional conflict escalating into large scale nuclear war, or weapons of mass destruction finding their way into the hands of non-state actors, have drawn world attention to South Asia, which has had visits in quick succession by political leaders and military delegations from the United States, UK, Germany, France, China and Russia. The paper, focused on India?s capacities, perceptions and institutional arrangements for the management of security, seeks to evaluate the significance of her status as an ?emerging? power for the security environment in Asia, and its implication for the international system. It analyses the main objective both empirically, and theoretically. The empirical aspect concerns the measurement of India?s economic and military resources according to the conventional indicators of power. These facts, based on experts? accounts, are supplemented by political and institutional factors which are significant for the estimation of the power of a country. In addition, the analysis seeks to juxtapose the views of observers and actors, and locate the strategic perception of the Indian voter, an important factor in her political landscape in view of her active democratic process. These factors of contemporary politics are to be seen in the larger context of India?s political and security culture, history, the structure of the political system. The issue of contextualisation needs to be understood in terms of its methodological implication at the outset, because, while all states are members of the international system, the use to which they put international politics varies from one context to another. Western nation states, products of a long process of nation building, industrialisation and state-formation, seek the promotion of national interest through their strategic initiatives. Post-colonial state-nations, engaged in the process of nation-creation, are more complex in their rhetoric. For these actors, international politics, in addition to being used as an instrument of national interest, also plays a symbolic role in the building of a national profile. The paper seeks to combine both the material and symbolic aspects of Indian policy in the concept of a security doctrine, one that can bring potential power into an effective focus, in the absence of which mere appurtenances of power like guns and ships are just that and not much more. Since the stability of the doctrine, in addition to its coherence is an important parameter of the significance of Indian power, the paper also takes into account the problems of implementation as well. Though there is considerable force to the argument that South Asian security is crucially contingent on the India-China-Pakistan triangle, India remains the biggest power in South Asia, and her significance, in terms of how India sees herself and how others see her, is a key consideration for regional politics. The need for a sophisticated methodological analysis arises paradoxically from the fact that India is a democratic state and an open society, both of which give a false sense of visibility to India?s security profile. Foreign observers, depending on their own national origin and the context, place their bets on predictions of India?s next move either as the ?regional bully? or the ?regional push-over?, and India, Janus-like, often proves both speculations to be right, appearing in the process to be either mystical-moral, or utterly devoid of principle or doctrine. The paper is in three parts. The first examines the state of play by ranking India with reference to her strategic resource endowments. The second part examines India?s strategic doctrine and the organigram of security, and evaluates her potential power in the light of her doctrine. The third part makes a prognosis of the challenging path ahead for India with reference to the unsolved problems concerning her national security. The conclusion reconsiders the main issue posed in the introduction in the light of the analysis undertaken here