Evaluation of large language models using an Indian language LGBTI+ lexicon

Large language models (LLMs) are typically evaluated on the basis of task-based benchmarks such as MMLU. Such benchmarks do not examine responsible behaviour of LLMs in specific contexts. This is particularly true in the LGBTI+ context where social stereotypes may result in variation in LGBTI+ terminology. Therefore, domain-specific lexicons or dictionaries may be useful as a representative list of words against which the LLM's behaviour needs to be evaluated. This paper presents a methodology for evaluation of LLMs using an LGBTI+ lexicon in Indian languages. The methodology consists of four steps: formulating NLP tasks relevant to the expected behaviour, creating prompts that test LLMs, using the LLMs to obtain the output and, finally, manually evaluating the results. Our qualitative analysis shows that the three LLMs we experiment on are unable to detect underlying hateful content. Similarly, we observe limitations in using machine translation as means to evaluate natural language understanding in languages other than English. The methodology presented in this paper can be useful for LGBTI+ lexicons in other languages as well as other domain-specific lexicons. The work done in this paper opens avenues for responsible behaviour of LLMs, as demonstrated in the context of prevalent social perception of the LGBTI+ community.Comment: Selected for publication in the AI Ethics Journal published by the Artificial Intelligence Robotics Ethics Society (AIRES

Geographic and behavioral differences associated with sexually transmitted infection prevalence among Indian men who have sex with men in Chennai and Mumbai

BACKGROUND: India has one of the largest numbers of men who have sex with men (MSM) globally, however, geographic data on sexually transmitted infection (STI) prevalence and associations with sexual behavior are limited. METHODS: Six-hundred-eight MSM in Chennai and Mumbai screening in for a behavioral trial and assessed bacterial STI (syphilis, chlamydia, gonorrhea), HIV, and past-month self-reported condomless anal sex (CAS). RESULTS: Mumbai (37.8%) had a greater prevalence of any STIs than Chennai (27.6%) (est=1.37, 95% CI:1.09,1.73). This pattern also emerged for gonorrhea and chlamydia separately but not syphilis. Conversely, Mumbai had lower CAS (M=2.2) compared to Chennai (M=14.0) (est=−11.8, 95% CI:−14.6,−9.1). The interaction of city by CAS on any STI prevalence (PR=2.09, 95% CI:1.45,3.01, p<.0001) revealed that in Chennai, higher CAS was not associated with STI prevalence, but in Mumbai it was (PR=2.49, 95% CI:1.65,3.76, p<.0001). DISCUSSION: Higher bacterial STIs but lower CAS in Mumbai versus Chennai, and the significant interaction of CAS with city on STIs suggests that either differences in disease burden or differences by city with respect to self-reported assessment of CAS. Regardless, the high prevalence rates of untreated STI and condomless sex among MSM suggests the need for additional prevention intervention efforts for MSM in urban India

Humanizing an Invisible Population in India: Voices from Bisexual Men Concerning Identity, Life Experiences, and Sexual Health

Research examining the sexual identities, behaviors, and experiences of bisexual men outside of Western contexts (including in India) is limited. Individuals who self-identify as bisexual due to their orientation toward partners of more than one gender face distinct psychosocial challenges relative to exclusively heterosexual, gay/lesbian, or other individuals. We conducted four focus group discussions (n = 22) and in-depth interviews (n = 50) with self-identified bisexual men (age 18 years and older) who were recruited from the metropolitan area of Mumbai, India, between June and August 2013. We triangulated and analyzed focus group and interview data using standard qualitative research techniques. Findings from our study suggest that multiple factors influence the sexual experiences of self-identified bisexual men in Mumbai, including contexts of sexual interactions, sexual positioning, and the gender of sexual partners. Participants described cultural meaning systems and psychosocial dynamics that regulate bisexual identity development, disclosure, and sexual decision making with male, female, and other partners. Secrecy, discretion, and sexual pleasure also influenced sexual behaviors and relationships. Although Western sexual identity categories are not necessarily equivalent in the Indian context, it is interesting and important to note that a number of individuals in India continue to use the identity label of "bisexual." Before developing interventions to meet unique sexual health needs of bisexual men, it is crucial to understand how these men perceive themselves, reconcile the ordinary aspects of their lives with their sexuality, and structure their relationships with partners

A randomized clinical efficacy trial of a psychosocial intervention to strengthen self-acceptance and reduce HIV risk for MSM in India: study protocol.

BackgroundMen who have sex with men (MSM) in India are a key group at risk for HIV acquisition and transmission. They are also an extremely marginalized and stigmatized population, facing immense psychosocial stressors including, but not limited to, stigma, homophobia, discrimination, criminalization, low self-esteem, low self-acceptance, distress, and, as a result, high rates of mental health problems. Although these multi-level psychosocial problems may put MSM at high risk for HIV acquisition and transmission, currently HIV prevention interventions in India do not address them. This paper describes the design of a psychosocial intervention to reduce HIV risk for MSM in India.MethodsFunded by the National Institute of Mental Health, this study is a two-arm randomized clinical efficacy trial of a self-acceptance based psychosocial HIV prevention intervention, informed by the minority stress model and syndemic theory, that was developed with extensive community-based formative work and input from the Indian MSM community and key informants who are knowledgeable about the experiences faced by MSM in India. Participants are MSM in Chennai and Mumbai who endorsed recent sexual behaviors placing them at high risk for HIV/sexually transmitted infection (STI) acquisition and transmission. Enrolled participants are equally randomized to either 1) the experimental condition, which consists of four group and six individual counseling sessions and includes standard of care HIV/STI testing and counseling, or 2) the standard of care condition, which includes HIV/STI testing and counseling alone. The primary outcomes are changes in the frequency of condomless anal sex acts and STI incidence (syphilis seropositivity and urethral, rectal, and pharyngeal gonorrhea and chlamydia infection. Major study assessment visits occur at baseline, 4-, 8-, and 12-months.DiscussionHIV prevention interventions that address the psychosocial stressors faced by MSM in India are needed; this study will examine the efficacy of such an intervention. If the intervention is successful, it may be able to reduce the national HIV/AIDS burden in India while empowering a marginalized and highly stigmatized group.Trial registrationClinicalTrials.gov Identifier: NCT02556294 , registered 22 September 2015

A randomized clinical efficacy trial of a psychosocial intervention to strengthen self-acceptance and reduce HIV risk for MSM in India: study protocol.

BackgroundMen who have sex with men (MSM) in India are a key group at risk for HIV acquisition and transmission. They are also an extremely marginalized and stigmatized population, facing immense psychosocial stressors including, but not limited to, stigma, homophobia, discrimination, criminalization, low self-esteem, low self-acceptance, distress, and, as a result, high rates of mental health problems. Although these multi-level psychosocial problems may put MSM at high risk for HIV acquisition and transmission, currently HIV prevention interventions in India do not address them. This paper describes the design of a psychosocial intervention to reduce HIV risk for MSM in India.MethodsFunded by the National Institute of Mental Health, this study is a two-arm randomized clinical efficacy trial of a self-acceptance based psychosocial HIV prevention intervention, informed by the minority stress model and syndemic theory, that was developed with extensive community-based formative work and input from the Indian MSM community and key informants who are knowledgeable about the experiences faced by MSM in India. Participants are MSM in Chennai and Mumbai who endorsed recent sexual behaviors placing them at high risk for HIV/sexually transmitted infection (STI) acquisition and transmission. Enrolled participants are equally randomized to either 1) the experimental condition, which consists of four group and six individual counseling sessions and includes standard of care HIV/STI testing and counseling, or 2) the standard of care condition, which includes HIV/STI testing and counseling alone. The primary outcomes are changes in the frequency of condomless anal sex acts and STI incidence (syphilis seropositivity and urethral, rectal, and pharyngeal gonorrhea and chlamydia infection. Major study assessment visits occur at baseline, 4-, 8-, and 12-months.DiscussionHIV prevention interventions that address the psychosocial stressors faced by MSM in India are needed; this study will examine the efficacy of such an intervention. If the intervention is successful, it may be able to reduce the national HIV/AIDS burden in India while empowering a marginalized and highly stigmatized group.Trial registrationClinicalTrials.gov Identifier: NCT02556294 , registered 22 September 2015

Longitudinal impact of stressful life events on HIV-related risk and psychosocial problems among MSM in Chennai and Mumbai, India

Background Studies show that stressful life events (SLE) (e.g., discrimination, financial problems) can lead to psychosocial problems and exacerbate condomless anal sex (CAS) without protection via pre-exposure prophylaxis (PrEP) among men who have sex with men. However, few studies have examined this relationship among men who have sex with men in India, and none have examined this longitudinally. Methods As a part of an HIV-prevention intervention, 608 MSM from Chennai and Mumbai, India, completed behavioral surveys at baseline, 4, 8, and 12 months. We used longitudinal generalized estimating equations (GEE) modeling to examine the relationship between SLE and its severity and subsequent psychosocial problems, CAS, and history of diagnosed sexually transmitted infection (STI). All models are adjusted for age, sexual identity, intervention arm, human immunodeficiency virus status, and recruitment city. Results The number of SLE and their corresponding perceived impact score remained consistent at each time point. In multivariable GEE models, the number of SLE was predictive of CAS, depression, and harmful drinking. Similarly, the ratio of the impact of SLE was predictive of CAS, depression, and diagnosed STI. However, harmful drinking was not predictive in this model. Conclusions These findings provide evidence that can inform future interventions, which can be used to enhance self-acceptance, coping skills, and other forms of resiliency

Strengthening resilience to reduce HIV risk in Indian MSM: a multicity, randomised, clinical efficacy trial

BackgroundMen who have sex with men (MSM) in India are extremely marginalised and stigmatised, and therefore experience immense psychosocial stress. As current HIV prevention interventions in India do not address mental health or resilience to these stressors, we aimed to evaluate a resilience-based psychosocial intervention in the context of HIV and sexually transmitted infection (STI) prevention.MethodsWe did a multicity, randomised, clinical efficacy trial in Chennai (governmental tuberculosis research institute) and Mumbai (non-governmental organisation for MSM), India. Inclusion criteria were MSM, aged 18 years or older, who were at risk of HIV acquisition or transmission, defined as having any of the following in the 4 months before screening: anal sex with four or more male partners (protected or unprotected), diagnosis of an STI, history of transactional sex activity, or condomless anal sex with a man who was of unknown HIV status or serodiscordant. Participants were required to speak English, Tamil (in Chennai), or Hindi (in Mumbai) fluently. Eligible individuals were randomly assigned (1:1) to either a resilience-based psychosocial HIV prevention intervention, consisting of group (four sessions) and individual (six sessions) counselling alongside HIV and STI voluntary counselling and testing, or a standard-of-care control comprising voluntary counselling and testing alone. The primary outcomes were number of condomless anal sex acts with male partners during the past month (at baseline and 4 months, 8 months, and 12 months after randomisation), and incident bacterial STIs (at 12 months after randomisation). Resilience-related mediators included self-esteem, self-acceptance, and depression. Recruitment is now closed. This trial is registered with ClinicalTrials.gov, NCT02556294.FindingsBetween Sept 4, 2015, and June 28, 2018, we enrolled 608 participants; 305 (50%) were assigned to the psychosocial intervention condition and 303 (50%) were assigned to the control condition. 510 (84%) of 608 men completed an assessment at 4 months after randomisation, 483 (79%) at 8 months, and 515 (85%) at 12 months. 512 (99%) of 515 men had STI data from the 12-month assessment. The intervention condition had a 56% larger reduction in condomless anal sex acts (95% CI 35-71; p&lt;0·0001) from baseline to 4-month follow-up, 72% larger reduction (56-82; p&lt;0·0001) from baseline to 8-month follow-up, and 72% larger reduction (53-83; p&lt;0·0001) from baseline to 12-month follow-up, compared with the standard-of-care control condition (condition by time interaction; χ2=40·29, 3 df; p&lt;0·0001). Improvements in self-esteem and depressive symptoms both mediated 9% of the intervention effect on condomless anal sex acts. Bacterial STI incidence did not differ between study conditions at 12-month follow-up.InterpretationA resilience-based psychosocial intervention for MSM at risk of HIV acquisition or transmission in India was efficacious in reducing condomless anal sex acts, with evidence for mediation effects in two key target resilience variables. HIV prevention programmes for MSM in India should address mental health resilience to augment reductions in the risk of sexually transmitted HIV.FundingNational Institute of Mental Health

Strengthening resilience to reduce HIV risk in Indian MSM: a multicity, randomised, clinical efficacy trial

Men who have sex with men (MSM) in India are extremely marginalised and stigmatised, and therefore experience immense psychosocial stress. As current HIV prevention interventions in India do not address mental health or resilience to these stressors, we aimed to evaluate a resilience-based psychosocial intervention in the context of HIV and sexually transmitted infection (STI) prevention. We did a multicity, randomised, clinical efficacy trial in Chennai (governmental tuberculosis research institute) and Mumbai (non-governmental organisation for MSM), India. Inclusion criteria were MSM, aged 18 years or older, who were at risk of HIV acquisition or transmission, defined as having any of the following in the 4 months before screening: anal sex with four or more male partners (protected or unprotected), diagnosis of an STI, history of transactional sex activity, or condomless anal sex with a man who was of unknown HIV status or serodiscordant. Participants were required to speak English, Tamil (in Chennai), or Hindi (in Mumbai) fluently. Eligible individuals were randomly assigned (1:1) to either a resilience-based psychosocial HIV prevention intervention, consisting of group (four sessions) and individual (six sessions) counselling alongside HIV and STI voluntary counselling and testing, or a standard-of-care control comprising voluntary counselling and testing alone. The primary outcomes were number of condomless anal sex acts with male partners during the past month (at baseline and 4 months, 8 months, and 12 months after randomisation), and incident bacterial STIs (at 12 months after randomisation). Resilience-related mediators included self-esteem, self-acceptance, and depression. Recruitment is now closed. This trial is registered with ClinicalTrials.gov, NCT02556294. Between Sept 4, 2015, and June 28, 2018, we enrolled 608 participants; 305 (50%) were assigned to the psychosocial intervention condition and 303 (50%) were assigned to the control condition. 510 (84%) of 608 men completed an assessment at 4 months after randomisation, 483 (79%) at 8 months, and 515 (85%) at 12 months. 512 (99%) of 515 men had STI data from the 12-month assessment. The intervention condition had a 56% larger reduction in condomless anal sex acts (95% CI 35-71; p<0·0001) from baseline to 4-month follow-up, 72% larger reduction (56-82; p<0·0001) from baseline to 8-month follow-up, and 72% larger reduction (53-83; p<0·0001) from baseline to 12-month follow-up, compared with the standard-of-care control condition (condition by time interaction; χ =40·29, 3 df; p<0·0001). Improvements in self-esteem and depressive symptoms both mediated 9% of the intervention effect on condomless anal sex acts. Bacterial STI incidence did not differ between study conditions at 12-month follow-up. A resilience-based psychosocial intervention for MSM at risk of HIV acquisition or transmission in India was efficacious in reducing condomless anal sex acts, with evidence for mediation effects in two key target resilience variables. HIV prevention programmes for MSM in India should address mental health resilience to augment reductions in the risk of sexually transmitted HIV. National Institute of Mental Health

The cost-effectiveness of a resilience-based psychosocial intervention for HIV prevention among MSM in India

OBJECTIVE: MSM in India are at a high risk for HIV infection given psychosocial challenges, sexual orientation stress, and stigma. We examined the cost-effectiveness of a novel resilience-based psychosocial intervention for MSM in India. DESIGN: We parameterized a validated microsimulation model (CEPAC) with India-specific data and results from a randomized trial and examined two strategies for MSM: status quo HIV care ( SQ ), and a trial-based psychosocial intervention ( INT ) focused on building resilience to stress, improving mental health, and reducing condomless anal sex (CAS). METHODS: We projected lifetime clinical and economic outcomes for MSM without HIV initially. Intervention effectiveness, defined as reduction in self-reported CAS, was estimated at 38%; cost was $49.37/participant. We used a willingness-to-pay threshold of US$2100 (2019 Indian per capita GDP) per year of life saved (YLS) to define cost-effectiveness. We also assessed the 5-year budget impact of offering this intervention to 20% of Indian MSM. RESULTS: Model projections showed the intervention would avert 2940 HIV infections among MSM over 10 years. Over a lifetime horizon, the intervention was cost-effective (ICER = $900/YLS). Results were most sensitive to intervention effectiveness and cost; the intervention remained cost-effective under plausible ranges of these parameters. Offering this intervention in the public sector would require an additional US$28 M over 5 years compared with SQ . CONCLUSION: A resilience-based psychosocial intervention integrated with HIV risk reduction counseling among MSM in India would reduce HIV infections and be cost-effective. Programs using this approach should be expanded as a part of comprehensive HIV prevention in India

The cost-effectiveness of a resilience-based psychosocial intervention for HIV prevention among MSM in India.

OBJECTIVE: MSM in India are at a high risk for HIV infection given psychosocial challenges, sexual orientation stress, and stigma. We examined the cost-effectiveness of a novel resilience-based psychosocial intervention for MSM in India. DESIGN: We parameterized a validated microsimulation model (CEPAC) with India-specific data and results from a randomized trial and examined two strategies for MSM: status quo HIV care ( SQ ), and a trial-based psychosocial intervention ( INT ) focused on building resilience to stress, improving mental health, and reducing condomless anal sex (CAS). METHODS: We projected lifetime clinical and economic outcomes for MSM without HIV initially. Intervention effectiveness, defined as reduction in self-reported CAS, was estimated at 38%; cost was $49.37/participant. We used a willingness-to-pay threshold of US$2100 (2019 Indian per capita GDP) per year of life saved (YLS) to define cost-effectiveness. We also assessed the 5-year budget impact of offering this intervention to 20% of Indian MSM. RESULTS: Model projections showed the intervention would avert 2940 HIV infections among MSM over 10 years. Over a lifetime horizon, the intervention was cost-effective (ICER = $900/YLS). Results were most sensitive to intervention effectiveness and cost; the intervention remained cost-effective under plausible ranges of these parameters. Offering this intervention in the public sector would require an additional US$28 M over 5 years compared with SQ . CONCLUSION: A resilience-based psychosocial intervention integrated with HIV risk reduction counseling among MSM in India would reduce HIV infections and be cost-effective. Programs using this approach should be expanded as a part of comprehensive HIV prevention in India