Queer In AI: A Case Study in Community-Led Participatory AI

We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community's programs over the years. We discuss different challenges that emerged in the process, look at ways this organization has fallen short of operationalizing participatory and intersectional principles, and then assess the organization's impact. Queer in AI provides important lessons and insights for practitioners and theorists of participatory methods broadly through its rejection of hierarchy in favor of decentralization, success at building aid and programs by and for the queer community, and effort to change actors and institutions outside of the queer community. Finally, we theorize how communities like Queer in AI contribute to the participatory design in AI more broadly by fostering cultures of participation in AI, welcoming and empowering marginalized participants, critiquing poor or exploitative participatory practices, and bringing participation to institutions outside of individual research projects. Queer in AI's work serves as a case study of grassroots activism and participatory methods within AI, demonstrating the potential of community-led participatory methods and intersectional praxis, while also providing challenges, case studies, and nuanced insights to researchers developing and using participatory methods.Comment: To appear at FAccT 202

truera/trulens: TruLens Eval v0.13.0

Library containing evaluations of LLM Applications Changelog Updated all documentation to show context recorder usage Smoke Tests are tested with trulens eval Examples Examples are restructured for better discoverability Added a Milvus Vector DB Example Bug Fixes Removed metadata_fn in example

truera/trulens: TruLens Explain v0.16.0

<p>Library containing evaluations of LLM Applications</p> <p>Changelog</p> <ul> <li>[MLNN-1020] App runner UI updates by @walnutdust in https://github.com/truera/trulens/pull/503</li> <li>Fix groundedness aggregation flakiness + incorrect 0 resolution. by @joshreini1 in https://github.com/truera/trulens/pull/501</li> <li>[MLNN-1053] Add groundedness to Pinecone notebook by @ejisoo in https://github.com/truera/trulens/pull/506</li> <li>[MLNN-1046] Example app with TruLens by @daniel-huang-1230 in https://github.com/truera/trulens/pull/500</li> <li>[MLNN-1053] Update dependencies in pinecone notebook by @ejisoo in https://github.com/truera/trulens/pull/507</li> <li>threading robustness and feedback retrieval by @piotrm0 in https://github.com/truera/trulens/pull/480</li> <li>Merge generated docs and test files into main by @github-actions in https://github.com/truera/trulens/pull/508</li> <li>updating JSONPath and features by @piotrm0 in https://github.com/truera/trulens/pull/502</li> <li>handle no pii by @joshreini1 in https://github.com/truera/trulens/pull/504</li> <li>Update Instrumentation Overview page: fix link, add trucustom by @joshreini1 in https://github.com/truera/trulens/pull/505</li> <li>bugfixes by @piotrm0 in https://github.com/truera/trulens/pull/510</li> <li>dashboard appui quickstart by @piotrm0 in https://github.com/truera/trulens/pull/511</li> <li>Fix app UI showing the record twice by @walnutdust in https://github.com/truera/trulens/pull/513</li> <li>Release branch 0.16.0 by @daniel-huang-1230 in https://github.com/truera/trulens/pull/514</li> </ul> <p>Bug Fixes</p> <ul> <li>Fix App UI, links, icons</li> </ul> <h2>New Contributors</h2> <ul> <li>@ejisoo made their first contribution in https://github.com/truera/trulens/pull/506</li> <li>@daniel-huang-1230 made their first contribution in https://github.com/truera/trulens/pull/500</li> </ul> <p><strong>Full Changelog</strong>: https://github.com/truera/trulens/compare/trulens-eval-0.15.3...trulens-eval-0.16.0</p&gt

truera/trulens: TruLens Eval v0.17.0

<p>Changelog:</p> <ul> <li>Add criteria and improve chain of thought prompting for evals</li> <li>Allow feedback functions to be in different directions with appropriate coloring/emojis</li> <li>Filter leaderboard feedback function results to only those available for the given app id</li> <li>Add smoke testing/benchmarking for groundedness based on SummEval dataset</li> </ul> <p>Bug Fixes:</p> <ul> <li>Fix issue with LiteLLM provider</li> <li>Allow Groundedness to run with any LLM provider</li> <li>Examples</li> <li>Using Anthropic Claude to run feedback functions</li> </ul&gt

truera/trulens: TruLens-Eval-0.18.3

<p>Changelog:</p> <ul> <li>Migrate LiteLLM to v1 (https://github.com/truera/trulens/pull/644)</li> <li>Migrate AzureOpenAI to v1 (https://github.com/truera/trulens/pull/640)</li> <li>Expand groundedness smoke testing (https://github.com/truera/trulens/pull/634)</li> </ul> <p>Bug Fixes:</p> <ul> <li>Fix Langchain-based evals so that data is properly evaluated (https://github.com/truera/trulens/pull/636)</li> <li>Fix escape unicode on records table (https://github.com/truera/trulens/pull/632)</li> <li>Catch more endpoint exceptions when tracking costs (https://github.com/truera/trulens/pull/627)</li> <li>Add wrapper for dynamically generated functions in boto3 (https://github.com/truera/trulens/pull/626)</li> <li>Update AzureOpenAI example (https://github.com/truera/trulens/pull/647)</li> </ul&gt

truera/trulens: TruLens Eval v0.20.1

<h3>What's changed</h3> <ul> <li>Added instructions for installing from github by @piotrm0 in #740</li> <li>Updated instrumentation docs by @joshreini1 in #737</li> <li>Update structured extraction and chain of thought prompting for better results across providers by @joshreini1 in #732</li> </ul> <h3><strong>Examples</strong></h3> <ul> <li>Improve clarity of human feedback notebook by @piotrm0 in #746</li> </ul> <h3>Bug Fixes</h3> <ul> <li>Fixed the Langchain provider by @piotrm0 in #759</li> <li>Added missing Langchain provider docs by @joshreini1 in #760</li> <li>Converted structures to strings in feedback results table by @piotrm0 in #758</li> <li>Improve instrumentation of langchain apps by @piotrm0 in #757 and #755</li> <li>Fix subscripted generics typechecking for python < 3.10 by @coreyhu in #754</li> <li>Make Llama-Index completely optional by @piotrm0 in #745</li> <li>Bugfixes for WithClassInfo by @piotrm0 in #741</li> <li>Fixes for AzureOpenAI serialization by @piotrm0 in #735</li> <li>Pass prompt instead of messages to Bedrock by @joshreini1 in #732</li> </ul&gt

truera/trulens: TruLens-Eval-0.19.1

<p>Changelog:</p> <ul> <li>Remove cohere provider</li> </ul&gt