Chun, Jon

Elkins, Katherine

Peterson, Grace

English

Kenyon College: Digital Kenyon - Research, Scholarship, and Creative Exchange

Digital Kenyon: Research, Scholarship, and Creative Exchange IPHS 300: Artificial Intelligence for the Humanities: Text, Image, and Sound Digital Humanities Fall 2019 Guns in D.C. Appellate Court: Sentiment Analysis on Opinions from the Court of Appeals for the D.C. Circuit Grace Peterson Kenyon College, peterson3@kenyon.edu Jon Chun Kenyon College, chunj@kenyon.edu Katherine Elkins Kenyon College, elkinsk@kenyon.edu Follow this and additional works at: https://digital.kenyon.edu/dh_iphs_ai  Part of the Digital Humanities Commons Recommended Citation Peterson, Grace; Chun, Jon; and Elkins, Katherine, "Guns in D.C. Appellate Court: Sentiment Analysis on Opinions from the Court of Appeals for the D.C. Circuit" (2019). IPHS 300: Artificial Intelligence for the Humanities: Text, Image, and Sound. Paper 20. https://digital.kenyon.edu/dh_iphs_ai/20 This Article is brought to you for free and open access by the Digital Humanities at Digital Kenyon: Research, Scholarship, and Creative Exchange. It has been accepted for inclusion in IPHS 300: Artificial Intelligence for the Humanities: Text, Image, and Sound by an authorized administrator of Digital Kenyon: Research, Scholarship, and Creative Exchange. For more information, please contact noltj@kenyon.edu. Guns in D.C. Appellate Court:Sentiment Analysis on Opinions from the Court of Appeals for the D.C. CircuitGrace Peterson, Jon Chun, Katherine ElkinsKenyon College, Integrated Program in Humane Studies“Crime in Washington, D.C.” Wikipedia, Wikimedia Foundation, 17 Nov. 2019, https://en.wikipedia.org/wiki/Crime_in_Washington,_D.C.#cite_note-49.Dale, Robert. “Law and Word Order: NLP in Legal Tech.” Medium, Towards Data Science, 31 July 2019, https://towardsdatascience.com/law-and-word-order-nlp-in-legal-tech-bd14257ebd06.“Firearms Control Regulations Act of 1975.” Wikipedia, Wikimedia Foundation, 6 Nov. 2019, https://en.wikipedia.org/wiki/Firearms_Control_Regulations_Act_of_1975.Free Law Project. “Facts About PACER and CM/ECF.” Free Law Project, https://free.law/pacer-facts/.Hutto, C. J. “VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text.” Eighth International Conference on Weblogs and Social Media (ICWSM-14), 2014.“Ideological Leanings of United States Supreme Court Justices.” Wikipedia, Wikimedia Foundation, 16 Dec. 2019, https://en.wikipedia.org/wiki/Ideological_leanings_of_United_States_Supreme_Court_justices#Ideological_leanings_over_time.King, Colbert. “Gun Violence in D.C. Is Treated as a Normal Part of Life. That Needs to Change.” The Washington Post, WP Company, 23 Aug. 2019, https://www.washingtonpost.com/opinions/gun-violence-in-dc-is-treated-as-a-normal-part-of-life-that-needs-to-change/2019/08/23/ac30d87c-c4f5-11e9-b72f-b31dfaa77212_story.html.“Most Liberal States 2019.” World Population Review, 2019, http://worldpopulationreview.com/states/most-liberal-states/.“Political Polarization in the American Public.” Pew Research Center for the People and the Press, 11 Oct. 2016, https://www.people-press.org/2014/06/12/political-polarization-in-the-american-public/.Tashea, Jason. “Want to Improve AI for Law? Let's Talk about Public Data and Collaboration.” ABA Journal, 22 May 2018, http://www.abajournal.com/lawscribbler/article/to_improve_machine_learning_for_law_we_need_public_data_teamwork.ReferencesThe lack of obvious fluctuation in sentiment demonstrates almost no emotional bias towards or against guns in the Court of Appeals for the D.C. Circuit. This could potentially be a result of the standardization of tone in legal writing and the encouragement of reason and objectivity. Bias may be present in court opinions, but sentiment analysis reveals little on the matter. Implicit judicial bias may be more successfully revealed using other methods of NLP (Natural Language Processing).ReflectionResearch on judicial bias in the appellate courts currently exists using methods other than sentiment analysis. Machine learning methods such as word clustering and analysis of stylistic features such as punctuation and article usage may provide a better understanding of how guns, and other topics, are viewed in appellate court. With SCOTUS hearing the first case concerning the 2nd Amendment in 9 years, it would also be illuminating to conduct analysis on SCOTUS cases handling questions regarding the interpretation of the 2nd Amendment, potentially comparing the results to data on ideological leanings of Supreme Court Justices, plotted here:Additionally, one major issue I stumbled across in my research process is the lack of access to opinions at the Federal District Court level. Discovering an efficient and low-cost means of obtaining District Court opinions would open up the opportunity to apply machine learning methods that could reveal information about potential biases within regions or individual judges that could impact legal strategy in courts the public interacts with much more often, although there are a number to ethical issues regarding access to information with this that are highly debated within the field of legal analysis. Future ApplicationsWhat can VADER sentiment analysis opinions produced by the Court of Appeals for the D.C. Circuit containing the word “gun” reveal about judicial sentiment towards gun crime between the years of 1935 and 2016?Research QuestionAdd your information, graphs and images to this section.IntroductionU.S. in the 90s. Crime rates began to fall in the mid-90s as the crack epidemic gave way to higher incarceration rates and economic revitalization, and the crime rates dropped to their lowest by the mid-2000s (Wikipedia). The following chart shows the D.C. homicide rate from 1960 to 2015 in blue in comparison to the U.S. homicide rate in red:D.C. currently has strict gun laws, but illegal guns remain an issue, as over 1,000 illegal firearms have been recovered this year. The homicide rate is also climbing daily and is set to surpass last year’s rate (“Gun Violence in D.C.”). In light of Washington D.C.’s history of homicide and gun violence, this analysis aims to uncover a deeper understanding of how the city’s appellate court has viewed guns throughout the city’s history. I chose to conduct sentiment analysis on D.C. appellate court opinions from 1935 to the present because the files were available in JSON format through Court Listener. I originally intended to conduct my analysis on Federal District Court opinions, but the language in District Court opinions is far more straight forward because the document does not describe the judge’s view on the ruling, which would not be conducive to sentiment analysis. Additionally, access to opinions from lower courts is a significant roadblock in Legal Tech, as they are much more challenging to gain access to in an efficient manner, unlike SCOTUS opinions, which are widely used in machine learning applications. I also intended to select the circuits with the three most liberal and the three most conservative states to compare different appellate courts’ sentiments toward guns. However, due to the extensive nature of the dataset, I focused my analysis on the D.C. appellate court as a starting point. I refined the dataset further by selecting all unique opinions from the Court of Appeals for the D.C. Circuit containing the word “guns.” 2,800 unique opinion files were used in the sentiment analysis, although some were eliminated from the dataset due to issues in retrieving their content. The date, author, and citation were pulled from the caption, citation, and author section found at the beginning of each opinion. The facts of the case and the legal reasoning used by the three-judge panel to decide the verdict follows the first three sections. After cleaning the opinions, we ran the data set through the Syuzhet package in R using the VADER (Valence Aware Dictionary and sEntiment Reasoner) lexicon. The VADER lexicon is attuned to catch sentiment nuances such as negations, punctuation, and contractions. The overall sentiment of each document was calculated by averaging the positive, negative, and neutral sentiment scores. The emotional valance for each opinion in the dataset was plotted over time.MethodologyThe following graph shows the plotted sentiments of the data set, with the dotted line a trend line with a period of 20. The grey lines indicate the neutral sentiment scores, the yellow lines indicate the positive sentiment scores, and the purple lines indicate the negative sentiment scores. Timeline of Significant Gun Issues in the Courts:1975: Firearms Control Regulations Act restricted city residents from acquiring handguns2007: D.C. Circuit Court of Appeals found the Firearms Control Regulations Act of 1975 unconstitutional 2008: D.C. v Heller- Supreme Court deemed Firearms Control Regulations Act. Of 1975 unconstitutionalOctober 2011: Heller II- upheld D.C.'s registration requirements and ban on assault weapons/high capacity magazinesSeptember 2015: Heller III- the case returned to circuit court where some requirements were invalidatedJuly 2014: Palmer v D.C.- ban on open and concealed carry was struck down as unconstitutionalJuly 2017: Wrenn v D.C. + Grace v D.C.- appeared in court of appeals, invalidated "good reason" argument for qualifying for a concealed carry permitResultsApplying artificial intelligence to legal writing is not a new process, as systems for searching legal writing online were created as early as the 1960s. However, the multitude of advances in machine learning over the past few years has opened up a wide variety of avenues for analysis, including sentiment analysis. Robert Dale, the Principal Consultant at the Language Technology Group, has identified five main areas where Natural Language Processing is currently being applied: legal research, electronic discovery, contract review, document automation, and legal advice. One of the most substantial barriers to applying machine learning to the legal system is access to datasets. Many online resources for access to public legal documents require a subscription and/or are constructed so that it is nearly impossible to gather the dataset need without infinite time and money. It would cost around one billion dollars to download all of PACER, the Federal Court System’s document portal, according to the Free Law Project’s calculations. Even with unlimited monetary resources, many of the websites that do provide access to court documents require arduous and time-consuming methods to retrieve the dataset. Datasets containing opinions from the SCOTUS or Appellate Courts are easy to access, but obtaining large amounts of opinions from lower courts such as Federal District Courts remains nearly impossible. Considering that most Americans will never interact with the high levels of the US Court System, the impenetrability of the lower court’s document databases raises questions of who has access to justice. Applying research questions to the Appellate Courts is a first step in identifying trends and biases in the court system.Background on the State of Legal TechWith the Supreme Court of the United States hearing its first case related to the 2nd Amendment in nearly ten years and the explosion of mass shootings in the news, gun rights are a particularly relevant topic in 2019. America has also become increasingly polarized in recent years. Republicans and Democrats are “more divided along ideological lines – and partisan antipathy is deeper and more extensive – than at any point in the last two decades,” according to Pew Research Center. World Population Review has identified Washington D.C. as the second most liberal city in the country, making Court of Appeals for the D.C. Circuit a particularly exciting place to start an analysis of judicial sentiment toward guns. Washington, D.C. also has a tumultuous history of violent crime. Population loss peaked in the 1980s when D.C. had lost nearly a quarter of its population. This resulted in the isolation of the affluent areas to the west and more impoverished, crime-riddled areas to the east. The nationwide crack epidemic of the 80s and 90s caused crime rates to skyrocket, and the city was deemed the “murder capital” of theIntroductionOverall the results showed a high level of neutral sentiment with very little fluctuation over time. Although the data is not distributed evenly over the span of 1935 to 2016, the steady nature of the sentiment plots suggests that the graph would look similar were an even distribution of opinions related to guns over the time period available. The lack of fluctuation and high level of neutrality appears almost inhuman, which may indicate that the conventions of opinion writing in the D.C. appellate court is highly standardized and does not leave much room for sentimental language. Some variation in the graph does exist, particularly in 1998,1999, and 2007 where there are small rises in the valance of negative sentiment. The years 1998 and 1999 do fall at the end of the crack epidemic and a lower homicide rate after the peak in 1990. However, the fluctuations are too small to attribute their cause to any major events confidently. In the overall graph there is a sight upward trend in negative sentiment, which may suggest that the court has become slightly more opposed to gun violence over time.Conclusion

Guns in D.C. Appellate Court: Sentiment Analysis on Opinions from the Court of Appeals for the D.C. Circuit

https://digital.kenyon.edu/cgi/viewcontent.cgi?article=1014&amp;context=dh_iphs_ai

Guns in D.C. Appellate Court: Sentiment Analysis on Opinions from the Court of Appeals for the D.C. Circuit

Abstract

Similar works

Full text

Available Versions

Kenyon College: Digital Kenyon - Research, Scholarship, and Creative Exchange