Department of Computer Science, Institute of Mathematics and Statistics, University of Sao Paulo, São Paulo, Brazil, e-mails: jackson@ime.usp.br, mfinger@ime.usp.br"/> Department of Philosophy and Legal Theory, University of Sao Paulo Law School, São Paulo, Brazil, e-mails: jorge.alberto@ufma.br, julianomaranhao@usp.br"/>
Search
  • All Journals
Advanced Search
Search articles by Title, Author, Keywords, DOI
Submit Manuscript
7.7CiteScore
2.0Impact Factor
Advanced Search
MAP  >  The Knowledge Engineering Review
2023 Volume 38
Article Contents
Next Previous
RESEARCH ARTICLE   Open Access    

Selecting and ranking leading cases in Brazilian Supreme Court decisions

More Information
  • Abstract: This work studies quantitative measures for ranking judicial decisions by the Brazilian Supreme Court [Supremo Tribunal Federal (STF)] and selecting leading cases, which are understood as those with broadness of influence on different legal fields. The measures are based on a network built over decisions whose cases were finalized in the Brazilian Supreme Court between 01/2001 and 12/2019, which were obtained by crawling publicly available STF records. Three ranking measures are proposed; two are adaptations of the PageRank algorithm, and one adapts Kleinberg’s algorithm. Such measures are compared with respect to agreement on top 100 rankings; we also analyze each robustness measure based on self-agreement under perturbation.We examine whether the resulting quantitative ranking is congenial to a qualitative intuition of what the legal community usually considers as relevant precedents. We also discuss some possible criteria of relevance in the seek for patterns that suggest how quantitative and qualitative measures would better align. The ranking of leading cases and relevant decisions improved after building decision networks without irrelevant appeals and decisions that overflow the court offers a starting point to discuss the role of STF in the Brazilian judicial system.In our last work, both versions of PageRank and Kleinberg algorithms produced different rankings and all of them were robust with respect to 10% and 20%-perturbation levels, but none of them retrieved leading cases at the top of these rankings. Then, we took a further step in the studies of the STF decision network and we introduced better filtering of network nodes guided by legal expertise on the works of the Supreme Court. We also introduced more fine-grained perturbance levels to understand the impact of such filters in the STF decision network. We concluded that after filtering low-relevance decision types, the STF decision network is still robust under 10%-perturbation, but it presents higher degradation by increasing perturbation levels. The two versions of PageRank still produce different rankings. Kleinberg’s algorithm provides a different ranking, with many relevant criminal cases. Although we improved algorithms rankings filtering decisions from the network, which represents an important methodological step, there is still room for improvement. Given that relevant decisions are well ranked after filtering out a large amount of irrelevant decisions, the results set a starting point to discuss the role of STF in the Brazilian judicial system.
  • 加载中
  • Agnoloni , T. & Pagallo , U. 2015. The case law of the italian constitutional court, its power laws, and the Web of scholarly opinions. (Proceedings of the 15th International Conference on Artificial Intelligence and Law, Association for Computing Machinery, New York, NY, USA, pp. 151–155. URL: http://doi.acm.org/10.1145/2746090.2746108

    Google Scholar

    Albert , R., Jeong , H. & Barabási , A.-L. 2000. Error and attack tolerance of complex networks. Nature 406(6794), 378–382. DOI 10.1038/35019019. doi: 10.1038/35019019

    CrossRef   Google Scholar

    Almeida , G., Nunes , J. L. & Chaves , L. 2016. Explicando o dissenso: Uma análise empírica do comportamento judicial do supremo tribunal federal e da suprema corte dos estados unidos (Explaining dissent. In An Empirical Analysis of Judicial Behaviour of the Brazilian Supreme Court and Supreme Court of the United States), Available at SSRN: https://ssrn.com/abstract=2580904, or 10.2139/ssrn.2580904.

    Google Scholar

    Barabási , A.-L. & Pósfai , M. 2016. Network science. Cambridge.: Cambridge University Press, URL: Available at: http://barabasi.com/networksciencebook/.

    Google Scholar

    Bommarito , M. J., II, Katz , D. M., Zelner , J. L. & Fowler , J. H. 2010. Distance measures for dynamic citation networks. Physica A: Statistical Mechanics and Its Applications 389(19), 4201–4208. URL: https://www.sciencedirect.com/science/article/pii/S0378437110004930.

    Google Scholar

    Brin , S. & Page , L. 1998. The anatomy of a large-scale hypertextual Web search engine. Comput Netw. ISDN Syst 30(1-7), 107–117. DOI 10.1016/S0169-7552(98)00110-X. doi: 10.1016/S0169-7552(98)00110-X

    CrossRef   Google Scholar

    Clauset , A., Shalizi , C. R. & Newman , M. E. J. 2009. Power-law distributions in empirical data. SIAM Review 51(4), 661–703. DOI 10.1137%2F070710111. doi: 10.1137%2F070710111

    CrossRef   Google Scholar

    Cushman , C. 2011. Courtwatchers: Eyewitness Accounts in Supreme Court History. Rowman & Littlefield Publishers.

    Google Scholar

    de Souza , J. J. & Finger , M. 2020. Robust ranking of brazilian supreme court decisions. In Intelligent Systems. BRACIS 2020. Lecture Notes in Computer Science, 12319, Cerri , R., & Prati , R. C., (eds). Springer International Publishing, Cham, Switzerland, 581–594. URL: DOI 10.1007/978-3-030-61377-8_40.

    Google Scholar

    Dworkin , R. 1986. Law’s Empire. Harvard University Press, 45–86, chapter 2.

    Google Scholar

    Falcão , J., Cerdeira , P. & Arguelhes , D. 2013. I relatório do supremo em números-o múltiplo supremo. Revista de Direito Administrativo 262, 399–452.

    Google Scholar

    Falcão , J., Hartmann , I. A. & Chaves , V. P. 2014. III relatório supremo em números: O supremo e o tempo. Available at: URL: http://hdl.handle.net/10438/12055.

    Google Scholar

    Federal , Supremo Tribunal 2022. Pesquisa de jurisprudência - STF. Accessed 17-August-2022. Available at: URL: https://jurisprudencia.stf.jus.br/pages/search.

    Google Scholar

    Fowler , J. H. & Jeon , S. 2008. The authority of supreme court precedent. Social Networks 30(1), 16–30. URL: https://www.sciencedirect.com/science/article/pii/S0378873307000378.

    Google Scholar

    Green , E. 2020. No one likes amy coney barretts abortion answer. The Atlantic, 2020-10-13. URL: https://www.theatlantic.com/politics/archive/2020/10/amy-coney-barrett-roe-v-wade/616702/.

    Google Scholar

    Hagberg , A. A., Schult , D. A. & Swart , P. J. 2008. Exploring network structure, dynamics, and function using networkx. Proceedings of the 7th Python in Science Conference, Pasadena, CA USA, pp. 11–15, URL: http://conference.scipy.org/proceedings/SciPy2008/paper_2/.

    Google Scholar

    Hartmann , I. A., dos Santos Junior , F. A. C., Silva , F. A., Appel , O., et al. 2017. Pedidos de vista no tribunal superior eleitoral. REI - Revista Estudos Institucionais 3(2), 1074–1111. DOI 10.21783/rei.v3i2.180. doi: 10.21783/rei.v3i2.180

    CrossRef   Google Scholar

    Irons , P. 2002. Jim Crow’s Children: The Broken Promise of the Brown Decision. Penguin.

    Google Scholar

    Irons , P. 2006. A People’s History of the Supreme Court: The Men and Women Whose Cases and Decisions Have Shaped Our Constitution. Revised ed. Penguin.

    Google Scholar

    Klaus , A., Yu , S. & Plenz , D. 2011. Statistical analyses support power law distributions found in neuronal avalanches. PLOS ONE 6(5), 1–12. DOI 10.1371/journal.pone.0019779. doi: 10.1371/journal.pone.0019779

    CrossRef   Google Scholar

    Kleinberg , J. M. 1999. Authoritative sources in a hyperlinked environment. Journal of the ACM (JACM) 46(5), 604–632. DOI http://doi.acm.org/10.1145/324133.324140. doi: http://doi.acm.org/10.1145/324133.324140

    CrossRef   Google Scholar

    Mattiuzzo , L. 2018. Most-cited u.s. supreme court cases in heinonline: Part iii. Accessed on March 29 2021. Available at: https://home.heinonline.org/blog/2018/09/most-cited-u-s-supreme-court-cases-in-heinonline-part-iii/.

    Google Scholar

    Page , L., Brin , S., Motwani , R. & Winograd , T. 1999. The pageRank citation ranking: Bringing order to the Web. Technical Report 1999-66. Available at: http://ilpubs.stanford.edu. Stanford InfoLab. Previous number = SIDL-WP-1999-0120.

    Google Scholar

    Patterson , J. T. 2004. Legacies and lessons. In The Landmark School Desegregation Case in Retrospect, Supreme Court Historical Society, Cushman , C., Urofsky , M. I., eds , Black, hite , W., & rown , B., 297–318.

    Google Scholar

    Rehnquist , W. H. 2002. The Supreme Court. Vintage.

    Google Scholar

    Schneider , C. M., Moreira , A. A., Andrade , J. S., Havlin , S. & Herrmann , H. J. 2011, Mitigation of malicious Attacks on Networks, Proceedings of the National Academy of Sciences 108(10), 3838–3841. URL: https://www.pnas.org/doi/abs/10.1073/pnas.1009440108.

    Google Scholar

    Sedgewick , R. & Wayne , K. 2011. Algorithms. 4th ed. Addison-Wesley.

    Google Scholar

    Steinman , A. N. 2016. The rise and fall of plausibility pleading? Vanderbilt Law Review 69(2753142), 333–401. Available at SSRN: https://ssrn.com/abstract=2753142.

    Google Scholar

    van Opijnen , M. 2012. Citation analysis and beyond: In search of indicators measuring case law importance. In Legal Knowledge and Information Systems - JURIX 2012, 250, Schäfer , B., (eds). IOS Press BV, Amsterdam, The Netherlands, 95–104. DOI 10.3233/978-1-61499-167-0-95.

    Google Scholar

    Vojvodic , A.d M. 2012. Precedentes e argumentação no supremo tribunal federal: Entre a vinculação ao passado e a sinalização para o futuro, URL: PhD thesis. Universidade de São Paulo Law School, https://teses.usp.br/teses/disponiveis/2/2134/tde-27092012-094000/pt-br.php.

    Google Scholar

    Winkels , R., de Ruyter , J. & Kroese , H. 2011. Determining authority of dutch case law. In Legal Knowledge and Information Systems - JURIX 2011: The Twenty-Fourth Annual Conference, 235, Atkinson , K., (eds). University of Vienna, Austria, 103–112. Frontiers in Artificial Intelligence and Applications, IOS Press. DOI 10.3233/978-1-60750-981-3-103. URL: 2011-12.

    Google Scholar

  • Cite this article

    Jackson José De Souza, Marcelo Finger, Jorge Alberto A. de Araújo, Juliano Maranhão. 2023. Selecting and ranking leading cases in Brazilian Supreme Court decisions. The Knowledge Engineering Review 38(1), doi: 10.1017/S0269888923000073
    Jackson José De Souza, Marcelo Finger, Jorge Alberto A. de Araújo, Juliano Maranhão. 2023. Selecting and ranking leading cases in Brazilian Supreme Court decisions. The Knowledge Engineering Review 38(1), doi: 10.1017/S0269888923000073
    shu
Download PDF

Article Metrics

Article views(148) PDF downloads(100)

Other Articles By Authors

RESEARCH ARTICLE   Open Access    

Selecting and ranking leading cases in Brazilian Supreme Court decisions

  • Received: 07 April 2021
  • Revised: 01 April 2023
  • Accepted: 25 April 2023
  • Published online: 14 June 2023
The Knowledge Engineering Review  38,  2023, 38(1)  |  Cite this article

Abstract: Abstract: This work studies quantitative measures for ranking judicial decisions by the Brazilian Supreme Court [Supremo Tribunal Federal (STF)] and selecting leading cases, which are understood as those with broadness of influence on different legal fields. The measures are based on a network built over decisions whose cases were finalized in the Brazilian Supreme Court between 01/2001 and 12/2019, which were obtained by crawling publicly available STF records. Three ranking measures are proposed; two are adaptations of the PageRank algorithm, and one adapts Kleinberg’s algorithm. Such measures are compared with respect to agreement on top 100 rankings; we also analyze each robustness measure based on self-agreement under perturbation.We examine whether the resulting quantitative ranking is congenial to a qualitative intuition of what the legal community usually considers as relevant precedents. We also discuss some possible criteria of relevance in the seek for patterns that suggest how quantitative and qualitative measures would better align. The ranking of leading cases and relevant decisions improved after building decision networks without irrelevant appeals and decisions that overflow the court offers a starting point to discuss the role of STF in the Brazilian judicial system.In our last work, both versions of PageRank and Kleinberg algorithms produced different rankings and all of them were robust with respect to 10% and 20%-perturbation levels, but none of them retrieved leading cases at the top of these rankings. Then, we took a further step in the studies of the STF decision network and we introduced better filtering of network nodes guided by legal expertise on the works of the Supreme Court. We also introduced more fine-grained perturbance levels to understand the impact of such filters in the STF decision network. We concluded that after filtering low-relevance decision types, the STF decision network is still robust under 10%-perturbation, but it presents higher degradation by increasing perturbation levels. The two versions of PageRank still produce different rankings. Kleinberg’s algorithm provides a different ranking, with many relevant criminal cases. Although we improved algorithms rankings filtering decisions from the network, which represents an important methodological step, there is still room for improvement. Given that relevant decisions are well ranked after filtering out a large amount of irrelevant decisions, the results set a starting point to discuss the role of STF in the Brazilian judicial system.

    HTML

    Acknowledgments

    • We thank Mayara C. Melo and Alessandro Calò for developing previous work that made it possible to push this research forward; Dr Luís Matricardi for helping a lot in understanding concepts of law and properties of STF decisions and suggesting readings regarding these subjects; Felipe Farias and the STF jurisprudence sector staff for helping with questions about the data in case entries.

    • This work was carried out at the Center for Artificial Intelligence (C4AI-USP), with support from the São Paulo Research Foundation (FAPESP grant 2019/07665-4) and from the IBM Corporation. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brasil (CAPES)—Finance Code 001. M. Finger was partly supported by Fapesp, processes 20/06443-5 (SPIRA) and 14/12236-1 (Animals), and CNPq grant 303609/2018-4 (PQ).

    • The Judiciary Act of 1925, also known as ‘Certiorary Act’, was designed to reduce the workload of the Supreme Court of the United States (SCOTUS). Since then, any appellant must file petitions for a ‘writ of certiorary’ with the Court. The acceptance of the petition depends on the discretionary favorable decision of four Justices. Usually, the Court hears 100–150 cases each year.

    • https://portal.fgv.br/en

    • In Brazil, preliminary injunctions are petitions that have temporary and urgent features. It aims to avoid likely violations of constitutionally guaranteed rights or irreparable harm in the absence of preliminary relief caused by delay in the legal process.

    • 347 U.S. 483, cited by 23 395 articles as of 2018.

    • 410 U.S. 113, cited by 21 692 articles as of 2018.

    • Justice is how a Magistrate is called in the United States.

    • ADPF stands for ‘arguição de descumprimento de preceito fundamental’, which could be translated as a complaint of a violation of a fundamental right. A petition of this kind is presented directly to the STF.

    • Articles 124–128 of the Brazilian Criminal Code.

    • STF, ADPF 54, 2012.

    • For instance, the ADPF 132 was cited in 2019, when the Court ruled in the ADO 26 that discrimination by sexual or gender orientation should be regarded as criminal according to legislation that punished racism (ADO stands for ‘ação declaratória de omissão’, which could be translated as a motion for a declaration of legislative omission—in this case, the omission in creating a crime punishing discrimination by sexual or gender orientation).

    • The President of the Court (which would be the Chief Justice in the United States) is not a member of any of the two chambers, but leads deliberations when the whole Court convenes.

    • In Portuguese, it is called Instituto Nacional do Seguro Social (INSS), which would translate literally to National Institute of Social Security. We decided to use ‘office’ instead to better convey the nature of the institution as part of the Executive branch.

    • The source code for the scraper, the experiments run and the analysis of the results are available at https://github.com/jacksonjos/analise-juridica

    • As the removal of decisions is done before building the network, if a decision entry is removed, all similar decisions related to this decision are also removed.

    • For data about IPCA, the inflation index adopted in Brazil, look at: https://clubedospoupadores.com/ipca-anual

    • ‘ED, or ‘embargos de declaração’—which could be translated as a ‘clarification motion’—are appropriate when the decision has some ambiguity, obscurity or contradiction.

    Rights and permissions
    • © The Author(s), 2023. Published by Cambridge University Press2023Cambridge University Press
References (31)
  • About this article
    Cite this article
    Jackson José De Souza, Marcelo Finger, Jorge Alberto A. de Araújo, Juliano Maranhão. 2023. Selecting and ranking leading cases in Brazilian Supreme Court decisions. The Knowledge Engineering Review 38(1), doi: 10.1017/S0269888923000073
    Jackson José De Souza, Marcelo Finger, Jorge Alberto A. de Araújo, Juliano Maranhão. 2023. Selecting and ranking leading cases in Brazilian Supreme Court decisions. The Knowledge Engineering Review 38(1), doi: 10.1017/S0269888923000073
    shu

    • Catalog

      /

      DownLoad:  Full-Size Img  PowerPoint
      Return
      Return