The Sisk-Leiter rankings of scholarly impact use a Westlaw search to determine a scholar’s citation count. However, the search does not review the citations to determine if they actually are citations to a scholar’s work rather than other hits such as blog posts (whether authored by the scholar or by another on the scholar’s eponymous blog), citations to works by others in books that the scholar edited, citations to the work of other scholars who only mention the scholar under study (such as a citation in a work to the work of a scholar that had reviewed the book of the scholar under study), media mentions, or author acknowledgements for comments not eliminated by the search term, such as those that appear in footnotes or the body of the article. I use citations to Brian Leiter’s work to show that the Sisk-Leiter Westlaw citation count is overstated by about 40% in Leiter’s case, with 398 of 557 attributed cites being to citations to Leiter’s academic work. While Leiter’s case may be more upwardly biased than others because of his popular industry blog, the fact is that media mentions, citations to the works of others in edited volumes, and citations to works that discuss the scholar under study but are not cites directly to that work have an unknown bias that cannot be assumed away easily. Moreover, by ignoring cites in judicial opinions to scholarly work, the method as applied by Sisk undervalues the impact of scholars in some fields of more practical importance and, therefore, likely the scholarly impact and ranking of faculties with strong scholars in antitrust, bankruptcy, corporate and securities law. A combination of Westlaw (with judicial citations) and Google Scholar would provide more reliable results.
