New Macleans Ranking of Canadian Law Schools

The second annual Macleans ranking of Canadian law schools has been released.  Here are the rankings for the common law schools this year, as compared with last year:

Common   Law School	2008	2007	Change
Toronto	1	1	0
McGill	2	2	0
Osgoode	3	3	0
UBC	4	9	+5
Victoria	5	8	+3
Dalhousie	6	6	0
Ottawa	7	4	-3
Queen's	8	5	-3
Alberta	9	7	-2
Calgary	10	15	+5
Manitoba	10	11	+1
Western	12	12	0
UNB	12	12	0
Saskatchewan	12	10	-2
Windsor	15	14	-1
Moncton	16	16	0

There were some changes to the methodology used this year from last year, with the two biggest changes being (i) the addition of a new set of criteria relating to law professor positions secured by a school's LL.B. or JD graduates; and (ii) an expansion of the number of journals in which citations to faculty publications were counted.  Both of these changes ought to be regarded as significant improvements, in that they capture in new ways important, reliable, and publicly-verifiable information about a school's graduates and faculty scholarship, respectively. 

A problem common to each, however, is that improvements to a school's graduates or faculty will have an effect on each measure only with a dramatic lag.  Hiring strong young faculty, for example, will temporarily hurt a school's ranking in the per capita citation measure since young faculty will only begin being cited frequently a number of years after joining a faculty, but will immediately be added to the denominator.  A law dean who is acutely sensitive to this measure might be disinclined to hire promising young scholars and might prefer to try to poach scholars from other schools.  Hiring established scholars laterally from other schools would permit a much more immediate improvement on this measure.  Graduating stronger LL.B. and JD classes will also take some time to show up in the law professor counts.  It was a nice touch to include a per capita measure of both the total number of LL.B. and JD graduates of a school who are law professors, as well as the number of LL.B. and JD graduates of a school working as law professors at schools other than their alma mater (since it is well-known that schools tend to have a preference to hire their own graduates).  It may be unnecessary, however, since the correlation between the two measures is 0.96, which suggests that breaking the data out in this way doesn't add much information.  On the other hand, this blunts some of the incentive for schools to game the system by loading up on their own graduates, though I suspect that this would be unlikely in any event.  Perhaps in future years it would be better to count only LL.B. and JD grads hired by other schools as law professors, since this measure cannot be gamed directly (though a school would retain an indirect preference to hire one's own to preclude a rival from claiming an additional academic hire).

It seems there is still a problem with the elite firm hiring and national reach indicia.  It's unclear why the quotient is important, rather than the per capita number of graduates at more than the top three drawing elite firms.  Consider the following:

National Reach	Quotient	Quotient Rank	Per Capita	Per Capita Rank
Toronto	0.7332	5	1.5111	1
McGill	0.6757	8	1.1765	2
UBC	0.7028	6	1.0913	3
Alberta	0.7030	6	1.0625	4
Western	0.7467	2	1.0338	5
Calgary	0.6718	9	1.0057	6
Queen's	0.7430	3	0.9755	7
Osgoode	0.7425	3	0.9288	8
Victoria	0.7920	1	0.9124	9
Saskatchewan	0.5879	13	0.8151	10
Dalhousie	0.6179	12	0.8037	11
Manitoba	0.5522	14	0.7255	12
Ottawa	0.6692	9	0.6692	13
UNB	0.3133	15	0.5281	14
Windsor	0.6414	11	0.4593	15
Moncton	0.2500	16	0.1282	16

It seems to me, at least, that the per capita measure provides a more intuitive sense of the depth of influence of a school's graduates than the quotient measure, which measures the concentration of lawyers at the non-top three drawing elite firms versus the total number of associates at elite firms.  Better to get a sense of how broadly a school's graduates are placed at elite firms by using the per capita measure, i would have thought.  As it stands, the national reach data correlates somewhat poorly with the other measures of law school strength: with elite firm hiring (0.315), with SCC clerkships (0.091), with faculty citations (0.535), and with law academics (all) (0.188), and law academics (non-alma mater) (0.369).  If instead the per capita national reach measure was used, some of this weakness would evaporate, with correlations moving to: with elite firm hiring (0.821), with SCC clerkships (0.376), with faculty citations (0.632), and with law academics (all) (0.527), and law academics (non-alma mater) (0.661).  And this gain in consistency with the rest of the indicia does not come at a dramatic cost, actually; the correlation of the (in my view, flawed, quotient measure of national reach) with the per capita national reach measure is 0.754, which suggests a fairly strong relationship.  So the higher correlations with the other indicia is not being purchased at the cost of losing a ton of information from the quotient measure of national reach.  The difference is that the per capita national reach measure is not affected by the wonkiness of the quotient measure (which rewards not placing many students at elite firms, but placing students at many elite firms).  The per capita measure does not penalize as perversely placing an additional graduate at an elite firm that is a top three drawer of that school's graduates.  The reason is that it doesn't add a unit to the denominator (as in the quotient method), but instead is associated with the opportunity cost of not adding another unit to the numerator of the per capita measure.  I would have thought that this incentive was a better one to engender for career development offices, and that the per capita national reach measure more intuitively captures a law school's national reach.

Comments

[yakov_a]

Wouldn't the proposed measure grossly favor regional schools as opposed to a truly national metric? Especially a smaller school in a market with some very large firms requiring a large number of articling students? The measure would also favor pre-existing relationships, as opposed to newly established ones.

Shouldn't a top law school have a truly national reach? As such, it would seem per capita placement in the elite firms nationally would be a far better measure.

[Ben Alarie]

I agree that per capita placement is a better measure. That's the argument. If there's a sense that some law schools concentrate too many grads in a few large elite firms, one way to take that into account is to ignore the "top three" elite firms for each school, and compare the per capita figures for the remaining (i.e. non-top three) elite firms. An inferior way of taking it into account is to measure the proportion of elite firm grads that go to elite firms other than the top three. That's the point.

[yakov_a]

My apologies. I should have read the post more closely.

[yakov_a]

benalarie,

What are your thoughts on: http://ablawg.ca/2008/09/16/macleans-ranking-canadian-law-schools/#more-220), especially the portion of the class finding articles?

[Omar Ha-Redeye]

Hmmm.

They also say, "Lawyers are Rats."