Is Gender Really More Important than Appointing Prime Minister?

Professors Jame Stribopoulos and Moin Yahya recently published an article in the Osgoode Hall Law Journal entitled, Does a Judge's Party of Appointment or Gender Matter to Case Outcomes? An Empirical Study of the Court of Appeal for Ontario.  The abstract explains:

This study reveals that at least in certain categories of cases, both party of appointment and gender are statistically significant in explaining case outcomes.  Between these two variables, gender actually appears to be the stronger determinant of outcome in certain types of cases.  While these findings are cause for concern, this study also points toward a simple solution.  Diversity in the composition of appeal panels both from the standpoint of gender and party of appointment dampened the statistical influence of either variable.  In other words, in the case of gender, a single judge on a panel who is of the opposite sex from the others, or in the case of political party, a single judge appointed by a different political party, is sufficient to eliminate the potential distorting influence of either variable.  This finding suggests a need to reform how appeal panels are currently assembled in order to ensure political and gender diversity and minimize concerns about the potential for bias.

The methodology adopted by the authors attempted to isolate the influence on appeal outcomes of the gender and party of appointing Prime Minister composition of panels sitting on each case.  Their dataset included all reported judgments of the Court of Appeal from 1990 to 2003, a total of 4,906 cases.  The authors found that in several types of appeals that gender and party of appointment appear to have a significant effect on whether an appeal is likely to succeed.

One of the unanswered questions arising out of this study is the degree of variation in the policy preferences of the individual justices.  A methodology that is well suited to estimating these preferences is one developed by two American political scientists, Andrew Martin and Kevin Quinn.  It involves an item response theory model that is estimated using a Markov Chain Monte Carlo approach.  Stribopoulos and Yahya were kind enough to share their data so that I could estimate the policy preferences of the 40 justices who served on the Court of Appeal during this period with this different approach. 

Here is a summary of the findings:

Judicial Traits	Average Ideal Point
Conservative appointee (n = 18)	-0.131
Liberal appointee (n = 20)	0.237
Male (n = 31)	0.029
Female (n = 9)	0.263
Male Conservative appointee (n = 14)	-0.249
Male Liberal appointee (n = 15)	0.233
Female Conservative appointee (n = 4)	0.280
Female Liberal appointee (n = 5)	0.249

There are a number of interesting aspects to these results.  First, the gap between the average ideal points of Conservative and Liberal appointees is 0.368.  The gap between male and female judges is considerably smaller, at 0.234.  In other words, the gap in average ideal point differences between Conservative and Liberal appointees is about 50% larger than the gender gap.

Second, the gap for party of appointment is driven entirely by male justices.  The gap between average ideal points of justices who are male and Conservative appointees and those who are male and Liberal appointees is much larger than the gap including both genders at 0.482.  For females, the gap is of the opposite direction and is a negligible -0.031.  In other words, party of appointment appears to have a polarizing effect only on male judges and not at all on female judges.

There are several weaknesses of the Martin-Quinn method in this context that I should briefly acknowledge.  One is that the method looks only to cases in which there is a dissent.  The Court of Appeal has an incredible rate of unanimity in its judgments--around 95%--meaning that there were only 277 appeals to analyze.  The second weakness is that there is no control for different types of cases.  Contracts cases are lumped in with criminal and constitutional cases with reckless abandon.  Third, there is no way to control the fact that the cases were heard over a number of years.  The method treats them as if they were decided contemporaneously.  Finally, because the Court of Appeal is ostensibly bound by judgments of the Supreme Court of Canada, the judges may not feel as if they have complete freedom to vote consistently with their policy preferences.

In light of all these weaknesses, the point of this analysis is not  to cast doubt on the results reached by Stribopoulos and Yahya.  Instead the goal is to point out the benefits of deploying multiple methodologies on the same data to check the robustness of the results and to rule out alternative explanations for the results reached.  The curious result that party of appointment seems only to matter for males on the Court of Appeal during this period is an interesting one that ought to be examined further.  The analysis here cannot be conclusive because there were just four female judges appointed by Liberals and five female judges appointed by Conservatives yielding this result, which cannot really be considered to be generalizable due to the small numbers involved.

Additional insights may well flow from an examination of the full results for each of the individual judges, which are reproduced below for those who are interested.  I invite comments on what can be made of them.

	Mean	SD	Gender	Appointing PM
Doherty	-2.823	0.658	Male	Conservative
Feldman	-1.518	1.442	Female	Liberal
Abella	-1.509	1.278	Female	Conservative
Lacourciere	-1.078	1.107	Male	Conservative
Carthy	-0.961	1.019	Male	Conservative
Morden	-0.796	0.761	Male	Conservative
Krever	-0.626	0.824	Male	Conservative
Sharpe	-0.583	0.694	Male	Liberal
Moldaver	-0.580	0.619	Male	Liberal
Tarnopolsky	-0.563	0.728	Male	Liberal
Finlayson	-0.457	1.525	Male	Conservative
Zuber	-0.441	0.936	Male
Laskin	-0.420	0.613	Male	Liberal
Armstrong	-0.363	0.754	Male	Liberal
Catzman	-0.340	0.438	Male	Conservative
Rosenberg	-0.283	0.636	Male	Liberal
Cronk	-0.239	1.098	Female	Conservative
Goudge	-0.209	1.111	Male	Liberal
Griffiths	-0.188	0.765	Male	Conservative
Labrosse	-0.112	0.480	Male	Conservative
Cumming	-0.058	1.175	Male
Austin	-0.046	0.623	Male	Conservative
Arbour	0.215	0.737	Female	Liberal
Grange	0.261	0.917	Male	Liberal
Dubin	0.268	0.846	Male	Liberal
Houlden	0.302	1.082	Male	Liberal
Robins	0.448	0.654	Male	Conservative
O'Connor	0.586	0.869	Male	Liberal
Gillese	0.635	0.874	Female	Liberal
Osborne	0.653	0.424	Male	Conservative
Brooke	0.702	0.977	Male	Conservative
McKinlay	0.885	0.712	Female	Liberal
Charron	0.956	0.589	Female	Conservative
Blair	0.966	0.965	Male	Liberal
Borins	1.016	1.365	Male	Liberal
Simmons	1.030	0.699	Female	Liberal
McMurtry	1.230	0.804	Male	Liberal
Goodman	1.418	0.783	Male
MacPherson	1.867	0.668	Male	Liberal
Weiler	1.912	0.923	Female	Conservative
Galligan	2.139	1.402	Male	Conservative