A Statistical Analysis of the Gender Gap by Ariel Shin

Introduction

My name is Ariel Shin and I am a Computer Science Major at UC Davis. I have been analyzing trends in debate rounds from the national high school debate circuit in R, a programming language utilized by data analysts and statisticians, with a computer science professor, Norm Matloff. I debated four years for Immaculate Heart and attended the Tournament of Champions my senior year.

For those uninterested in how I created these graphs in R, skip over the italicized parts.

Previous Research

The first step in debate and undergraduate research is reading the topic literature. The only form of debate rankings that currently exists is Kirsch’s ranking on vbriefly, which determines a debater’s ranking on number of rounds won. Secondly, research on tournaments that include pre-elimination rounds and elimination rounds do not exist. A vast majority of the research focuses on elimination tournaments, such as March Madness predictions and tennis rankings. Lastly, there has been previous research conducted on the gender gap. Cai et al1 examined performances during competitive pressure, their specific example being China’s National College Entrance Exam, considered one of the world’s most competitive exam. Cai et al found that gender differences translated to a 15% decline in the likelihood that females are eligible for admission into a Tier 1 university.

Data

I collected my data from tabroom.com and included 24 bid tournaments from the 2015-2016 season. I will continue to add more tournaments and possible more variables to my data set. I recorded school names, debater’s names, pre-elimination records, elimination records, and speaker-points. I was able to generate a list of genders from camp attendance sheets, Facebook, and websites that determine gender from names.

Results

I generated this first graph by plotting seed frequency by gender. This graph shows that males consistently have better seeding than females, as the blue line, which represents males, is more towards the left than the red line, which represents females. Based on this graph alone, we can extrapolate that male debaters perform better than female debaters.

For elim rounds, I marked debaters who were not present in an elim round as -1, a debater who lost an elim round as a 0, and a debater who won an elim round as a 1. For instance if I were to lose in quarters, I would be marked as a 0. For octas, I would be marked as a 1 and for semis and finals, I would be marked as -1. This graph charts seed vs. mean elim wins means by gender.

The previous graph told us that boys have better seeding than girls. However, when we look at the same exact seed, for example seed 3, we see that boys won more elim rounds than girls. A seed is supposed to be a non-biased and equal measure of performance. Instead, we see a gender bias. If I were to win the same number of rounds as a boy and get the same speaker-points, I am more likely to lose in elimination rounds because I am a girl. And that’s only if I were to get the same wins and speaker-points, which is less likely because according to the previous graph — girls have worse seeding than boys.

The graph also diverges for smaller seeds. This implies that at higher levels and at more competitive levels, boys perform significantly better than girls. While at lower levels, boys only perform slightly better.

For this graph, the x-axis is rounds won and the y-axis is frequency of rounds won. This third graph continues to show support for the gender gap. The many bumps are due to the different number of rounds. This graph shows that boys tend to have more winning records than girls. Moreover, girls tend to have more losing records. Out of 6 rounds, boys are winning a majority of rounds, while girls are losing a majority of rounds.

I use lm, which is a tool in R that allows me to perform linear regression analyses. This tool allows me to see the impact a coefficient has on its covariates. In very simple terms, I am able to find the impact of one variable on other variables. I have created a table with the results. My coefficient was the total number of elim wins, which I called totalw. My covariates are the variables in the first column.

 Estimate Standard Error Confidence Interval Male 0.1602454 0.0363653 0.088969412 0.231521388 Speaks 0.0004085 0.0009326 -0.001419396 0.002236396 Seed -0.0093938 0.000394 -0.01016604 -0.00862156

 Estimate Standard Error Confidence Interval Male 0.0835484 0.0339467 0.017012868 0.150083932 Speaks -0.0071 0.0009462 -0.008954552 -0.005245448 Seed -0.0018409 0.0005317 -0.002883032 -0.000798768 rWon 0.3695685 0.0189042 0.332516268 0.406620732

The first table shows the confidence intervals when we compared total number of elim wins and its impact on gender, speaker-points, and seed. The second table shows the confidence intervals when we added one more variables, rWon which represents the number of prelim rounds won. We compared the total number of elim wins and its impact on gender, speakerpoints, seed, and rWon.

Once again, these results show that gender has a huge impact on number of elimination rounds won. We see that the confidence interval is from 0.089 to 0.23 in the first table. This suggests there is a positive relationship with being male and the number of elim rounds won. The second table also suggests that there is a positive relationship between the male gender and the number of elim rounds won. On top of the graphs, these confidence intervals support the gender gap.

This graph shows the frequency of speaker-points by gender. The variety of bumps in this graph is due to the different number of rounds since the number of prelim rounds range from 5 to 7. Since I kept the speaker-points given on tabroom, which is a sum and not an average, our graph shows 3 peaks to represent the varying number of rounds and the speaker-points given.

The blue line is slightly towards the right in the right two peaks – indicating that boys tend to have higher speaker-points in tournaments with 6 to 7 rounds. This graph shows that there is a higher density of boys having higher speaks than females. Speakerpoints are important because they determine your seeding and whether or not you will advance to another round. In my personal experience, girls have been told that their voice is too high-pitched and that they should change their natural voice. Moreover, if a girl acts too aggressive, she is marked down for it while a male debater is simply assuming his dominant role.

As a novice debater, I thought to myself if I had the resources and a team of coaches, I could do just as well as any successful male debater.

When we start debating, we become aware of powerhouse schools. Their teams have a very strong reputation, a lot of funding and support from their school, and they have a strong attendance at the TOC. I defined a powerhouse school, as a school with at least one bid to the TOC each year for the past 6 years. I compiled this list by using NSD’s bid lists from the past 6 years. According to my metric of a powerhouse school, there are 13 schools that are powerhouses.

The first graph compares the number of rounds won among powerhouse schools and non-powerhouse schools. This graphs shows that powerhouse schools tends to have more winning records than non-powerhouse schools.

The second graph compares seeding among powerhouse schools and non-powerhouse schools. This graph further shows the dominance of powerhouse schools as they have higher seeds than non-powerhouse schools. So, my theory was correct – Having resources, coaches, and a strong reputation can aid a debater’s success.

However, I also assumed my gender would not play a role in my success if I had the resources of a powerhouse school.

So, I compared the different performance of powerhouse school debaters by gender. The first graph shows that males from powerhouse schools won more rounds than the females from powerhouse schools. The second graph shows that male debaters from powerhouse schools had better seeding than females from powerhouse schools.

Among most debate teams, you can easily identify the A debater and the B debater. A majority of times, both the A and the B debaters are males. These graphs are very telling of team dynamics. When you’re on a team that has the resources, the coaches, the funding, there is still a gender gap. Could there be favoritism on the team? In my personal experience, I have witnessed that big teams tend to split up. They send their A debaters to one tournament and the B debaters to an easier local tournament. This means the people who do not gain as much success at the start of the season or the coaches assume will not become successful, are not trained to reach their full potential as a debater.

When there are fewer girls on a team or even none at all, girls are less likely to join debate. Moreover, a female debater faces more obstacles than her male teammate.  For instance, it becomes more expensive as the debater has to pay for her own hotel room and the school must pay for a separate female chaperone. Therefore, I believe that a school has less incentive to groom girls to become the top debaters and may favorite boys. I also believe it’s also easier for schools to hire male debate coaches because not only do males win more, but also they are also more likely to stay in the activity after they graduate.

When we analyze the different performance among genders from non-powerhouse schools, we still see that males perform better than females.

The first graph shows that males have better seeding than girls. On the second graph, we see that males have more winning records than girls.

As a debater from a non-powerhouse all-girls school, what are we supposed to do when we see that girls systematically lose to boys when we have access to the resources and don’t have access to the resources? This begs the question of what makes a good debater, if its not access to resources or coaches.

Conclusion

My last slide for my presentation at the Undergraduate Research Conference was a collage of the pictures of finalists vbriefly posts when they report tournament results. I asked the audience if they could spot the female. You can scroll through the main website and see how long it takes to spot a female debater. I often joke that female debaters can write on our resumes and college applications that we are one of the top 10 female debaters in the nation. However, we can’t write we are one of the 10 debaters in the nation. The female qualifier allows us to jump to the top 10. In fact, the first girl to appear on Kirsch’s debate ranking is number 15.

While my data provides some insight as to how severe the gender gap is, the question remains: Is there a solution?

References:

[1] Cai, Xiqian, et al. “Gender Gap under Pressure: Performance and Reaction to Shocks⇤.” (2014).

Edited for typos.

• Passerby

For the regressions that you did, I think it would be nice to also include the t-statistics, as this would give more quantitative information into the degree of statistical significance. Additionally, I believe that enlarging the dataset with data from more time periods would also be beneficial. At that point, you could potentially account for some confounding variables with fixed effects modeling as well.

• Ariel Shin

Daniel just shared his enlarged dataset with me so I’ll be working on that soon!

I agree that graphs are not enough. Graphs are just more visually appealing to a non-statistics audience so I included more graphs. I tried to show more statistical significance with the confidence intervals. This is only a small portion of my research so I’ll try to include more next time.

• anonymous

I think that it is a misnomer to really call this a statistical analysis. Yes, you did statistics, but you interjected with personal anecdotes, and personal opinions. Anyone with a basic HS stats course can say that doing those things makes the article inherently un-statistic.

There are a huge amount of confounding variables at play that you have no accounted for. I think its a disgrace to call this statistics

• Ariel Shin

This article is based off a presentation I gave at a research conference to a non-debate audience. I thought it would be interesting to interject the statistics I provided with some of my personal experience. I thought the audience would enjoy hearing my story and understanding why I was passionate about this project. I didn’t want my research to be just numbers.

My research shows a relationship between different variables and I provide one point-of-view on this relationship. If you believe my personal opinions are incorrect, I would be interested in starting a discussion on your different experiences.

• lowell

what is the benefit of the powerhouse school? It seems to make a normative decision on what *is* a powerhouse school. Certainly no one would say that Leland or Torrey Pines isn’t a powerhouse schools.

I would be interested to see if a comparison of girl participation and retention at so-called powerhouse schools versus non-powerhouse schools. Or even schools with an active/involved head coach v. a figure-head head coach.

• Ariel Shin

My metric for powerhouse schools was one bid each year for the past six years, which would indicate consistent success. If you look back at the schools that are consistently sending kids to the TOC, they are indeed what many debaters would refer to as powerhouse schools.

I have created software that allows any user to easily create graphs based on any variable, which I will make public soon. While I don’t have a variable for active/involved head coach v. a figure-head head coach, you create one by school and use the software. For the powerhouse variable, there are a lot of permutations you can create when you have three variables (e.g. gender, power-house, and seeding) so the software could easily create any/all the permutations for you!

• lowell

that is cool. thank you. i take issue with using bids as a determination of powerhouse. there are lots of much more successful programs that choose not to care about the toc. and powerhouse schools on the toc circuit often get maligned or accused of being unfair while non powerhouse schools are seen as more noble. which schools, by your analysis do you determine are powerhouse?

• Ariel Shin

I used bid tournaments because their data is easier to access. They post their tournament results on tabroom, which I can easily access and work with. While some local tournaments do use tabroom, not all local tournaments do. The goal is to include all debate tournaments, and even sports tournaments, however getting access to data is very hard and time-consuming.

If you look at bid lists posted on various websites, you can get a general feel for what schools I marked as powerhouse schools,

• Emilio Rivera

Is the data used solely based off of LD bid tournaments, or does it also include other forms of debate when the tournaments host them?

• Ariel Shin

It is only based off of LD bid tournaments currently.

My goal is to include other forms of debate and possibly sports tournaments.

• kek

” In my personal experience, girls have been told that their voice is too high-pitched and that they should change their natural voice. Moreover, if a girl acts too aggressive, she is marked down for it while a male debater is simply assuming his dominant rule.”
Above is the entirety of the warranting for causation. The rest is either blatant implication that statistical correlation implies a systematic causation, or that certain trends reinforce. The only problem is that this line of argument is a fallacy. You have not demonstrated any need for “solutions,” just that you harbor some seeming resentment towards your debate experience and that girls tend to do less well. These only take on the characteristics described above when you apply ideology to them, which is not appropriate for a study supposedly founded on data.

• Ariel Shin

I never make the conclusion that correlation implies causation. I have been presenting my research to a non-debate audience so I thought it would be interesting to the audience if I provided my personal insight on the results of my data. My data can also be interpreted differently. I simply provide one viewpoint.

• JOHN JULIAN, SR

Hello Ariel,

First of all, great job on your presentation of these analyses.

I’d very much like to perform a professional-level peer review of your paper and findings over the Summer. In addition to having been a coach since 1989, I have been a Data Engineer and Data Science Consultant for the better part of the last 2 and a half decades and own a Data Quality Consultancy I founded this year. I’ve designed and implemented systems for data analysis for numerous organizations, including Amazon, Microsoft, Oracle, Visa, Expedia, US Departments of Defense, Energy, and Education, and CalTrans.

Would you be willing to publicize your data sets, code, and assumptions? Or at least send them to me for a confidential review. I’ll sign whatever IP and Disclosure documents you require. Please contact me via email.

Regards,

John N. Julian, Sr.
Owner/Founder, ThinkDriveSolutions, LLC
Head Coach, The Bear Creek School, Redmond, WA

• Ariel Shin

I can’t seem to find your email

• JOHN JULIAN, SR

I got your LinkedIn request. I’m a little swamped with things this week, but I will reach out over the weekend.

• Anonymous

This is awesome (though tbh statistics aren’t really my strong suit so if you could “translate” a lot more of what these graphs mean, that would be awesome) Also, may i ask how you determined the brightline for a “powerhouse school?”

• Ariel Shin

I defined a powerhouse school as a school that obtained at least one bid each year for the past 6 years to the TOC.

• anonononon

Your research on the gender gap is incredible tbh. Thanks for doing this.

I’m not as on board with the powerhouse research though. You define a powerhouse as a team that’s consistently successful. With that definition, you’re bound to get a conclusion that a powerhouse performs better at tournaments.

Maybe I’m reading into it too much, but you also seem to be making a normative conclusion that powerhouse schools shouldn’t be outperforming small schools in an ideal world. If so, I have to disagree. A lot of what makes a powerhouse school a powerhouse is not objectionable. You *should* do better in debate if you have more coaches. Resources and coaching help you perform better in virtually any competition, so it’s unsurprising debate is the same. From my view, what’s objectionable about powerhouses is that they can win because of rep or through influencing tab. I doubt there’s a metric for that though.

Also: “a male debater is simply assuming his dominant rule.” Did you mean to say “dominant role?”

• Ariel Shin

The purpose of the powerhouse research, in the beginning, was to confirm our assumption that powerhouse schools do indeed perform better than non-powerhouse school.

The purpose of including the research on powerhouse schools was to show that males who attend powerhouse schools outperform their female teammates. I do not make the suggestion that small schools or even non-powerhouse schools should outperform powerhouse schools.

• hogonquin

really good article, i would like to know why you use the phrasing gender gap when your data seems to be based off of biological sex? again i think this a really good article that reifies anything that anyone afab in this community knows.

• Ariel Shin

I haven’t really heard of the term “sex gap” so my professor and I have been using the term gender gap, which is more prevalent in the topic literature.