How Well is MIT Really Doing at Creating the Next Generation Entrepreneurs?

A Critical Analysis Through the Lens of the Oct 31, 2022, Pitchbook’s Ranking of Universities Based on Entrepreneurial Output.

How does one assess the effectiveness of one’s educational programs? This is an incredibly hard problem in any case especially when it comes to entrepreneurship education. There are so many contributing factors it becomes impossible to know for sure. That does not mean we should give up. We must strive to constantly look for data points to objectively provide feedback on the reality of how well we are serving our mission.

In this regard, the just released “Pitchbook Study of Universities: Top 100 colleges ranked by startup founders,” by Jordan Rubio & James Thorne provides one relevant and insufficient benchmark. While I have an immediate negative reaction to rankings in general (we should never pit entrepreneurship educators against each other, we are all in this together), this one is extremely data-driven with a powerful data set that Pitchbook has been able to accumulate and does not have ambiguous formulas to determine ranking. Jordan and James are very transparent and even give filters to look at the data, which will come in very useful.

This study provides a benchmark to give a sense of how the overall output of various top institutions.  There could be a list of weaknesses hundreds long of what this fails to capture (I will highlight some later) but because we get a view of the raw data, I would like to make a few observations and add to the dialogue.

On a first pass of the numbers, thinking about ourselves, we are in great company. MIT ranks 4 in undergraduates, 3 in graduates, 6 in female founders undergraduate, and 3 in Female founders graduate. That being said there is another way to interpret the data that I am immediately left wanting. What I want to see is these results in some ratio relative to the total population. In that way, we could compare like to like. Think of this as the density of VC-backed founders (as a proxy, imperfect as it might be) per institution.  I have not seen this before and wished we could normalize the numbers, and we have made a first pass at this below.

But even before this is done, the beauty of this study is that Jordan and James have given us a first step in this direction because in the filters, you can see “Total Enrollment” as the second filter and there are three options: “15,000 or fewer students,” “15,001 to 29,999 students,” or “30,000 or more students”.  When you click on the filter to look at the institutions of 15,000 students or less, MIT is the clear #1 leader in all of these by a wide margin (on average 2.3 in number of founders who raised). This aligns more with what we see on the ground. The MIT ratio would be very high. Is it the highest? We don’t know for sure but normalizing the numbers makes me feel like this is a much more valuable benchmark. Well, when we discussed this at our center and everyone agreed. In fact, one of our EIRs, George Whitfield, took the initiative to utilize publicly available numbers and do this normalization.

This yielded the following (unaudited) results for Undergraduates:

Likewise for Graduate students:

Note: We did not have a data set we were comfortable with regarding the dimension of women founders which is important to provide insights into the inclusivity of the entrepreneurship education programs so that can be a later exercise.

This is in no way saying that these are the only KPIs or the key KPIs to measure our success, they are not in our opinion. It is but one data point that is flawed in so many ways. Does this mean that MIT and Stanford are the best? No, that is not what we are saying and we know that we are imperfect and always strive to get better in any case. In fact, the ranking mentality of pitting educational institutions can have a positive side to benchmark what is working but it also can have a very dark side. Entrepreneurship educators must work together. This is not a competition between each other. We have a common goal. The competition is against bad or no entrepreneurship education. We radically share and open source everything we do. That is what we all must do if we are to advance the field in the manner it needs to so as to address the challenges we face as a society.

That being said, it is good to have objective measurements of how we are doing—this is one data point and it is heading in the right direction. I wish it would go a step further to normalize the numbers based on student population size. It would still not at all be the singular or even most important metric to measure progress, but it would be a proxy for progress and success.

What are some of the other areas for improvement in this study?

1. Assumes that VC-backed founders are a proxy for success. We know this is not the only path to success. Entrepreneurs may not need to get nor should they seek VC funding to be successful.
2. We do not know the success rate of these companies, rather we just know how much they raised.
3. It does not normalize for input. Maybe MIT students would have raised money and been successful no matter what we did?
4. Which raises the question, how much a difference did we really make? Did we instill an entrepreneurial mindset into them? If so, how much? How much did we improve their skill set? How much did we improve their ability to operate like an entrepreneur in a community-based manner with resources beyond their control?
5. Ultimately, we as educators should have a goal (i.e., teaching objectives) that we instill in our students (e.g., mindset, skill-set, and way of operating per point #4 above) and measure our students before and afterward to determine the progress made in all of these categories.
6. In this ideal scenario, we would be able to measure the positive progress in the key areas mentioned above in #5 and our entrepreneurs would thrive as founders but also as entrepreneurs joining startups and also as entrepreneurs in existing organizations (government, academic, corporate, government, non-profit or others) and we won’t have to rely so heavily on numbers like those in Pitchbook but until then we have to work with what we have and keep making it better.

I would also note that recently we completed a ten-year empirical research study of the success of our delta v program that relates back to this topic of assessment and those extremely interesting results are available here.

Assessment will continue to be the 800-pound gorilla in the room for entrepreneurship education in my opinion. How do we know what we are doing works? To know, we have to have measurements and while there is some work done in this area, there is so much more that needs to be done to differentiate the effective from the ineffective… and do more of the effective. Like most other things in entrepreneurship, we can’t wait for the perfect, we have to go with what we have and keep improving it.

Afterword: I wrote this up and then read an email from my original mentor at MIT, Professor Ed Roberts, who had done numerous landmark studies on the impact of entrepreneurship and he had done a similar analysis so I guess I was trained well.