A Critical Analysis Through the Lens of the Oct 31, 2022, Pitchbook’s Ranking of Universities Based on Entrepreneurial Output.

How does one assess the effectiveness of one’s educational programs? This is an incredibly hard problem in any case especially when it comes to entrepreneurship education. There are so many contributing factors it becomes impossible to know for sure. That does not mean we should give up. We must strive to constantly look for data points to objectively provide feedback on the reality of how well we are serving our mission.

In this regard, the just released “Pitchbook Study of Universities: Top 100 colleges ranked by startup founders,” by Jordan Rubio & James Thorne provides one relevant and insufficient benchmark. While I have an immediate negative reaction to rankings in general (we should never pit entrepreneurship educators against each other, we are all in this together), this one is extremely data-driven with a powerful data set that Pitchbook has been able to accumulate and does not have ambiguous formulas to determine ranking. Jordan and James are very transparent and even give filters to look at the data, which will come in very useful.

This study provides a benchmark to give a sense of how the overall output of various top institutions.  There could be a list of weaknesses hundreds long of what this fails to capture (I will highlight some later) but because we get a view of the raw data, I would like to make a few observations and add to the dialogue.

On a first pass of the numbers, thinking about ourselves, we are in great company. MIT ranks 4 in undergraduates, 3 in graduates, 6 in female founders undergraduate, and 3 in Female founders graduate. That being said there is another way to interpret the data that I am immediately left wanting. What I want to see is these results in some ratio relative to the total population. In that way, we could compare like to like. Think of this as the density of VC-backed founders (as a proxy, imperfect as it might be) per institution.  I have not seen this before and wished we could normalize the numbers, and we have made a first pass at this below.

But even before this is done, the beauty of this study is that Jordan and James have given us a first step in this direction because in the filters, you can see “Total Enrollment” as the second filter and there are three options: “15,000 or fewer students,” “15,001 to 29,999 students,” or “30,000 or more students”.  When you click on the filter to look at the institutions of 15,000 students or less, MIT is the clear #1 leader in all of these by a wide margin (on average 2.3 in number of founders who raised). This aligns more with what we see on the ground. The MIT ratio would be very high. Is it the highest? We don’t know for sure but normalizing the numbers makes me feel like this is a much more valuable benchmark. Well, when we discussed this at our center and everyone agreed. In fact, one of our EIRs, George Whitfield, took the initiative to utilize publicly available numbers and do this normalization.

This yielded the following (unaudited) results for Undergraduates:

Likewise for Graduate students:

Note: We did not have a data set we were comfortable with regarding the dimension of women founders which is important to provide insights into the inclusivity of the entrepreneurship education programs so that can be a later exercise.

This is in no way saying that these are the only KPIs or the key KPIs to measure our success, they are not in our opinion. It is but one data point that is flawed in so many ways. Does this mean that MIT and Stanford are the best? No, that is not what we are saying and we know that we are imperfect and always strive to get better in any case. In fact, the ranking mentality of pitting educational institutions can have a positive side to benchmark what is working but it also can have a very dark side. Entrepreneurship educators must work together. This is not a competition between each other. We have a common goal. The competition is against bad or no entrepreneurship education. We radically share and open source everything we do. That is what we all must do if we are to advance the field in the manner it needs to so as to address the challenges we face as a society.

That being said, it is good to have objective measurements of how we are doing—this is one data point and it is heading in the right direction. I wish it would go a step further to normalize the numbers based on student population size. It would still not at all be the singular or even most important metric to measure progress, but it would be a proxy for progress and success.

What are some of the other areas for improvement in this study?

1. Assumes that VC-backed founders are a proxy for success. We know this is not the only path to success. Entrepreneurs may not need to get nor should they seek VC funding to be successful.
2. We do not know the success rate of these companies, rather we just know how much they raised.
3. It does not normalize for input. Maybe MIT students would have raised money and been successful no matter what we did?
4. Which raises the question, how much a difference did we really make? Did we instill an entrepreneurial mindset into them? If so, how much? How much did we improve their skill set? How much did we improve their ability to operate like an entrepreneur in a community-based manner with resources beyond their control?
5. Ultimately, we as educators should have a goal (i.e., teaching objectives) that we instill in our students (e.g., mindset, skill-set, and way of operating per point #4 above) and measure our students before and afterward to determine the progress made in all of these categories.
6. In this ideal scenario, we would be able to measure the positive progress in the key areas mentioned above in #5 and our entrepreneurs would thrive as founders but also as entrepreneurs joining startups and also as entrepreneurs in existing organizations (government, academic, corporate, government, non-profit or others) and we won’t have to rely so heavily on numbers like those in Pitchbook but until then we have to work with what we have and keep making it better.

I would also note that recently we completed a ten-year empirical research study of the success of our delta v program that relates back to this topic of assessment and those extremely interesting results are available here.

Assessment will continue to be the 800-pound gorilla in the room for entrepreneurship education in my opinion. How do we know what we are doing works? To know, we have to have measurements and while there is some work done in this area, there is so much more that needs to be done to differentiate the effective from the ineffective… and do more of the effective. Like most other things in entrepreneurship, we can’t wait for the perfect, we have to go with what we have and keep improving it.

Afterword: I wrote this up and then read an email from my original mentor at MIT, Professor Ed Roberts, who had done numerous landmark studies on the impact of entrepreneurship and he had done a similar analysis so I guess I was trained well.

MIT delta v Demo Day 2022 happened on September 9th in from of 1,500 energized MIT students live and thousands more online. What you all are probably most interested in is that the videos are now available for viewing here (there is also a very nice digital version of our Demo Day book summarizing all the teams and the program). Every year, the student teams improve and this year was certainly no exception. We could not be prouder of this year’s cohort and we are absolutely confident they will continue and amplify the strong tradition of delta v entrepreneurs positively changing the world in a big way.

A few headlines of note:

1. Women Rule: While the number of participants and CEOs for those identifying as female has been strong from the beginning 10 years ago and rising since, this year marks a watershed in that 50% of the participants identified as female and 55% of the CEOs. You will see this in the videos for sure. If we want entrepreneurship to be the best it can be and the most impactful, we need the biggest tent possible to be inclusive. Very proud of this.
2. Back in Person Really Matters: Zoom is good for certain things but for teaching entrepreneurship, it can’t replace the high-touch instruction and mentoring that can be provided in person. Have done the program remotely in 2020 and then hybrid last in 2021, you could just see the energy at a different level. More importantly, we saw first-hand that the improvement of the teams was at a much faster rate in person and we got a lot more done this year than last year. So much more rewarding for the educators and mentors running too. There is a role at times for Zoom but for a fully immersive acceleration program teaching entrepreneurship, thank goodness we are back in person.
3. Trust the Process: We have always focused on developing the entrepreneurs much, much more than the companies that they are creating in delta v summarized by the statement “companies << entrepreneurs”. We stand by this statement completely because the entrepreneurs are the ones who create the companies and if we teach them how to do so (“teach them how to fish”), they will create multiple companies (“catch many fish”). This year I realized that there is something even more important than entrepreneurs, wait stay with me, and it is the process of creating entrepreneurs. After 10 years of delta v and seeing us start from raw students who don’t have an idea, a team, and certainly not a company one year earlier, and each year take them from a standstill (“0 miles per hour”) and by the end of delta v (note it is more than delta v in that the full year program includes the t=0 festival, courses, fuse micro-accelerator, co-curricular programs) these same students are legitimate entrepreneurs with exciting teams, plans and customers (“15 miles per hour”) never fails is the real gold here. To put this in simple terms, I love new mission-driven economically sustainable companies but I love even more high-quality innovation-driven entrepreneurs but most of all, I love the process to create these entrepreneurs because that has the biggest force multiplier effect of all. In other words: companies << entrepreneurs << process of creating entrepreneurs.
4. Impact Study: We had the fantastically great fortune of having Professor Daniela Ruiz Massieu, who is the head of entrepreneurship at ITAM in Mexico City, on sabbatical at our center this past year and she did a longitudinal study of the impact of delta v after 10 years. The results were stunning. Even going back to the beginning ten years ago, the companies that were started in delta v are still operational (or were acquired) at a rate of over 60% and if you look at the past five years, that rate is closer to 70%. The amount of money they have raised is over US $1 Billion for these companies. Again, remember they started from scratch. 38% have gone on to other accelerators like Y Combinator, TechStars, and MassChallenge. One of the most interesting statistics in my estimation is that participants have gone on to found another 120 companies that we can identify and they have raised over US $2 Billion. This reemphasizes the educational mission of our accelerator that we are teaching them how to create companies rather than creating one company. You can see a summary of the report here (https://entrepreneurship.mit.edu/delta-v-10-year-study/)  and we hope to have a full report out soon. Deepest appreciation to Daniela for doing this very important research.

There is so much more but I think the best thing you can do now is go and watch the videos of the students presenting. They are so inspirational and we hope they will inspire others as well that “Yes You Can!”

It is a new year and every year we like to continue to update the teaching materials we make available to all Disciplined Entrepreneurship educators. This year we have added examples of excellence in final deliverables that we make available to students in our course at MIT. These final deliverables are concrete examples of what students have submitted at the end of a semester of work. They provide inspiration and a benchmark for others – and students have found it very helpful to see what others have done to help guide them as they work throughout the semester toward their goal of something similar. No path in entrepreneurship or this course is ever the same, nor should they be, so these should not be used as templates that if repeated assure success but rather are imperfect but pretty damn good role models of what success looks like in the end and how achievable it is.

If you do not already have access to the teaching materials, you can gain that by filling out this form. It is free and provides a treasure trove of materials to help you that are open source, meaning they can be used under the Creative Commons licensing arrangement (feel free to use it with attribution and if you produce derivative works, they will be made available for free to the community on the same basis).

This is meant to help achieve our goal of creating more entrepreneurs, higher quality entrepreneurs, and better-connected entrepreneurs globally!

There was a lot of good material and updates in the red Disciplined Entrepreneurship Workbook which came out after the original white Disciplined Entrepreneurship book that had a chance to be digested by the public and myself. There is a lot of very good material in the book, like the guide on primary market research, the DE canvas, new material on windows of opportunities & triggers, and more that I do not want to trivialize.

The thing I get asked most about is whether the worksheets can be made available in digital format. So here they are and also with a few other good things that I use in our classes each year that you might find interesting and helpful as well.

PS—Please let others know this material is now available.

For well over 20 years, the MIT Entrepreneurship Development Program (EDP) has been the largest and most highly rated Executive Education program offered by MIT Sloan. Think about that for a second as it is quite an achievement. “Executives” are usually looking to take courses to help them become better leaders in big corporations and what we teach in EDP is entrepreneurship!  And among all the excellent courses taught by fabulous MIT professors in MIT Sloan Executive Education’s portfolio, our humble EDP ranks the highest.

But success always has an expiration date on it. Like our friend Alberto Rodriguez de Lama of The Cube in Spain and a member of the EDP community likes to say, “Always in beta, always improving.” So this year we made some significant changes to experiment with new ideas. We kept the good we know works and made the following changes for what we called “EDP+”:

1. Timing:  Since its inception, EDP has run in January. We joke we do this to keep the participants focused and make sure we only get the most serious people in the program. After all, it can be really cold in January in Boston. The real reason is this is the window in our academic calendar where the program fits best. EDP is a fully-immersive environment that requires a full-time dedicated core faculty team for a full week. This year, the innovation in #2 below allowed us to run the program the first week of June for the first time ever. Yes, there were more distractions (Red Sox games, concerts, Celtics games, just going outside in the gorgeous weather, etc.), but it still worked out really well. Because it was early in the summer, we were also able to get key faculty to participate as well.
2. Integration with MIT Students: For the first time I believe in any MIT Exec Ed program, the participants were in the same classroom and sharing breaks with MIT students who were learning and applying the same content. These were not just any students either; these were the best, the brightest, and the most committed entrepreneurial students at MIT, the delta v summer accelerator teams. While this sounds great, there were some risks with this plan. First, it has never been done before, but that does not scare us as entrepreneurs. Second, they were at different starting points. The EDP participants were starting from nothing with no idea, no team, and little to no knowledge of the Disciplined Entrepreneurship process. The delta v teams had all of these things and momentum coming into the week. Third, there was a significant age and experience gap between the two groups.
3. Even More New Content than Usual: Every year, we look to enhance our content and integrate new materials, but this time, we did this in two critical areas. The first was to integrate the strategy, frameworks, and tactics that entrepreneurs should consider as they develop their “Go-To-Market” plans taken from our new course at MIT. The second was to integrate into EDP key material from another new course, Paul Cheek’s “Venture Creation Tactics”. While this was material that has been tested during the school year, it was extremely leading edge, very actionable, and went beyond the traditional 24 Step Disciplined Entrepreneurship framework. It was more new material than we would traditionally integrate, but we felt it was important to equip the participants with the latest in the field.

Because of these changes above, we positioned this version of EDP as a new experiment, as “EDP+,” and charged ahead to run it while anxious to see the results. As anyone who has attended EDP can attest, this course is like no other program. After 6 days of fully-immersive entrepreneurship sprinting by Friday, everyone is exhausted, including the staff. I am pleased to say that while everything did not go perfectly (perfect only exists in comic books), at our team debriefing on Friday afternoon we were *very* pleased with how it all went.

Most importantly, the EDP participants and the delta v teams really enjoyed and got extraordinary value out of the program. Here are a few of the posts (I am sure I have missed a lot, but please add the missing ones in the comments):

1. “10 Things I Learned in EDP+” from the Tie Guy https://www.linkedin.com/feed/update/urn:li:activity:6940702979356307456/
2. “Great Artists Ship” https://www.linkedin.com/feed/update/urn:li:activity:6941152446693949440/
3. “Life-Changing Experience” https://www.linkedin.com/feed/update/urn:li:activity:6941483147129724928/
4. “Mind-Blowing What You Can Achieve in a Week” https://www.linkedin.com/feed/update/urn:li:activity:6941423204875812864/
5. “Yes You Can” https://www.linkedin.com/feed/update/urn:li:activity:6941508000297701377/
6. “Started the week as an explorer and ended as a believer that, taking an idea and commercializing it, is a craft that can be taught.“ https://www.linkedin.com/feed/update/urn:li:activity:6941568760440901632/
7. “My Cliff Notes from EDP+ https://www.linkedin.com/feed/update/urn:li:activity:6941759839056408576/

Anyone who has taken EDP is part of the EDP community and will never forget the experience. Know you are part of a special collective. One of the questions we get most often about the program is can we run it twice a year, and until now the answer has been “Heck No!” However, with these latest modifications, maybe we can and expand our community. That would be wonderful.

Special shout-outs to the MIT Executive Education team, especially Ann Marie Maxwell and Peter Hirst for believing this was possible. Also to the Northern Ireland Catalyst program led by John Knapton and Adele Ward for believing in us and bringing your great entrepreneurs over for this experiment. Also to Jennifer Craw, CEO of Opportunity North East (ONE), and Freda Miller of Scottish Enterprise who kept our long-term (10+ years) relationship with Scotland rolling and also brought a wonderful cohort of entrepreneurs to give us a great core to start the class with. And all the rest of the colorful cast of characters from all parts of the world and all walks of life who made the room so vibrant this past week. Hopefully, every single one of you is feeling more antifragile and knows: (1) Yes you are an entrepreneur; (2) Yes you can continually get better at it; and (3) Yes you are now a damn good entrepreneur with your training and you can lead a more meaningful life and put a positive dent in the universe.

Huge thanks to all the instructors and guest speakers, judges, mentors, and others who we had come in including the great Catherine Tucker, Adam Blake, Matthew Rhodes-Kropf, and David Frankel (and his mother). And of course to my co-lead for this course, the fabulous Paul Cheek.

Lastly, enormous thanks to the Trust Center team who helped pull off this intense one-week-long doubleheader – EDP+ and delta v kickoff. The center is nothing without the people who make it all happen. Here is to you!

Onward and upward and if you want more info on EDP, follow the link at the beginning of this post.

Finally, remember, while we joke around and have fun in EDP, what we are doing is so important. While we should not take ourselves very seriously, we must take the bigger mission very seriously. The world desperately needs more entrepreneurs, high-quality entrepreneurs, and better-connected entrepreneurs. Keep your eyes on the prize.

#donnachisholm #nicoladouglas #HIE

On March 10th, I returned to the Gulf Region for the first time in four years at the invitation of the UAE Entrepreneurs Organization (EO). I was not sure what to expect after such an extended time period and so many dramatic changes in the world. After a three-day workshop with an extremely active group of entrepreneurs, I leave very encouraged.

My high-level observations are as follows:

1. The gap is closing between the Gulf Region and the rest of the world with regard to innovation-driven entrepreneurship. Coming from a history of trading, the region was always right with SME (Small Medium Enterprise) entrepreneurship but the highly scalable variety created by innovative products was a rare beast. That has definitely changed very rapidly.
2. This progress has been caused by increasing connections to the rest of the world and the development of an increasingly vibrant community. Entrepreneurship is not a solo sport and thrives in clusters. It is not just a support structure but it is also an accelerator from the additional ambition that it creates and the lateral learning that takes place between entrepreneurs. EO UAE has done a fabulous job of catalyzing this and developing programs that are unambiguously focused on and in the best interests of the entrepreneur.
3. COVID has been another unexpected accelerant. When the proverbial chessboard gets knocked over as COVID did, it is an advantage for new agile, and innovative companies. Speed is the friend of the entrepreneur and speed and creativity were essential to survive and thrive in the pandemic, and Gulf entrepreneurs seized the opportunity.
4. Confidence is at an all-time high. Seeing the successes of Careem, Souk, and others from the region, has given not just provided a blueprint for successful innovation-driven entrepreneurship, it is made the entrepreneurial mindset to be something to be proud of. Now the prestige jobs are not just working for a big company or the government, but these entrepreneurs are the new rock stars of the region similar to how they are viewed in the US.
5. Being in person again reminded us of how important personal interactions are to entrepreneurship. COVID brought opportunities for entrepreneurs but there was also a cost. So much of entrepreneurship is about sensemaking and integrating many different intentional and unintentional ambiguous signals so we can see gaps in the marketplace and move forward. Entrepreneurship is a team sport where you are building your team and working with others in the community to quickly garner resources beyond your control. In a Zoom world, this was possible in an intentional way but when in an environment like the three-day workshop, it reminds you of how valuable the unintentional serendipitous collisions are for entrepreneurship.

For me, it was nice to be back in the region and also leading an in-person workshop for the first time in years. The energy was palpable. The community was vibrant. The skill level and accomplishment are higher than ever. The mindset was confident. All of this bodes well for innovation-driven entrepreneurship in the region. A region that very much needs this to create new jobs for the wave of young people hitting the marketplace. The supply of the new young people entering the marketplace cannot be fully employed by the government or large company jobs. As the data shows for the US, the lion’s share of the new job is being created by innovation-driven entrepreneurs and hence their critical role in our future.