What Gives Me Confidence That We Are Successfully Teaching EntrepreneurshipClassics 

One of the great challenges for me as an entrepreneurship educator is knowing whether the material I am teaching works — that my students get value out of the curriculum and that they are more successful entrepreneurs as a result. I know that the methods I teach helped me go from losing money in my first venture to exiting my third venture after it had increased in value by a half a billion dollars, but that is just one data point. I believe in what I teach, but how do I know it works for others? How do I answer the logical and appropriate challenge of “prove it?”

This question has been in the back of my mind since I started teaching over ten years ago. In that time, several systematic analyses of the programs we’ve run have given me the confidence to say “I know it works.” It will be impossible to unequivocally prove scientifically that they work because we will never have a perfect control set to measure against – it would be unethical to deprive a random subset of students of entrepreneurship educational opportunities – but the three data points I present here do as good a job as possible at evaluating the programs’ impact. I should also say that as an educator, I deeply believe that the measurement of our success should be more than economic impact from new companies because I believe that entrepreneurship is about more than startups. All of this considered, while incomplete, the success of new companies is the metric most often used in these analyses. New company formation is the one clear metric that seems to validate these programs in the eyes of the public. It is also the one we can currently best quantify.

The first data point validates what I see firsthand on a daily basis at MIT’s entrepreneurship center working with students inside and outside of the classroom. In late 2017, we hired a highly talented outside researcher, Tory Lee, to study how successful our delta v graduates were. “delta v” is our fully immersive 3-month program that provides the space, structure and support to launch a new venture. The structure of the delta v program is based off the Disciplined Entrepreneurship approach. I believed the approach and methodology were positively impacting students because I saw our students make great advancements during and after the program, but I wanted to see data to verify this belief.

Our study got us that data in spades and it certainly positively surprised all of us. Of the 76 teams surveyed (a statistically meaningful number), a stunning 75% of the companies were either still in business or had been acquired. At the time the study was completed, January 2018, these teams had raised over $150M (this number is now well over $200M). So on average, each company raised $2M, with the median being $300K. This exceeded the range of our most optimistic projections.

The second data point is a study done by Highlands and Islands Enterprise (HIE), part of the Scottish government, to understand if they made a worthwhile investment in training their entrepreneurs at MIT using the Disciplined Entrepreneurship frameworks.  HIE made a long-term commitment to entrepreneurship skills development in a comprehensive program called the Entrepreneurship Support Programme (ESP). At the core of this program were two key elements based off Disciplined Entrepreneurship. The first was a series of two-day master classes, but the even more substantial commitment was sending a dozen or so entrepreneurs every year to the MIT Entrepreneurship Development Program (EDP).  EDP is an intense one-week program that uses Disciplined Entrepreneurship as its core methodology.  

Since HIE was using public funds, they had to be extremely diligent in proving the money was being used effectively.  As such, HIE undertook two external, rigorous assessments of the value-for-money of all their entrepreneurship investments. In 2012, Cambridge Economic Associates assessed that EDP was providing a return on investment (ROI) in excess of £8 back into the Scottish economy for every £1 spent.  This was based on what entrepreneurs from Scotland subsequently went on to do, in terms of creating jobs and building wealth, and whether they attributed their actions to their learnings from EDP. In 2015, HIE externally commissioned a second economic impact assessment. This assessment indicated a potential future ROI of up to £28 for every £1 spent, which indicates an improvement in the relevance and effectiveness of the course content from the increasingly refined Disciplined Entrepreneurship approach.  A 28 times return on investment is an extremely positive outcome by any measurement.

The third and most recent data point came from the “Way to Scale” program, which is part of the Catalyst Organization in Belfast, Northern Ireland run by John Knapton. A core part of this program is the previously mentioned one-week full-immersion Entrepreneurship Development Program (EDP) at MIT. I saw the transformation of these entrepreneurs as they became more systematic in how they approached entrepreneurship, and resultantly more confident, but it was hard to quantify the impact. Fortunately, John and his team went back a year later and did just that. Again, the results were extremely positive. For the nine CEOs in the program, their annual revenue for 2019, the year before their new plans and execution resulting from their EDP training, was £4.2M. The projected revenue for 2020, resulting from applying what they learned in the program, is conservatively forecasted to be £24.6M.  Since the analysis, the CEOs now believe that the 2020 revenue numbers will be even higher. Exports, jobs and profitability also took big steps forward, as you can see in the chart below.

So what is the conclusion to be drawn from these three analyses? They do have their caveats. First, as previously mentioned, there is no control set to compare them to, so they do not strictly meet the scientific testing protocol to prove causality. As Professor Scott Stern points out, the focus of our center is to provide an educational service and not be an experiment.  Second, there is selection bias in that they the entrepreneurs studied should perform above average because they were chosen in a competitive process that would bias in favor of above-average performers. Third, the Disciplined Entrepreneurship approach and methodology were not the entire program in all three cases (but it was the fundamental framework in all three) so there are other elements integrated into the programs that contribute positively and it is not possible to disaggregate them. Finally, it is important to note that these programs depend on the people running them as much as they do the methodology.

With all those disclaimers, the results of the analyses are still compelling to me in showing that the methodology works very well. This lines up with the anecdotal evidence that we experience every day with not just our students at MIT but also people like Mary Rodgers of PorterShed in Galway, Ireland. I think she describes the methodology extremely well when she says:

“There is no one answer – the disciplined entrepreneurship process combines multiple publications such as lean business models, ‘crossing the chasm’ and others, to create a cohesive actionable disciplined process. I use these learnings every day to stress-test a solution or to help a company to problem-solve. Following the 24 steps of disciplined entrepreneurship will accelerate success and failure, and takes the subjective out of the equation.”  

I am confident but not arrogant about Disciplined Entrepreneurship, because I know it is not the only approach that works.  It is a successful core approach and methodology that, as shown above, must be part of an integrated approach that includes an excellent educational team and complementary programs as appropriate. For example, at MIT, credit goes to the Venture Mentoring Service, Deshpande Center for Technological Innovation, Sandbox Innovation Fund, Technology Licensing Office and Gordon Engineering Leadership Program, to name a few of the organizations we collaborate with. Scotland and Northern Ireland have outstanding teams to execute their integrated programs as well.  Each situation integrates other material and important context to make the overall educational experience a success.

My fervent belief and hope is that Disciplined Entrepreneurship is not a fixed methodology, but is an evolving open source approach that is not deterministic. The approach does not guarantee success, but rather significantly increases your odds of success, as described so well by Mary Rodgers above. It does, however, have rigor and relevance that is so badly needed, that only results when the approach is developed as an ongoing collaboration between academics AND practitioners.

With these caveats and understanding that the current Disciplined Entrepreneurship methodology will continue to improve over time, we can still use the above data points to test what we have today. For this existing approach and frameworks, after what I have seen directly with my own two eyes and the three compelling studies analyses above, I feel pretty damn confident saying: This stuff works.

To see a partial list of now over 200 schools using DE materials, go to our Resources for Educators page. Educators can also sign up there to get free access to the teaching materials used at MIT and others offered by the community.