Degree of Risk

There are two types of risks with the SROI estimates we champion: 

  • Measurement risks - that SROI estimates are based on faulty estimates or assumptions
  • Programmatic risks - that the program will not produce the outcomes we expect.  We discuss both risks below, along with our approach to quantifying and minimizing them. 

We champion the use of ex post SROIs, obtained as a result of an investigation of long-term benefits actually delivered to and value created for specified beneficiaries.  We recognize, though, that after-the-fact assessments of all long-term benefits may be impossible or prohibitively expensive, particularly for programs that generate benefits years after beneficiaries exit the program or that have been in existence for only a few years. 

In such cases, we suggest the use of estimates of benefits, gathered from recognized experts, as a useful and cost-effective substitute, until ex post data is more readily available.  Where new initiatives are being considered, we recommend the use of ex ante SROIs, which rely on experts’ best estimates of expected benefits as a means of gauging likely SROIs.  We feel that projected SROIs can usefully inform giving decisions, particularly where such estimates lead to clear delineations between funding opportunities (e.g., an SROI of between 10 and 15:1 for one project, compared with SROIs ranging between 1:1 and 5:1 for all other available opportunities).

However, ex ante SROIs and experts’ estimates have inherent risks.  They may be based on faulty estimates (e.g., of outcomes, benefits or costs) or assumptions.  For instance, an SROI of a tutoring program may assume that graduating from high school increases a student’s projected real income by about $7,000 per year.  This projection, based on national studies, may be inappropriate in inner cities where employment rates and wages among twenty-somethings are well below national averages.

Prospective SROIs also are vulnerable to what might be called programmatic risks – the possibility that the program fails to produce expected outcomes.  Programs may lose funding or key personnel or beneficiaries may stop participating, resulting in decline in the rate and value of outcomes that underlie past SROI estimates.

We recommend funders factor both measurement and programmatic risks into their evaluation of program SROIs.  Where appropriate, adjustments should be made so that risk-adjusted comparisons are possible.  Below, we discuss our approach to these risks and how we help our clients quantify and minimize them.

  • Transparency 

We attempt to discuss all major known issues and risks openly, both in consumer-friendly language and in technically precise language.  These include such things as issues caused by our choice of comparison standards, our use of expected value techniques, as well as possibilities of our failures to implement our chosen approaches well.

  • Expertise

We use our professional expertise to provide our best estimates of range estimates of SROI estimates in a consistent, fair way across projects.  Cases where compelling decisions seem possible given risks of both project success and accuracy of information failure are identified.  Risks of project success failure are always explicitly listed and estimated as a way of inviting comment on them.  We also regularly provide roughly costed options for what we think are useful methods of improving estimates and our estimates of their likely results.

  • Humility

We invite and require users of our information and approaches to form their own options on the value of the results.  We champion some specific standards for consistency in an attempt to establish one useful standard while also inviting dialogue on it.  As long as all projects are evaluated on the same standards, this is probably useful.

  • Risk Markets

We plan to invite others to value risks of project success in a marketplace.

Some component ex ante SROIs that project benefits and values likely to be generated may be useful.  Ex ante SROIs are incomplete and may significantly overstate the value beneficiaries are likely actually to receive unless appropriate adjustments are made for risks of non-success.  Adjustments must also be made to reflect the risk that the estimate is overstated. 

For instance, ex ante SROIs frequently rely on national studies correlating specified results (e.g., high school graduation) with values to beneficiaries (e.g., $200,000+ in additional annual income) of the results.  We identify and attempt to quantify the risk that national studies are not a fair measurement of what is likely to happen in a particular situation.