FAQs

1.  There many approaches to measuring SROIs.  Which is the right one and how do I compare the SROIs you report with those reported by other organizations?

There are many valid approaches to measuring SROIs – and some we regard as less good. While we do champion the ones we use for the cases to which we apply them, we primarily champion transparent analysis that lists assumptions used and that quantify the impacts of each of them.  This will allow useful, consistent, fair comparisons across projects.  We consider these things to be very important.  Comprehensive inclusion of all benefits and costs is also important.

We lean toward applauding asking SROI questions and getting at least some information on them.  This generally will start a dialogue on these issues that, even if confusing for a time, should get sorted out reasonably quickly.

We do champion consistent comparisons.  SROIs that include multiplier effects and benefits to taxpayers should not be compared to those that could, but do not, include these factors, for example.  Comparisons of the SROIs we and others report may require inputs from those who have experience in this field.  We are open to inquiries of our expertise can be helpful in such matters.

2. How do you respond to the criticism that your approach encourages funders and service suppliers to pick just the “low-hanging fruit” – helping only those who need one-time or very low cost interventions to dramatically improve their circumstances and leaving behind harder, but worthy cases?

First acknowledging that this risk is likely inherent to any effort to evaluate the effectiveness of charitable and social programs, we respond as follows:

We advocate the use of SROIs as one factor to informing decisions.  While highly informative, it appropriately may be one of several factors considered.  Wisdom and judgment are required.  Some causes require more time, expense, and innovation than others and, consequently, have a low SROI.  Donors may find these causes to be important acts of mercy or justice that they will fund, even given an opportunity to fund higher SROI projects.  Getting certain drug addicts off the streets and into permanent housing may be extraordinarily time-intensive and expensive but some donors are called to this cause over others with more impact or lower costs.

On the other hand, if certain programs do allow individuals to self-select in and produce high SROIs for them, one might prefer to prioritize those. Our hope is that such donors will have information that will enable them to fund the most cost-effective programs serving these particular causes.  For this reason, we recommend that all SROI evaluations identify the beneficiaries impacted and the likely extent of their need absent intervention.

3. Are there any charities you recommend giving to at present?

Yes.  We feel that the charities recommended by GiveWellTM are worthy of serious consideration. They have made a serious effort to identify charities that generate high benefits relative to costs and risks, and their recommendations and evaluation process is one of the best we have seen to date.  GiveWell seems to feel that the highest value for the poorest people is found in international health enterprises.  We would be surprised if the charities they recommend would not also show up as projects and charities SMI would also recommend on our criteria.

If a person wants to give to a benefit/cost vetted charity in the US – and in New York City in specific – the Robin Hood FoundationTM seems to have done excellent due diligence in this area.

While GiveWell and Robin Hood do not currently publish expected SROI magnitude estimates as we might prefer, the charities they recommend are probably worth investigating.

4. Does SMI plan to recommend places for donors to give money to in the future based on its own evaluations?

Yes.  We are thinking of using either a 1-5 Star rating approach or a more direct “Strong Give, Give, Consider, Neutral, or Avoid” set of categories.  If we perform charity ratings, we intend to ensure our full financial and relational independence from any charities we rate or whose evaluations we publish, using principles that are well established for these functions in financial markets.

We also plan to work with the DMPE Foundation and other groups to help establish actively managed portfolios of high SROI projects that donors can consider or participate in.

5. Does SMI publish comments on non-recommended organizations?

No.  If funders ask us to investigate specific charities, we will report privately our views on those charities, as they stand at the time of our assessment, and possibly with respect to things the charity might do to improve itself.  We will not publish these views, at least in the near-term future, and, in any case, will abide by any confidentiality agreements we sign.
We do insist on the right to report on SROIs we develop in general terms although we will work to ensure that particular organizations cannot be identified from our presentations. 

7. Are charities informed about SMI reviews? Do they have an opportunity to respond?

Yes, to both questions. The only time a charity will not be notified of our review is when we have been engaged by a donor to conduct a review and the donor has requested that we not inform the charity.  This is unlikely because SMI strongly prefers that we work with charities to do our evaluations.

We are believers in cost-effective due process and want to explain to charities why we have our views and give them a chance to respond, ideally convincing us that our views are wrong or that the charities are going to take remedial steps in places where we offer constructive suggestions.

8. Do you provide advice to charities on how to qualify for an SMI recommendation?

Yes.  As a general rule, we believe SROI calculations for human assistance endeavors are valuable to help service suppliers know what is working, to what extent, and at what cost and risk so they can compare themselves fairly to other projects and/or raise their productivity.  We are aware that every individual charity is unique and that a full evaluation requires a lot of work - often more than we can provide.  In such cases, there are other service providers and evaluators that are available to provide these services.

9. What is the ratio of overhead to program expenditures SMI recommends?

SMI presumes that every charity needs to raise money and have non-program overhead to pay for office space, taxes, etc. (even if this is donated by someone else).  While this is recognized, our approach emphasizes the amounts of value that specific donations within various programs produce for beneficiaries.  Some programs produce a lot of value, others do not.  Higher administrative spending may be necessary to optimize the value creation of individual programs and, therefore, may be fully justified.

Where appropriate, we will consider how spending on overhead impacts the value-creation of an organization’s programs and, with that lens, may comment on recommended overhead levels (for example, suggesting that middle managers be hired so the Executive Director can develop additional programs or raise funds for highly effective, established ones).

10. How do SMI charity evaluations differ from those of other prominent charity evaluators?

To date, organizations publishing charity evaluations have focused primarily on the percentages of costs that go toward program work and fundraising and other measures of transparency about how funds are spent.  We applaud these evaluations – they provide useful information about the integrity and, to some extent, fiscal responsibility of evaluated programs.  However, we believe more information is needed, namely analysis and data on whether programs are producing reasonably measured value for beneficiaries.

Consider what it would look like to rely on the common used metrics of charity evaluators in making other expenditures.  When shopping for a car, for example, the focus would be on the car manufacturer’s overhead rates and marketing budgets and not on data about performance or reliability relative to cost.  We aim to provide funders with performance, reliability and cost-effectiveness data, so that they can be considered alongside information about transparency.

11. Do you offer instruction manuals about how to do self evaluations?

Yes, please see our Guidelines to Measuring SROIs.  We also suggest charities read our reviews of other charities we have recommended.  These reviews provide examples of the types of questions we ask, the standard of evidence we look for and the level of detail we think is appropriate.  If you have additional specific questions, please don’t hesitate to send us an email.

12. Are SMI’s communications with charities, beneficiaries or others on the record? Will SMI publish documents (e.g., evaluations and SROI analyses) we send to clients?

No, to both questions.  We respect all confidentiality requests and are willing to sign  nondisclosure agreements when it is appropriate to do so.

13. Can I make donations to SMI?

No.  Currently, we are not accepting unsolicited donations of more than $5 from any donor for any SMI activities.  We do aim to build around $200,000 in assets to provide employment stability for staff to do independent charity evaluations and to allow us to provide productivity-enhancing incentives to charity participants, especially beneficiaries, in 2012-3.