In early 2015, Honolulu-based Blue Startups was ranked as the 17th-best U.S. seed accelerator, putting it alongside the pioneers and benchmarks of startup acceleration, like Techstars, 500 Startups and others. The criteria for that list was fundraising, valuations, exits, survival rate, and alumni satisfaction — all private data, but information which Blue Startups had to share with the researchers.

Now that 2016 is underway and the Legislature will undoubtedly look at additional funding for Blue Startups and other initiatives backed by the Hawaii Strategic Development Corporation, the question is this: Is Blue Startups helping to enhance Hawaii’s nascent tech industry?

“Blue Startups (has) been very, very effective at galvanizing support and recognition for Hawaii startups, both locally and nationally,” said Karl Fooks, HSDC president. “This is what we should all want.”

Beyond just the success of its startup graduates, its contribution to the state’s overall tech sector includes the experience and connections it provides to entrepreneurs, the sharing of wisdom it facilitates, and the satisfaction rate of graduates. But those measures are highly subjective, and Blue Startups’ contribution to the overall tech ecosystem here has been overwhelmingly positive.

Just look at the number of events, groups, and startups that have surfaced in the past few years. And, of the dozens of entrepreneurs, mentors, and related supporters I’ve talked with, their comments on Blue Startups have been overwhelmingly positive.

However, being a numbers guy, I like quantitative measures. The seed accelerator rankings mentioned above were based on strictly confidential measurements of fundraising, valuations and more submitted by the accelerators themselves. But, more ego-driven and publicly accessible metrics are available through sites like Crunchbase, AngelList, and simple Google searches. That research formed the basis of this analysis.

I reached out to Blue Startups to confirm my numbers, but its spokesperson, Jared Kushi, operations manager, provided only high-level data and its numbers didn’t quite match what I found. I’ve included both its numbers and my findings below.

Looking Closer At Success Rates

Blue Startups says that it’s funded 47 companies and only six are out of business, for a survivability rate of 87 percent.

Through Blue Startups’ own cohort listings, various blog posts, articles, and its Wikipedia entry, I counted a total of 48 teams that were announced as entering its accelerator since day one (some teams either combined or were listed in more than one cohort).

Note the difference there: “funded” versus “announced as entering its program.” Also note that a few of those who entered — Minded, Doctory and others — have been deleted from Blue Startups’ cohort pages, but I included them anyway. Sure, they may have failed during the program, been kicked out, walked away on their own, or not “graduated” for another reason. However, they were an “input” so I’m counting them in the success/fail ratios.

Of those 48 incoming startups, 31 are still operating, meaning 17 are “not operating.” I defined “operating” as having a live website and having either posted to social media or had an article written about them in the past few months. Not the most trustworthy of metrics (I didn’t count automated tweets, like paper.li), but publicly available. It’s also my largest discrepancy with Blue Startups’ own numbers.

To put both of these numbers into perspective, a 2014 article has Techstars’ active rate of graduates over its previous eight years as 78 percent. The same article puts Y Combinator’s active rate over eight years as 68 percent. 500 Startups’ active rate is estimated at about 66 percent over 550 startups since 2010.

Blue Startups’ success rate is good in comparison, but it’s only been around about for three years. One would assume that older grads will continue to fail because that’s what most startups do, regardless of location. Barring larger cohorts in the future to bring in new, active startups to tip the scales, Blue Startups’ success rate is likely to fall. But, for now, it’s definitely competitive, even at the lower rate of my calculations.

“Failure is normal,” Fooks said. “It doesn’t take much these days to get started, but the process is not about making everyone a winner. This is venture.”

‘Given Great Opportunities’

A big metric for accelerator success is “follow-on funding,” which is the amount of venture capital raised after graduating from an accelerator program. “After” is the key word here, which matches “follow” in “follow-on funding.” Keep that in mind.

Since Techstars is fairly transparent with its metrics, it offers a good, albeit somewhat apples-to-oranges, comparison as it has advantages in locations, connections and talent that Blue Startups does not. Techstars says that “about 75 percent of companies have received follow-on funding and/or immediately became profitable after Techstars ended.” It also says that, on average, its grads “raise $1-2 million post-demo day.” MuckerLabs, another top 20 accelerator, claims a 100 percent funding rate.

According to Crunchbase, Angel List, and other sources, 12 graduates of Blue Startups have received follow-on funding. That works out to 25 percent, or a rate one-third that of Techstars, but still a respectable number given the size, location and relative age of Blue Startups’ program. It’s also progress towards getting Hawaii’s startups on the radar of venture capitalists.

Blue Startups’ Kushi didn’t comment on this number either way.

“This shows that the right kind of network activity is happening,” said Fooks. “This exposure and experience is important for Hawaii. It shows that these startups are being given great opportunities.”

Value Of Venture Capital

The studies and articles mentioned all use standard startup metrics: venture capital raised, valuation and exits. If you’re a traditional businessperson outside of tech, that’s “funny money” and somewhat arbitrary. But, in tech, that’s what is used to measure a company. Revenue numbers would be nice, but these companies are all private and frequently cover up or inflate revenues, or simply don’t have any.

I couldn’t find valuations, not even estimates or rumors, of any post-Blue Startups company, so that metric is out.

Exits are easy: zero.

The value of venture capital raised is more interesting. Using my mentioned sources, I counted a total of $20.6 million in funding raised by Blue Startups grads. Blue Startups claims $26.7 million in total funding.

I found that Volta Industries has raised $12.5 million and Vantage Sports has raised $4.3 million. Blue Startups claims that Area Metrics (which is in the process of moving from Honolulu to Seattle), Flowater, and Meeting Sift have all also raised over $1 million. Others raising money, according to my listed sources, but under $1 million, are Tealet, Tow Choice, Gibi, Workers on Call, Comprend.io, Benjamin, and CandyBar.

Here’s where this all gets murky, especially for Volta and Vantage: some of that money was raised before entering Blue Startups.

While having raised money before entering an accelerator isn’t odd, it is unusual to see startups at the stage of Volta and Vantage do it. If you’re a promising startup that has already raised several million dollars, or has a few million in the process of being raised, why would you enter an accelerator?

One potential answer is you’re repaying a favor. Another potential answer is you’re supporting your hometown. Yet another is that you think the connections and mentorship offered by the accelerator will enhance your chances of success. Again, simply raising money before entering an accelerator is not unusual, and both have raised money since graduating.

I’ll leave it up to you to consider why Volta Industries entered Blue Startups after securing $3 million in funding and during the raising of an additional $1.9 million. Same goes for Vantage Sports, who entered Blue Startups in April 2014, then closed on a $1.6 million round just a month later (some of it from Maui-based mbloom).

Two other startups, Workers on Call and Flowater, also have interestingly timed funding rounds, leading one to believe that the funds were raised, or at least in process, before entering Blue Startups. So should Blue Startups take credit for helping with those funds because they weren’t “follow-on?” For marketing purposes, it’s definitely fair game, so I’m counting it.

By tech metrics, Volta Industries and Vantage Sports are ongoing success stories, as are Meeting Sift and the others. It’s all a sign of startup success, and it all reflects well on Hawaii.

“Blue Startups is definitely accomplishing a lot,” said Fooks. “We really believe in the accelerator model, and that it’s good for Hawaii. GVS Transmedia have been developing nicely for media and content ventures. Energy Excelerator is doing very well, getting national support. And Blue Startups has put Hawaii on the map. We just had their East Meets West conference (earlier in January). Their startups are attracting mainland venture capital and investments from local firms, like Ulupono Initiative and mbloom.”

If your startup benchmarks are “unicorns”, like Slack or Domo or Uber, then smaller markets like Hawaii will never be seen as successful. But if you’re realistic as to what a smaller market can or should accomplish, then Hawaii seems to be doing pretty well.

“What Blue Startups has done since their start is no small thing,” said Fooks.

With nearly 50 startups accelerated, nearly two-thirds still operating, and well over $20 million in total funding raised by their graduates, no one can argue that Blue Startups isn’t operating at a respectable level.

(Note: if you’d like to review my data, click here to view the Google Sheet.)