AcaRank

Since it's slow, subtle roll out began last month, AcaRank has quickly emerged as one of the most controversial new institutions in scholastic a cappella. The idea is to use clearly defined criteria, including performance in live competition and achievement in studio recording, to measure and comparatively rank a cappella groups.

Some parties have met the idea with excitement—particularly those that have been pleased with their groups’ high rankings or who are happy to be ranked at all. However, much of the attention focused on the project so far has been far less positive, with critics decrying a furthered sense of competition rather than collaboration in the a cappella community and questioning how objective any set of ranking could be.

Patrick Hockberger and Matt Kania, who previously collaborated as part of The Northwestern University Undertones and for VoxLabs, an a cappella finance and consulting company, co-founded AcaRank. Hockberger was kind enough to sit down for an interview with me to clarify the intentions behind AcaRank, their practices, and where the project is headed.

“We started because we had fun working on it,” Hockberger said. “We wanted to share what we came up with.” He went on to liken debates about the best a cappella groups to bar arguments about the relative merits of sports teams from different eras, positing the question of whether The SoCal VoCals were the best collegiate group in the world the years when they chose not to compete in the ICCAs. He conceded that AcaRank’s system is “not the only answer” to such quandaries, “but it is one possible answer."

In response to critics who questioned how any a cappella ranking could be objective, Hockberger explained, "The scoring system doesn’t allow you to extrapolate that the top ranked group is the best in the world right now. It only measures who has achieved the most through competitions, awards, and studio recordings. A lot of people who have had criticisms have pointed to particular groups that seem too low, like The SoCal VoCals at 16, and I understand that because they’re probably the greatest college a cappella group of all time. And if we did retroactive rankings back to 2008, they’d probably be top three if not number one for six consecutive years. But our leaderboards are about what you’ve achieved this year.” Hockberger went on to say, “There are an infinite number of ways you could compare groups—this is just one."

Hockberger and I discussed the criteria and the computations behind AcaRank. The rankings are currently based in live competition and studio recordings. “A lot of groups release an album every two years. Some compete every year, some every other year,” Hockberger said. He went on to note that RARB reviews and awards like the CARAs often don’t come out the same year as an album’s official release, thus much of the criteria for the ranking is built to straddle two years, so group rankings don’t necessarily plummet for not competing in a particular year, or waiting a year before an album is reviewed. The live competition component focuses on Varsity Vocals’ ICCA and ICHSA tournaments, but also considers competitions at CASA festivals, Harmony Sweepstakes, South Asian a cappella group competition Gathe Raho, and the Kol HaOlam National Collegiate Jewish A Cappella Competition, as well as appearances on The Sing-Off. (A fuller explanation of the scoring metrics is available here.

“I think one of the mindsets we can change is that ICCA is the one [competition] that really counts,” Hockberger said. “With ICCA, it’s so devastating when you don’t advance because you don’t get another shot for twelve months, and you don’t get to apply what you just learned. We want to encourage groups to take those things and apply them elsewhere. We reward groups that go to BOSS by improving their score.”

Based on early backlash against AcaRank, Hockberger said that they were instituting some changes. Effective with this interview, the people behind the ranking are not anonymous anymore. They are also taking steps to make their scoring systems more transparent and he said they will “encourage groups to calculate their score themselves to cross reference with what we got so we can resolve any discrepancies.”

Another important shift is giving groups the opportunity to opt out. “I know, specifically some high school music teachers have criticized us for not having a way to opt out,” Hockberger said. "Groups now have the option to have their names and scores removed from the leaderboards, and they will not be re-added in subsequent years without their permission," though he noted their scores will still be tabulated to contribute to their school’s overall score. He said AcaRank will consider more options as time goes on.

AcaRank also intends to be one of the first sources in the a cappella world to meaningfully recognize YouTube videos. “It’s one of the most popular outlets for listening to a cappella,” Hockberger said. He went on to explain that the new initiative will focus only on US collegiate groups for the 2014-2015 school year, and will score groups in categories such as soloist, arrangement and backing vocals, visual performance, audio technical, visual technical, originality, and a final subjective score. AcaRank is also invested in offering awards for various criteria, most notably Video of the Year. Hockberger would also like for the awards to encompass a component of fan voting, but details are not yet in place for that.

Hockberger explained that he envisioned AcaRank not as a stand-alone project, but rather as a platform for a number of services. He indicated that one way in which the work can have a positive impact is being demonstrated now at Northwestern where they’re working with seven groups to create a two-track-a-piece compilation CD. “I believe we will allow Northwestern groups to increase their total number of gigs and ask for more money for gigs.” He’d like to extend the model to other schools.

One additional note Hockberger made in favor of AcaRank: “The top two college groups and top high school group at the end of this year will get a prize. It's a studio recordings service from VoxLabs called the Valet Package. Winning groups can choose to take advantage of any or all of the following for free: assistance building a budget, timeline, and fundraising plan, full handling of the labor for licensing, distribution, and manufacturing, and around the clock availability via phone and email to offer advice or answer any questions about the process of making an album.” Hockberger estimated the value of the package to be roughly $400, depending on the number of tracks on the album.

AcaRank is likely to continue to grow and change, as might the questions controversies surrounding the initiative. For now, though a number of questions have been answered and The A Cappella Blog thanks AcaRank for sharing their story with us.

You can visit AcaRank here.