Many colleges and universities who are refusing to cooperate with the National Review justify their opposition with the claim that NCTQ is not being sufficiently transparent about its methodology. A more credible enterprise—they assert—would have fully disclosed more details, particularly the weight that will be given to each one of our 18 standards and the qualifications of the people doing the evaluations. 

We went ahead and released as much information as we could without jeopardizing the integrity of either the first review or those that follow.  We promised to provide much more information with the publication of the review—as we have done in all of our earlier reports. Any further disclosures—we argue—would be akin to passing out the answer sheet in advance of the final exam. Judging by continued complaints about our lack of transparency, that argument seems to have gotten us nowhere.

So we have done a bit of investigating, researching the publicly disclosed methodologies of other enterprises which would be universally labeled as "credible." Closest to home, we looked at other endeavors in the field of education, including the process by which NCATE confers accreditation on a teacher preparation program as well as the process that the College Board has in place for grading the writing portion of the SAT. We also looked at some really high stakes competitions, like Race to the Top, the MacArthur Genius grants, and one of the biggest prizes of them all, the Nobel Peace Prize.

Not one of these enterprises divulges much of anything.  Take a look.
Kate Walsh with research by Derek Wu