HomeCareer Management

How to evaluate certifications

Like Tweet Pin it Share Share Email

Capitalism, Winston Churchill might have said but didn’t, is the worst system of economics … except for all the others.

Its virtues are celebrated whenever politicians speak and it has many, not least its ability to balance supply and demand, including demand nobody knew existed until someone invented a gadget to satisfy it.

On the other side of the ledger is its susceptibility to feedback loops, both positive, leading to bubbles, and negative, leading to economic depressions.

I just thought I’d share that cheery thought. Bubbles and depressions have nothing to do with this week’s topic.

What does have a lot to do with this week’s topic … certifications and what to do about them … is the corrupting influence capitalism has on so much of what it touches, the desire for wealth being the root and all that.

Take any truly meaningful certification and you can bet those responsible for maintaining its integrity are either insulated from the economic impact of the certification or benefit from keeping the certification restrictive.

Start with something simple — grades. Once upon a time I taught an IT-related topic in a local university’s graduate program. I awarded A’s to those students who excelled, B’s to those who did well, C’s to those who achieved a basic level of understanding, and D’s to everyone worse.

The Dean asked me to change my grading. Why? Most of the program’s students were eligible for tuition reimbursement from their employers, but only if they maintained a B average. The school’s revenue depended on lax standards.

Now, a related development threatens to make earning a college diploma a deeply meaningless achievement. Increasingly, graduation rates are considered an important measure of a school’s worthiness.

KJR hasn’t delved into the seriously dull subject of metrics for some time, so as a reminder, there are four metrics fallacies: Measuring the right things wrong; measuring the wrong things, right or wrong; failing to measure something important; and extending measures to those with a personal stake in them.

This one’s easy. Once colleges and universities are assessed based on graduation rates, they’ll have three obvious courses of action: Make admissions more selective; educate students better; or A’s for everyone!

Which looks easiest and most certain to you? Yup, and easy-and-certain is a good predictor of future behavior.

The target graduation rate for colleges and universities seems to be around 90%. For contrast, the U.S. Air Force Academy only passes 75% or so, which makes sense once you figure whoever passes might be the one you rely on to shoot down the enemy plane shooting at you.

Then there’s the Bar examination. Rates vary widely by state, from 41% in Louisiana and California to New Mexico’s 85%. The lawyers who make up state Bar Associations benefit from restriction, as everyone who passes increases competition in an already overpopulated field, and everyone who passes without being fully qualified further discredits a field with a poor reputation.

Compare that to the MCSE. Oh, wait, you can’t, because that little statistic doesn’t seem to be available. My guess: Those who teach Microsoft technology benefit from high pass rates. Microsoft benefits by having a large population of IT professionals certified in its technology.

But both rely on a perception that receiving the certification is a difficult achievement (for all I know it is — I’ve never tried to earn one.) Publishing pass rates would let everyone know where test administrators draw the line.

Not to pick on Microsoft or the MCSE — it’s mentioned here because it’s well-known, not because it’s better or worse than any other vendor-managed certification.

Quite a few respondents challenged last week’s contention that certifying bodies should ban the use of certifications as a qualification for job applicants. And I have to admit, that’s probably too extreme: There are plenty of fields where the certifications are downright reassuring, from medicine to the law to construction to beautician (okay, I admit, I don’t actually get that one).

So I’ll back off a bit. Instead, before you use a certification to screen applicants for a position, look carefully at the incentives associated with its administration. A shortcut: Whatever else, certifications administered by independent non-profit associations are more likely to be focused on protecting the integrity of a profession than those administered by for-profit vendors.

And if your plan is to receive a certification, including a college diploma, some related advice: Given a choice between studying for the exam and for gaining actual knowledge and ability, go for the knowledge and ability.

Both will help you get certified. But only one will help you in the job once you get in the door.

Comments (11)

  • Both will help you get certified. But only one will help you in the job once you get in the door.

    Uh… you skipped the step, there: Only one will help you get the job in the first place!

    Sure, maybe you know you could do the job, but that’s not much help if you can’t convince someone to hire you.

    Yeah, it’d be nice if hiring managers had an easy (cost-effective) way to assess “actual knowledge and ability”, but as it is, given a choice between a 22-yr-old who’s been working McDonald’s and studying at night, but hasn’t earned a diploma; and another 22-yr-old who’s been working McDonald’s and studying at night and DID earn the diploma, who do you think is going to be hired?

    • Answer: With a good manager, the one who is more persuasive throughout the selection process that he/she can do the job. I agree – the certification is useful in passing the HR screen. But very few hiring managers will base their hiring decision on whether an applicant has a certification. It might be a part of it, but I doubt it’s ever even an important ingredient … other than as a screen.

  • You took a long time getting to the point. But a worthwhile point it is. Education started out, and still should, be about learning and learning how to think.

    Keep it up

  • Certifications wind up being useless because you get what you measure for – fair summary? Whether vendor supplied or “open” like CompTIA / Linux LPI, I give Novell credit for proving certification is a money maker. Just like for-profit colleges with someone else paying the bill (employer/Federal GSL), Novell extracted more money from businesses over and above their nominal product line. Good for them! College Board has made a lot of money that way, too.

    A Bachelors in IT is functionally worthless as anything but a differentiator for HR, the MCSE is equally worthless – I have both (#11686). The problem for non-degree/non-certified IT is the schmuck loudly proclaiming “I’m too good to sit for a certification” is quite possibly the same low quality as the certification bootcamp paper MCSE – but at least HR can point to SOMETHING on the CV. I very much agree with your closing paragraphs. If you know the technology the tests are easy, for the MCSE I took all 6 in a morning. We do harder stuff every day in IT than the tests.

    It comes back to attitude as much as anything. You don’t want a papered zombie, but an a-hole talking loudly about how smart they are is not an “independent thinker” they’re an a-hole.

    Regards,
    Brian

  • If beauticians weren’t certified, television might be filled with ugly people. Imagine the horror.

    Perhaps the reason they scientifically justify having certs is to ensure that whatever beauty they apply only goes skin deep. 😉

    • My father was the only “white-collar” member of his family, and my siblings and I are the only “white-collar” members of my cousin’s generation. (My cousin’s kids have Ph.D’s et al.) So I always jump to the defense of those with a “skill,” rather than a “profession.” Ironically, apart from doctors and nurses, most “professionals” are less likely to cause you physical harm than “crafts-people.” You want your plumber, carpenter, electrician to know what they are doing; their mistakes might cost you more than money. Similarly, “beautician” is someone who puts chemicals on your face and head — do you want someone doing that to be untrained? They also need to know about diseases (micro-animals) and actual animals (lice, et al.) that can be transferred from one head to another by improperly sterilized tools. Actually, the difference between a good and bad certificate may matter more in the beautician space than it would in most computer programming spaces. So let’s have a little respect for those folks that cut your hair.
      Rollie Cole, PhD, JD
      Founder, Fertile Ground for Startups, Small Firms, and Nonprofits

  • While I’m not sure if this is related to your tenure as a professor at a traditional university, but it seems to always be a common thread in discussions about grades or qualifications. My experience shows that folks engrained in traditional education always leave out the most important idea when it comes to grading or certifications: standards.

    Notice what you wrote: “Once upon a time I taught an IT-related topic in a local university’s graduate program. I awarded A’s to those students who excelled, B’s to those who did well, C’s to those who achieved a basic level of understanding, and D’s to everyone worse.” You wrote “I awarded (actually you wrote “I awarding” but I knew what you meant) but you failed to mention anything about the standard the student should have been judged by. Without standards and objective measurement we maintain this nonsensical “grade curve” as if it’s real. If I write an objective, teach the students effectively and they study, they SHOULD be capable of making A’s.

    Instead we have a system of over-entitled professors who have no training in courseware or test development. Let’s not even discuss their course syllabus. Having seen what these professors call tests, my faith in our system of higher education has always been dubious.

    Why does all this matter? Because this is the exact problem with certifications! Each vendor hires people who write tests but who appear to have had no training in writing tests. In addition, where are the objectives and standards they expect from those who are certified? Studying “everything” in a 500 page book is ludicrous and hoping it’s on the test is just plain ridiculous!

    I highly recommend the Instructional System Development (ISD) process as the standard everyone should use when developing courseware and tests. To expect certifications to be meaningful without a standard gives us the system we currently have.

    -Signed
    Degreed, Certified and Certifiable!
    Bill Yohman

    • The standard by which I judged the students? Mine, based on their comprehension of the material as evidenced by examinations that included both multiple-choice/short-answer (objective knowledge) and essay (less-objective knowledge) questions, along with a research paper.

      My opinion, for whatever it’s worth: Objectivity is over-rated, when grading students, and, for that matter, when assessing employee performance. Since the subject is grading students, here’s the problem with objectivity: It encourages teachers and professors to ask only those questions for which there are objective answers … memorization of facts, that is.

      But what’s important is comprehension – understanding concepts and how they fit together. If there’s an objective way to assess that, I’ve never found it.

      – Bob

      • “Since the subject is grading students, here’s the problem with objectivity: It encourages teachers and professors to ask only those questions for which there are objective answers … memorization of facts, that is.”

        Au contraire! Having objectivity has nothing to do with learning level. The fact that someone would believe that multiple choice/true false are the only way to measure learning shows they know nothing about testing or courseware development. Maybe they should take a couple of courses in Instructional Systems Design. It’s very comprehensive and is a constant feedback loop that provides mechanisms for objectivity measuring student performance. Surely we should expect professors to not only understand their chosen field but by virtue of “professorhood”, shouldn’t they also know HOW to educate?

  • This really hit home! I gave up teaching at the college level here in Ontario many years ago when over 95% of a class wanted a diploma and were not the least bit interested in gaining knowledge and abilities. This attitude was encouraged by the funding formula for the colleges – money given per student attending. I sat in a meeting where the Dean encouraged us to drop any part of a course that caused the students frustration as they might drop out and the college would lose funding. I resigned shortly after that meeting.

    Now that I am in the position to hire developers I ignore any certification given by our education system and instead ask the candidates questions where their answers can indicate evidence of logical thinking and problem solving ability.

  • “Both will help you get certified. But only one will help you in the job once you get in the door.”

    In many cases, only the certification will get you past the HR filters and into the job interview.

Comments are closed.