CAHPI(BC) Spearheads Development of H.I. Exam

[hausdok]

According to an article in Daily Commercial News, a Canadian Construction Industry Publication, the Canadian Association of Home and Property Inspectors of British Columbia has begun collaborating with the University of British Columbia to develop a more rigorous home inspectors proficiency exam. To read more, click here.

[Scottpat]

So essentially they want to make their own version of the NHIE.

When I was on the board of EBPHI, CAPHI approached EBPHI about using the NHIE in Canada. CAPHI at this time could not afford the cost that would be incurred to create their own "new" exam. Upwards of $2M have been invested in the NHIE over the past 10 years in its development and maintenance.

Arrangements were made a couple of years back and several Canadian subject mater experts(SME) have been participating in writing new Canada specific questions(metric and words that have "U" are even included) that would appear in the Canadian version of the NHIE. As with all questions on the NHIE being verifiable the Canadian questions are all verified to a published Canadian source.

I'm pretty sure that is already available or will be very soon in Canada through the testing network that already administers the NHIE across the states. I know they have several testing centers all across Canada.

I guess time will tell if the wheel can be reinvented.....

[ghentjr]

This is typical of the competing interests in the HI "profession". No one want's to be a part of something someone else put together. "They" always think they have a better way.

[John Kogel]

The use of the US based NHIE was put forward and discussed at length. In the end the members voted to keep the exam local for a number of reasons. One simple reason was that all the code rule questions have to be rewritten into Canadian. This was a few years ago. Now that has apparently been done.

But the main reason was that we will now retain ownership of the exam and the exam process. Exams generate a fair bit of income. Why give that away?

[hausdok]

Hee, hee,

Somehow I expected Scott to be the first to respond to this one.

John, I can't imagine that they're going to make gobs of money at this unless they can develop a test that gets taken country-wide. Given the attitudes I've seen displayed by Canadian inspectors toward anyone and anything not from whatever particular association they belong to, I think it's liable to end up relegated to BC alone, in which case CAHPI(BC) might find it to be an unwelcome drain on it's budget.

I'm guessing that CAHPI no doubt would like to maintain ownership of it; in which case it would end up being vilified by everyone not associated with CAHPI the same way the old NHIE was vilified (and still is) when ASHI decided to give up ownership of the NHIE to EBPHI and EBPHI stepped off on it's own as an independent entity. Even though today they are no longer affiliated with each other, you still have folks from the other associations constantly caterwauling that it's an ASHI test. Some folks in other associations are so convinced of that they refuse to participate in the exam writing sessions - kind of like cutting off one's nose to spite one's face, I think.

Do you think CAHPI(BC) will release it to CAHPI when they're done? If so, do you think that inspectors in other Canadian associations will accept it? How about the folks in the National Assessment Authority or whatever the folks promulgating the peer review process call themselves? Won't they see this as someone trespassing on something they think is their turf?

It does kind of look like they are simply reinventing the wheel. The code rule questions in the NHIE are already written into Canadian and have been since 2011; that's why a group of Canadians gets invited to sit every session now. They were shown a lot of respect by the other participants and a lot of time was spent discussing how to reformulate questions so that they worked for both the US and Canada. When questions couldn't be formulated to work for both they were put into a special category for a Canadian version of the test only.

I guess CAHPI(BC) is just looking for something to have bragging rights over. I bet this sparks another stink among Canadian inspectors that will make the contentiousness over the national peer review process look like a bed of roses.

ONE TEAM - ONE FIGHT!!!

Mike

[Scottpat]

The use of the US based NHIE was put forward and discussed at length. In the end the members voted to keep the exam local for a number of reasons. One simple reason was that all the code rule questions have to be rewritten into Canadian.

But the main reason was that we will now retain ownership of the exam and the exam process. Exams generate a fair bit of income. Why give that away?

Exams of the quality and type that is needed for a high stakes licensing exam is costly to maintain and administer, thus the profit is not a great as many think. EBPHI spends upwards of $100K a year on the average just maintaining the exam and publishing new questions to keep the exam fresh and updated.

Look at the number of folks that will need to take the exam and then multiply that number by your projected exam fee minus the cost for the administration of the exam and this is what you will have leftover to maintain the exam. I think it is estimated that there are around 3,000 to 4,000 home inspectors in Canada. I do not know how many of those are in BC.

So when your get down to the real number of folks it's really is not all that many but the first shot of exam takers will be the largest.

FYI, the Canadian NHIE is not the same exam as the one that is given in the USA, it is based on Canadian codes.

If y'all need some consulting help I'm available and I'm sure I could get Jim Funkhouser to assist as well for a nominal fee. Between the two of us we have about 12+ years of high stakes exams and home inspector exam knowledge It's been a few years since I have been to BC. []

[John Kogel]

Some of the BC members felt very strongly against giving up the testing process. No, it isn't big money, but it is historically in the tens of thousands per annum. Naturally NHIE is willing to take this on.

I don't think we are reinventing the wheel. We just chose to have one built for us, that we will maintain ownership of. I don't agree with everything my association does, but everybody got to vote.

You bet there is competition between the associations. We all claim to be the best. Nobody in the rival associations will have any interest in our test. That's fine. One of the associations recognized by the licensing board here has an open book test. Why would they want to adopt our exam?

[kurt]

If y'all need some consulting help I'm available and I'm sure I could get Jim Funkhouser to assist as well for a nominal fee. Between the two of us we have about 12+ years of high stakes exams and home inspector exam knowledge It's been a few years since I have been to BC. []

That'd be money well spent.

[John Kogel]

All CAHPI(BC) members are members of the National CAHPI association. So it is possible (I'm not saying likely, but possible) the other provincial associations will be looking at using our exam as well.

We have paid staff in the office, and they administer the exam proctoring and so on. If we gave it all over to NHIE, we would lose money on every new candidate.

We have members from the other provinces buying in to our E+O insurance program.

We exchange info with ASHI all the time, no rivalry there.

It was not an easy decision and there have been a lot of heated discussions, believe me. []

[Marc]

How can an association of home inspectors, which thrives on membership fees, be impartial to developing an examination standard that reflects what serves the public instead of home inspectors themselves?

Practicing home inspectors have a vested interest in an exam that heightens the bar, reduces the pass rate and give existing inspectors a grip on the market.

Exam creation should have psychometricians in the driver's seat and home inspectors assisting.

Marc

[hausdok]

Hi,

Marc, that's exactly how the NHIE is structured.

John, I think you'll find that having it controlled by your organization will mean it will end up getting watered down. As soon as someone high up in the organization fails it, it will be declared too difficult and then the knife will come out. According to the late Mel Chalfen, who I used to spend a lot of time talking to by phone, it happened to ASHI way back when they were new and first developed their own exam and had a peer review process. At least ASHI eventually came to recognize that for the exam to have credibility they needed to let the exam go.

EBPHI does not administer the exam, neither does the state. It's administered by whichever independent exam contractor the state chooses. The test writing panel isn't association dependent either.

You guys have already made your decision, so there's really no point in trying to convince just one or two folks on this board that this might not be a great idea. I wish you guys luck; maybe you'll be able to come up with a better mousetrap. I'm afraid though that you guys might have made your decision without knowing enough about how EBPHI works and how the NHIE is designed and administered.

ONE TEAM - ONE FIGHT!!!

Mike

[John Kogel]

How can an association of home inspectors, which thrives on membership fees, be impartial to developing an examination standard that reflects what serves the public instead of home inspectors themselves?

Practicing home inspectors have a vested interest in an exam that heightens the bar, reduces the pass rate and give existing inspectors a grip on the market.

Exam creation should have psychometricians in the driver's seat and home inspectors assisting.

Marc

"The University of BC is tasked with developing an algorithm with verifiable analytics to ensure test results reflect the competency of the candidate accurately. Just because a person can pass an exam doesn't mean they have fully comprehended the subject matter or are able to apply it properly, if the exam is flawed. Some industry exam methods fail to prove credibility of results by being too simple or are easy to beat. Applying verification analytics to an exam process will ensure exam results are verifiably accurate to the best possible level".

Marc, when I figure out what you are saying, I'll try to respond. []

[Marc]

Marc, when I figure out what you are saying, I'll try to respond. []

How's this: CAHPI shouldn't be in a position to influence the outcome of a exam qualifying home inspectors. As an association of home inspectors themselves, it has a conflicting and overriding obligation to it's membership.

Just my opinion. I've not studied this topic as well as others here. I just have an eye for conflicts of interest.

Marc

[John Kogel]

I understand what you are saying. I suppose there could be a conflict.

The organization, CAHPI(BC) is not a separate entity but a non-profit org made up of about 240 home inspectors. A new board of directors is elected every year, although we are extending the term to two years. Any major decisions are voted upon by the members.

Anyone can join by passing the tests. Nobody is obliged to join, because there are 3 other associations. We want the exam to reflect a higher standard of competency. It still doesn't prove the candidate can inspect a house properly. It is just the first step, followed by mentoring and a test inspection of an actual house. Of course this is all run by the members. No one else is going to do it. The government agency that issues the expensive license has handed the responsibility of testing back to the associations. We are trying to raise the bar by improving the quality of the exam.

[Scottpat]

I understand what you are saying. I suppose there could be a conflict.

The organization, CAHPI(BC) is not a separate entity but a non-profit org made up of about 240 home inspectors. A new board of directors is elected every year, although we are extending the term to two years. Any major decisions are voted upon by the members.

Anyone can join by passing the tests. Nobody is obliged to join, because there are 3 other associations. We want the exam to reflect a higher standard of competency. It still doesn't prove the candidate can inspect a house properly. It is just the first step, followed by mentoring and a test inspection of an actual house. Of course this is all run by the members. No one else is going to do it. The government agency that issues the expensive license has handed the responsibility of testing back to the associations. We are trying to raise the bar by improving the quality of the exam.

A little information on high stakes exams:The difficulty of an exam does not make it a good exam. A good exam should test an individuals knowlege of the subject matter which is usually defined by a role delineation study of that profession. The difficultly of the questions that are written come from what is called an Angoff score or rating that each questions undergoes during review. Each question will be given an Angoff score and this denotes their level of difficulty based on the degree of knowledge that is being tested. Many "difficult" exams have poorly written questions, such as questions with double negative and unfeasible distractors.

Most high stakes exams test recall knowledge and learned knowledge. Learned knowledge usually reflects and identifies a more advanced type exam.

Once you have all of the questions the psychometricians will compile the data/Angoff score and produce a "cut score". The cut score is the score that is needed to pass the exam. This is how you manipulate the difficulty of the exam.

[kurt]

Yes.

I took part (briefly) in developing the NHIE. I was a guinea pig that answered questions for the psychometricians. For a test to hold any validity within those learned groups that qualify this stuff, and accredit it to the world, it's a really complicated, expensive, and time consuming path. Really.

The canuck's got no steenkin' idea about what they're getting into. If what they're working toward is something that's only recognized in their teeny corner of the world, OK. If they're working for something bigger, they're going about quite back-asswardly.

[John Kogel]

Thanks, Scott, I would certainly welcome your input as we flounder along with our half-baked good intentions.

Mike, I missed your post before. Yes, the decision has been made and, yes, minds were made up with minimal information put forward.

Have you ever seen a room full of home inspectors come to a mutual agreement on anything? []

We needed a new exam, we explored the NHIE offering, and it was voted down. I hope to see something good come from this, in the form of a reasonably improved exam.

Passing that exam will still be only the first step to becoming a qualified inspector.

[hausdok]

I understand what you are saying. I suppose there could be a conflict.

The organization, CAHPI(BC) is not a separate entity but a non-profit org made up of about 240 home inspectors. A new board of directors is elected every year, although we are extending the term to two years. Any major decisions are voted upon by the members.

Anyone can join by passing the tests. Nobody is obliged to join, because there are 3 other associations. We want the exam to reflect a higher standard of competency. It still doesn't prove the candidate can inspect a house properly. It is just the first step, followed by mentoring and a test inspection of an actual house. Of course this is all run by the members. No one else is going to do it. The government agency that issues the expensive license has handed the responsibility of testing back to the associations. We are trying to raise the bar by improving the quality of the exam.

A little information on high stakes exams:The difficulty of an exam does not make it a good exam. A good exam should test an individuals knowlege of the subject matter which is usually defined by a role delineation study of that profession. The difficultly of the questions that are written come from what is called an Angoff score or rating that each questions undergoes during review. Each question will be given an Angoff score and this denotes their level of difficulty based on the degree of knowledge that is being tested. Many "difficult" exams have poorly written questions, such as questions with double negative and unfeasible distractors.

Most high stakes exams test recall knowledge and learned knowledge. Learned knowledge usually reflects and identifies a more advanced type exam.

Once you have all of the questions the psychometricians will compile the data/Angoff score and produce a "cut score". The cut score is the score that is needed to pass the exam. This is how you manipulate the difficulty of the exam.

Hi,

Well, fortunately CAHPI has already done a very extensive role delineation study; the best I've ever seen regarding what we do. It would make an excellent matrix for a proper peer review process except that their idea of what constitutes a peer review is all dinked up and, from what I've been able to see, is pretty subjective.

Like I said, maybe they'll come up with a better mousetrap.

Might as well wish 'em luck and drive on.

ONE TEAM - ONE FIGHT!!!