Saturday, May 8, 2010

Land of Extreme Rating Scales

We haven't been to our first post yet, but I'm beginning to pick up on the foreign service being a place of extremities. There is no room for grey area here. You're either all the way in or all the way out.

My first experiences of this new life philosophy is the bidding process. To recap, I'm not a huge fan of the bidding process.

To simplify, let's just say we had 100 posts to bid on. Then we divide those posts into three categories: eh, uh and oh. For example, in response to Ciudad Juarez, Mexico, I said, "Eh, no thanks." How about Manama, Bahrain? "Uh, maybe." Though I would spend all day everyday singing the Muppet's song, "Manamana," which might push this post into the next category. This is the "Oh, sign me up!" category.

So there isn't much room for waffling. All in or all out.

This has been reinforced during Natalie's language training process. She has periodic reviews in which they tell her where she stands and how this projects for her testing day. These categories also seem a bit extreme, too.

For example, there is the You-already-know-the-language-well-enough-to-pass category, which really is only there to taunt people because even native speakers seldom fall into this category. No, this is true; I've heard of native speakers only scoring a 2-plus in speaking, 2-plus in reading. (I'll talk a little more about the flawed grading scale further down.)

Next down the list is the You-are-likely-to-pass category followed by the, There-is-a-50%-chance-you-pass-or-fail category. Finally, there is the You-won't-pass, which is paradoxically not a death sentence. I've heard of people who fit this bill during review, yet passed the test. Go figure.

I have a theory about this grading scale. They tell people what they think they need to hear to inspire them to keep working. Is this person not trying that hard; tell them they are going to fail so they up the ante. Is this person a little insecure; tell them they are on pace to pass to encourage them.

I hope that my theory is right, otherwise, this scale seems rather nonsensical. Especially considering her first review period was so early on, it is impossible to use that as any sort of indicator of how some will do other than stress the foreign service officer a lot.

So what is a passing grade? For Spanish, it is a 3 on reading and speaking on a scale of 0 to 5. Here is one place, however, where they lessen the extreme grading scale system a bit, though I don't know if this was the right place to do.

For their solution to create a little wiggle room, the hot shots at Frustrating Scales Inc. (FSI) added a plus system that defies explanation. Can anyone tell me the difference among a 2, 2-plus and 3? Hell, the difference between a 2 and a 3 are borderline insignificant at best. And if you feel too limited by rating 0 to 5 that plus/minus became necessary, why not just stretch the grading scale to 0 to 10?

To come full circle, the plus system addition would have been nice for the bidding list. On our three-point scale, Maputo = Hermosillo. Going back to our theoretical 100-post bid list, Maputo would have been in our top five while Hermosillo was probably more around 30. In our top third, but definitely not equal.

4 comments:

  1. The thing that boggles me about the rating scale is that apparently most college majors in a language test in at a 1. So, focusing on a language for four years makes you a 1, but in six months I'm going to go from nothing to a 3? That just doesn't seem possible.

    ReplyDelete
  2. There were definitely moments during the bidding process where I wished it was on a 5-point scale instead of a 3-point scale. The fact that our 35th favorite post was ranked exactly the same as our 100th favorite post (or our absolute favorite being the same as our 10th) was something I wrestled with. I don't think I'd want to rank each one individually, but our mediums ranged from "Cool" to "It beats Juarez, I guess."

    ReplyDelete
  3. buhahahahahhaaha I loved watching manana. And I could totally see you singing that. Your posts crack me up!!!!!

    I can't even imagine grading people on that plus/minus scale. Government bureaucracy--pfffffffffffffffffft

    ReplyDelete
  4. I don't think State invented the 5-point language scale. I've taken a similar test before and was rated at 2+ for Russian. I know my company uses the same scale as a language assessment, and apparently it was made for evaluating translators. I've heard that a lot of native speakers don't get higher than a 3 because of their education level. You apparently have to have the equivalent of a U.S. college education in the language to even hope to score a 4, and 5's are what professional interpreters get.

    That said, the testing process itself has a some "voodoo" involved, as my husband's boss put it. They are testing for vocabulary you'd need for State, and they are more forgiving of certain kinds of mistakes than others. So even native speakers don't pass the test at State, because they aren't making the right kinds of mistakes. Or something like that.

    ReplyDelete