Scientists Reconsider what they think they know about autism

Been thinking on this awhile. I hate IQ tests. IQ tests were invented by people (the US Immigration) who are good that them to distinguish people that they like from people they don't like. They've morphed since inception but still hold to same ridicuous checks and balances based around pattern theory, numeracy and literacy. When people generally mention IQ as a measure of this ethereal facet of human cognition, I ask them three things:

1. What is intelligence? And see if they manage to get two out of th 8 facets the vague and babbling OED manages to conjure.
2. Does doing the test make you more intelligent?
3. If I made a computer program that could do the test better than any human, would that make it more intelligent?

I normally get 'No' for 2 and 3. Although you can do better in the tests, the more you do. I remember my sister doing a lot of IQ tests at school and her IQ hitched up each time. I can write a computer program that will pass most IQ test, given data and time. I can also write a computer program that can statistically organise many different sources much faster and more accurately than a human can and without bias. I assure that these computer programs would not be considered 'intelligent' on any human standard.

In the end human intelligence is subjective and comparitive, you just know when one person is more intelligent than another. Someone else might think otherwise but given a large enough sample set, you get a human baised meaure of intelligence.

I mention this because:
Among all autistics, 75 percent are expected to score in the mentally retarded range on standard intelligence tests — that's an IQ of 70 or less.

What the fuck are we doing using a test like this anyway? It's a ridiculous measure of intelligence and these sorts of comparisons are pointless. Humanity likes to boil everything down to statistics and in cases where human ability is not easily measured, they should just not use them but decide on a more suitable means of understanding the mind. It's widely known that people with autism often display incredible feats of recollection or memory, is that retarded? By the IQ test, they certainly are because they can't see a relationship between "bat" and "cat" but they certainly have an unmeasurable gift.

As a scientist, a good maxim is that if the test doesn't yield the results you were looking for, either you are wrong or the test is. In this case, I think the test is wildly wrong. There may be a correlation between people that we believe are intelligent through our human biased judgement and those that get high marks in the test. There are also people who slip through the net and we don't know why... they appear to be amazingly intelligent but could not get high IQ scores. Surely the test is so generalised that it's not applicable here.

Good luck to her, she's played a blinder.

Its a comparative measurement, and in most cases it does a fair job. its no better or worse than other measurements used to define people (and agai I will revert to BMI, which is nothing more than an arbitary benchmark...in and of itself it means nothing, but allows comparative measurement against others...the health risks they link to high BMI scores are utterly dependant on the individual ultimately).

A quick check on Wikipedia indicates IQ tests were initially developed to help identify children who need extra help at school, and it probably does that fairly well. By that means it is indeed a score of how well you can do school exams, and given that for most people who may be given an IQ test a formalised schooling is also an inevitability, maybe it has a purpose.

I know of at least 1 perso who got to a fairly old age (35) with a severe learning disability without it being diagnosed. A couple of tests later and he was given the help he needed. In that context how is a test bad?

You're putting too much importance on IQ, Pete. Please answer my three questions listed in first post. I've found it's the best basis of any argument. If you think it's useful then you're probably just making the results match what you think intelligence to be. "We're expecting X, we get X! Hurrah, isn't our test great?". As a measure of intelligence, it's rubbish.

People who argue that IQ is really useful tend to be those that score well in it.

P.S. I am really well versed in this.

Gravity is a repeatable test. IQ tests aren't. The tests Newton did were in regards to the acceleration of objects falling under the force created by large objects. It was all well defined and repeatable. It's been repeated in physics classrooms for years. The bounds of all of newton's laws are distinct.

Not for IQ. IQ is supposed to test for intelligence but it doesn't. The bounds it sets is 'human intelligence', so you do have to care and it does matter because you need to know what it is your testing for. The problem with that is that we don't actually know. If you did you, when you get results you can say "I tested for this and this is what I got". If you don't know what you're testing for, the test is shit. IQ doesn't know what it's testing for. It tests using a set of games that people think are a good judge of intelligence. It's based on human bias.

By getting better at the test, you're actually getting better at the game, which is not making you more intelligent - it's just making you better at the game. Is that improving your memory, recollection, ability to abstract, logical reasoning and all the other facets or is it just making you better at playing the game?

It's been shown that chimps can do better at some visualisation tasks than human, they concetrate better and have better spatial awareness. Are they more intelligent than humans? No, of course not. But why? Because they don't have language or numeric skills. What about abstraction, how is that tested? Classically missing from the test because it is something we assign to intelligent beings but we can't measure.

To take your last point, if they ever do make a measure for intelligence, they will see that IQ was woefully inadequate as a test for anything but the ability to play the game. Unlike subatomic physics, which respects Newton's laws for its continued correctness on large scale objects. A different set of laws may now be applicable for the very small but they are not refined laws, they are just set within their correct parameters.

And this does matter when the IQ test is used to classify and cluster people. It's inappropriate and misleading.

So how do you measure fitness? Thats a physical thing...

Sorry, my answer should have been:

Define fitness.

I knew you'd say that, here's my pre-empted reply.

We're approaching an argument on measurability and observability or worse, science as a religeon. In control theory (which applies here), there are things you can observe and measure, such as your height and weight. There are things you can't measure but you can observe and then there are things you can't observe or measure. You're always looking to be able to observe and measure things to tell what any system is doing.

However, because systems often have outputs that are linked to other things you can't observe, you can measure the things you can observe and draw conclusions on those things you can't observe. It's called an observability criterion.

That's ok for physical aspects. You can set up tests that directly measure certain aspects. How fast you can run 13 miles, for example. You'd set up a test to run 13 miles and time it using something suitable. Of course, to be statistically significant in a deterministic (which is different from non-repeatable) system you have to run 13 miles a few times and take an average. You know the test is measuring something definite - your time to run 13 miles. It might be slightly different as time goes on or on other factors but the test itself is always the same. 13 miles doesn't change, time doesn't change.

To then work out what's going on with the things you can't observe, you get experts to make the links between observable things and non-observable things.

That's the bit where IQ falls down. For a start, you don't actually know what it is you're trying to measure. You don't have a "time". You have a series of games you've made to get someone to do. You have no idea if that is actually measuring some facet of intelligence of whether it's just playing a game. As EMW says, being good at the game (or algebra) only proves you know the rules and are good at the game. The MASSIVE LEAP OF FAITH (science as a religeon) is that you believe the 'experts' can make direct ties between intelligence and the games you play. The links between the games and facet of intelligence you can't observe or measure.

IQ is not even a remotely a good test.

The 13 mile run is one test, one of many. The matching-shapes part of the IQ test is one of many. I can see why you think they are identical.

BUT!

And this is where you're completely mistaken. It's reasonable to assume that running 13 miles quickly is one test that could be appropriate to measuring fitness...

BUT!

No-one in their right mind would think that the silly games that IQ tests make you do are actually appropriate to measuring intelligence. If you do, then you're going further down the road of "science as a form of faith" than I and EMW would ever go.

I don't think a 13 mile run is a good test of fitness at all. There are people far fitter than me who I fully expect to beat on Sunday. Benjie (lad I train with) is younger, stronger, better over short distances, more flexible (did about 4 years of martial arts), and has been training for about 6-7 years. He's just not good at long distance running, and I have focused almost exclusively this year on endurance and speed. He's fitter, I'm more focused. I'll win though...

The same could be said for any physical fitness test you care to mention, however it does not make the tests in and of themselves pointless.

//added. As Rob's thrown in the towel, I'll do the final point now, and highlight that you are treating the concept of a half marathon in exactly the same way as an IQ test. Its an relative indication of fitness/intelligence, while at the same time being utterly inaccurate if you're interested in that area. In both cases you are measuring one thing to do a comparative on another. Neither are useless tests, while at the same time neither are definitive...

Of course, the only real way to measure intelligence is to roll 4D6, re-roll 1's and 2's, then take the highest 3 dice.

Of course not, I just like arguing, and being belligerent. I can't give a toss about IQ test one was or 'tother :-D

Running 13 miles is a good test of being able to run 13 miles. A sprinter would really struggle over the same distance...however by no definition are they unfit. Swimmers wouldn't be good at it as they wouldn't have the resiliance to the impacts. Likewise I would struggle in any swimming event, as I'm a shockingly poor swimmer (its all upper body)...does that mean I'm unfit?

Its _a_ test of fitness, a decent yardstick, and performance within the bounds of that test is a reasonable marker of overal fitness compared to others, however its not all encompassing, and has several caviats attached...just like an IQ test really.

I think the definitions of what intelligence or fitness are need to be defined more presicely before any test can be designed.

If fitness is defined by being able to run long distances quickly, then a 13 mile run would be an accurate test for fitness.
If fitness however was defined at being able to produce a certain amount of energy for a given period of time then a measure of calories burned over time and results produced would be more accurate. Pete is right, using the measure of a 13 mile run would not be a good test to determine how fit a swimmer was and would not allow comparision between two athletes of differing fields.

Both could be defined as fit, both could also be be compared, but as yet the definition of fit has not being made to any greater than extent than a subjective qualatative statement.

Until that definition is made then a bleep test, a 13 mile run, a 100m sprint or a biathlon are equally good (and poor) measures of how fit a person is.

The same works for intelligence. Is higher intelligence measured by how quickly a series of mental arithmatic questions are solved an aspect of IQ tests? Why not? It is valid to say that this would show that someone possibly has quicker mechanisms in place to think through problems. It is also valid to question this as a measure of intelligence. A computer can perform calculations quicker than a person. Is it more intelligent? Most would say not.

Yes there are various people that have put forward what counts as intelligence, but these are multiple and differing.

You are both arguing that the tests are not accurate measures of either mental or physical fitness. I say that the tests are good at testing the aspects chosen to define this fitness. I question that the aspects chosen to define the fitness are the correct ones.

At best I say they are indicators and nothing more. They are qualatative not quantative. Until you accurately define what "fitness" is for either aspect I say that this will not change.

But there is also no definition of "fitness". Yet many tests exist to a very small part of teh all-encompassing field, and within the scope of what they are testing they are valid. IQ tests are no different in that respect.

Consider Pandoras Box cheerfully open!

My camp is that you don't need to fuly define a subject in able to be capable of doing comparative tests on aspects of it.

I'm goin to move away from the 13-mile run as a test, as its a shockingly bad and overly specific example by anyones standards, and instead I'll use something called VO2Max (effectively a determination of how much oxygen you can utilize in a period of time...the higher, the better. Its impacted by high intensity training) as its far less specific, but still reasonably well tied to performance levels. Many athletes however (field events, sprinters (who don't use areobic respiration for their events), weightlifters, in fact anything with the focus on anaerobic/explosive energy expenditure) will have a low score, so its not a _complete_ measure of fitness.

So, on one hand you have an IQ test, which test some aspects of (and I may be missing some stuff here, its a while since I did one) pattern recognition, word association, language, spacial awareness/shape puzzles. Practicing in these areas would increase your IQ. Ingesting a theasuarus would aid imensely, if my recollections are anything to go by (there were some words used I didn't know the meaning of). It doesn't cover language, abstration, and many other areas, so is not a _complete_ measure of intelligence.

So, we have tests which are clearly not a full measure of their test subject. They are, however, still useful at comparative testing, and for those people who just the relevant skills in their life. Just as not all aspects of fitness are important to everyone (I, for instance, don't give a shit about strength and explosive activity), I also don't need language skills of exceptional level, and average spacial awareness is good enough.

So, in summary, I'm saying you don't need the definitions, just the defnintion of what the test covers...

No it isn't. It's like saying "This person scores highly on our standardised tests, which measure ability in some aspects of generalised reasoning ability, therefore we predict that, all other things being equal, he will perform above the average in similar reasoning tasks in the future." Which is what an actual psychologist would say. Let's not muddy the waters with an appeal to ridicule; this thread has been marred by enough bad arguments already.

No serious psychologist believes that IQ test results are conclusive in determining someone's "intelligence". In fact, pretty much every psychologist I've known has avoided using the word "intelligence" entirely, since, as has been pointed out, it's pretty much meaningless from a scientific perspective (and far too emotionally loaded a word to be used without prejudice.) What psychologists talk about IQ being a pointer to, is someone's generalised (or just general) reasoning ability. And even in that case, when talking about individuals, the data would almost never be taken in isolation, without context or other data sources, and absolutely never without taking into account the well-documented cultural and social biases in the test. Biases, incidentally, which were documented by psychologists who continued to consider the test useful.

And IQ tests, used carefully, are useful; test results are highly consistent in any one individual and variable between individuals, so it's definitely measuring something. It's possible that what's being measured is nothing more than someone's ability to do IQ tests, but that hypothesis is called into question by the fact that test results correlate strongly with academic success, professional success and financial success. This suggests that that whatever is being measured is fairly strongly related to something which is important to people's ability to achieve their goals in our environment. We don't know why that correlation is, but Occam's razor (talking of things that have to be used carefully) suggests that the correlation exits because the reasoning ability used to score well in IQ tests is the same as the reasoning ability we use to achieve our day to day goals. Even if that's not the case, the correlation remains, and makes the test useful, even if only as a proxy measure.

It's true that we really have very little idea of how the brain works, especially higher level functions, such as problem solving and reasoning, but that doesn't mean we can't measure test results and make statistical use of those results where they're shown to be useful and reliable. It's also almost certainly true that human cognitive ability is vastly too complex to be measured on a single axis, but it doesn't necessarily follow that measurements of a single axis therefore have no value, and, in fact, the IQ test is designed to attempt to cover a few different skills, in an attempt to improve it's utility. As Matt points out, this can be considered a weakness, since it could cause confusion, or mask variability within the test if the results were considered solely as a whole, without data about specific section results. This is often the case with those "test your own IQ!" books, or magazine articles, but much less common in actual studies.

What it comes down to is more or less what Pete's been saying from the off, which is that no, the test isn't perfect, but it doesn't have to be; it's not a binary decision; we don't have to say either that it's 100% correct and predictive or that it's useless and without value. The fact is that IQ tests do have value when used carefully and with due consideration. They don't give the whole picture, but that's OK, no-one expects them to; they're one of many tools in the psychologists' toolbox, which between them give us some insight into human cognition and cognitive ability.

(And yes, the toolbox of Psychology as a whole is woefully incomplete, but if we cast aside the tools we do have because of that, we'll never make any progress at completing it at all.)

Aggro,

"The fact is that IQ tests do have value when used carefully and with due consideration. "

I think the real issue is that the reasonable consideration i've quoted is not the general opinion, a straw poll of the public would say that an IQ test is a test of intelligence, not a test of logical thinking that excludes any measure of abstract thought.

Indeed, if your point is just that we need to better educate the public about the use of such tests, then I have no disagreement at all.

All other things are never equal, whatever you're measuring. A major part of the scientific endeavour is the attempt to reduce the impact of uncontrolled variables on tests. If you want to throw IQ tests away for running that risk, then you need to ask some serious questions about the rest of science.

This case tells us very little about IQ tests, beyond what I've already said; that the data can't be taken in isolation. It's true that Autistics score disproportionately low on IQ tests, and that, for them, it's not representative. This isn't really news. I know the article makes it out to be earth shattering, but I was taught it at University in the mid-nineties, so somebody has obviously known for longer (not about IQ specifically, but that it was a scale of structural differences with a range of cognitive effects, rather than specific mental defect to be treated.) The point is that this doesn't invalidate the IQ test; it doesn't mean the test is somehow unfair; the reason autistics score low is that IQ tests happen to measure things that they really are bad at. It's the same for dyslexics. As long as you're aware of that context, the test remains useful (and to a lesser degree, even if you're not.)

In fact the very fact that certain conditions skew the results of tests like IQ can be useful in detecting people with those conditions. In the case of autism, detection isn't usually a problem, but having a baseline method of rating generic reasoning allows us to ask interesting and potentially fruitful questions like "why is this test which is representative of most people, so woefully unrepresentative of people with this condition?"

It's true that autistics have traditionally had a very bad time of it, but I'd suggest that it's their inability to express themselves and their thoughts to others that had led to them being branded retards, rather than their low IQ scores.

This is the extreme example but even for the less extreme cases it would mean you could not rely on an IQ test to give anything approaching directly comparable results.

Not necessarily. The very definition of extreme is 'beyond normal bounds.' We expect extreme cases to deviate from normal models, that's why we call them extreme, but there is no reason at all to assume that that invalidates those models in the normal case. The question is where that line is drawn. As I've said before, statistically speaking, the IQ test is quite useful in the normal case, when we approach it with the care it requires.

To match coloured shapes, you need a perception of shapes, not just reasoning. Match similar words, you need language, not just reasoning. To match numbers in a row, you need numeracy, not just reasoning. They don't test reasoning, we don't know what the brain is doing.

What if your definition of reasoning includes those things? You don't need to know precisely what the brain is doing in order to measure it's ability to solve certain categories of problem, which is all the IQ test claims to do.

You can make up statistic correlations with people doing well in society and that might be nice for the 95th percentile but for the outliers, like those in the original article, where IQ is quoted so quickly as a measure, it means nothing.

No-one's 'making up' statistical correlations; they're right there in the data. It's true, that there will always be outliers, for whom the correlation doesn't apply, and that we need to guard against making unfounded assumptions, and I never disputed that.

That's science as a faith and is INSANE. You use the phrase "Suggests" rather a lot. How many hierarchical assumptions are you going to make before the who tree falls over?

Woah - careful with the ad hominems there, mate: you'll have someone's eye out.

I use the word "suggests" because, in science, that's all evidence can ever do.

... and a whole load of assertions of bias, (even dangerous bias,) that I think were covered in my first post (ie, who do you think detected that bias, and if they detected it, don't you think they might be able to correct for it?)

"You don't need to know precisely what the brain is doing in order to measure it's ability to solve certain categories of problem, which is all the IQ test claims to do."

I agreee that you don't need to know what the brain is doing but by doing problems, you're not solving "Categories" you're solving JUST THOSE PROBLEMS. Furthermore, the IQ Test claims to do much more - it's an intelligence quotient. If you mean IQ Test in the hands of Psychology Professors then fair play. You still don't know what areas of cognitive intelligence the test actually excites. You have no idea. The professors don't. You don't. I don't. We don't. There are 'n' variables involved here, where n is a very, very large number indeed. To properly reduce the effects of the variables, you'd need to invent an almost infinite number of tests, where the effect of each test is statistically insignificant. Even then, you still don't know what part of intelligence it is your testing for.

Do you really think that these simple pattern matching tests are a good measure of any part of a human's intelligent cognition?

No-one's 'making up' statistical correlations; they're right there in the data.

Right. There. In. The. Data. A correlation is made up. You take two sets of data and you draw conclusions. You choose the data. The data isn't generated automatically. I'm not going to let you have that. In general, I do agree that people who appear to have high IQs do tend be better at IQ tests but that's not what intelligence is about and turns the argument so vague to be pointless. As useless as the test.

detected that bias, and if they detected it, don't you think they might be able to correct for it?

No. Because you need to know what the bias is before you detect for it and there isn't the funding in the world to create a test that is free of bias. Furthermore, how do you correct for "Cultural Upbrining" in a test and still test for memory retrieval? You can't. You can try and create a test without words or free of culture but if someone who lives in a place where red is not a popular colour, your coloured shapes test will put them at a disadvantage. What about removing the bias of doing tests? Actually sitting and performing them? How do you remove those bias? You can't.

Which brings me neatly back to machine learning, which operates without bias and often produces results that are unpleasing to humanity so are modified by researchers pressed into keeping their paper count up for the next RAE.

No. Because you need to know what the bias is before you detect for it and there isn't the funding in the world to create a test that is free of bias ... How do you remove those bias? You can't.

Spot on; I miswrote what I mean to say, which was take account of those biases. Obviously, some known biases can be corrected for statistically, others cannot. In the places where no correction is possible, the only option is to not use the test in question. That's often, but by no means always, the case with IQ tests. Unknown biases remain just that; unknown. Hopefully rigourous experimental design and appropriate use of controls will detect them. It would be a bit weird to throw away an entire test on an unsubstantiated assumption that they're there.