According to an annual survey produced by Roy Morgan, doctors are one of the most trusted professions out there. They certainly make ministers of religion pale in comparison.

I reckon this is right. I put my life into my doctor’s hands. If I didn’t trust him I’d go somewhere else. It’s even more pronounced when you arrive at the ER – you don’t even know the doctor treating you there but you trust her implicitly. Why wouldn’t you?

But just because we trust doctors in medical situations doesn’t mean that the entire medical profession gets a free pass with everything. Actually let me narrow that right down. I have one particular set of medicos in mind

Children raised in same-sex families develop as their peers in families with heterosexual parents do, a group of senior pediatricians and adolescent health experts says.

And the group has called on the medical community to debunk “damaging misrepresentations” of the evidence being used by the “no” campaign in the postal vote on same-sex marriage, saying the real public health risk comes from discrimination.

They’re doctors. We should trust them, right? After all they’ve done the research…

The 13 specialists, who include Professor Frank Oberklaid from the Murdoch Children’s Research Institute, and Associate Professor Michelle Telfer, from the Royal Children’s Hospital in Melbourne, write in the Medical Journal of Australia: “The research tells us that children and adolescents with same-sex parents are doing well, despite the discrimination their families endure. This will not continue for long in the face of hostile debate.

Except, when you actually look at the research paper [pdf] (publicly available through a free registration on the AMJ site), we begin to see the cracks…

Methodological challenges

It should be acknowledged that there are methodological challenges in answering questions about children’s wellbeing in same-sex parented families. Samples of children with same-sex parents tend to be small, and many population-based studies do not ask for information on parents’ sexuality. This means that much of the research on same-sex parented families has used convenience or volunteer samples that can be biased. Despite this, the consensus of the available, high quality research is that children raised in same-sex parented families do as well as other children.

For a discussion of the methodological approaches within this body of research, see Dempsey.

So what did they do? Well it’s a summary paper. They took 3 other studies that summarised other research in the area and then they summarised the summaries. And they announced the “results” a couple of weeks out from the end of the postal survey. Politics over-riding academic process? Shame on you for even thinking so.

We aren’t given the detail of their statistical work on the public website. We have to trust them on it at this stage. But we don’t need to guess. Douglas Allen’s large-scale 2012 survey of the Canadian census [pdf] begins with a helpful survey of other research in the field. Even more helpfully he notes where the research is a genuine random sample, whether the data is “hard” or “soft” (i.e. hard is a clearly objective measure of child welfare such as graduation rates (which Allen uses) or suicide attempts etc whereas “soft” can be as soft as asking same-sex parents whether their own kids are “doing well”). He also shows us the sample size (which is very, very important  – if you want to make sure you’ve got a good sample you need a decent sample size; think of those political polls which have over 1000 sampled and still have a margin of error of 3%).

In a nutshell, the better research (all other things being equal) is going to have a large random sample with “hard” data. Clear on what to look for? OK, here’s Allen’s table:

It should be immediately clear that the last three on the survey are by far the superior data sets. Rosenfeld measures grade retention (the likelihood a child will repeat a school grade)

Children of heterosexual married couples had the lowest implied rate of grade retention: 6.8%. Children of lesbian mothers and gay fathers had grade retention rates of 9.5% and 9.7%, respectively. Children of heterosexual cohabiting parents had a grade retention rate of 11.7%, while children of single parents had grade retention rates between 11.1% and 12.6%.

The differences in childhood grade retention between all types of non–group quarters households were dwarfed by the high rates of grade retention of children living in group quarters. According to Table 1, children living in group homes, many of them awaiting adoption or foster parents, had an implied grade retention rate of 34.4%. Children who were incarcerated had a grade-retention rate of 78.0%.

The Regnerus study is well discussed (and has a helpful summary website showing all the results which also shows statistically significant differences between various family types). Allen’s own research also shows a clear difference in outcomes for different family types.

The point of all this? When the authors of the AMJ paper claim that

The consensus of the peer-reviewed research is that children raised in same-sex parented families do as well emotionally, socially and educationally as children raised by heterosexual couple parents. These findings have been replicated across independent studies in Australia and internationally, some of which we discuss below.

you should question the accuracy of their prescription. The evidence is not clear. The largest random samples using “hard” measures say otherwise. It’s not conclusive either way and there’s much reason to suggest that perhaps the strong claims made by the authors in the media need some examination.

I’d be getting a second opinion.

Comments

comments

30 comments on “Trust Me, I’m a Doctor

  1. Classic cognitive bias article. The author believes in doctors as long as it does not conflict with his ideology. He got 10s of opinions but wants a “second” opinion. Not surprisingly the author can’t find any good studies to support his position but keeps on believing his view.

    Maybe one day he honestly unveils  his true motive; no matter what studies say he will not change his mind because of the Bible.

    Every study shows kids of gay couples doing better than single parents’ kids. As long as Christians keep on villifying gays from the holy book that orders killing gays, why do expect gay families doing as well as hetero families?

    • hi Jon,
      I see you’ve decided to comment here again but, as before, you’re not so keen on engaging with the actual argument the piece is making.
      “Can’t find any good studies”. How are you defining “good”? The article you’re claiming is full of cognitive bias addresses directly what makes a study “good”. Perhaps you missed it?
      “Every study shows kids of gay couples doing better than single parents’ kids. ”
      This is simply untrue, as even a brief view of the survey of research makes clear. It’s astounding, I reckon, that someone can come and make an accusation of being highly selective and then, well, be highly selective.

      • I did engage. I called out your hidden premise, your bias, that you conveniently left out; no study would ever convince you to support gay adoption because of your religion.

        When I asked you to provide studies to support your position you resorted to apologist word definition games. Why don’t you just provide the best one?

        But you can’t because there are no studies that support your position. Don’t you think it would be honest to tell that to your readers?

        • again, Jon, I fear in your rush to throw stones you’ve not taken time to get to grips with what the article is and isn’t arguing.
          My piece simply notes that the claims of the most recent AMJ article cannot be sustained given that when one surveys all the research carried out in this area it immediately becomes apparent that the very best research points in the opposite direction, i.e. that there are some very solid pieces of research that suggest that children raised in homosexual households fare worse than those in heterosexual households.
          You claim that I have provided “no” studies to support this position, yet appear not to have read the original article since it contains a list of such studies. Indeed, one might even come to conclusion after reading your comments that you’ve not really read my article at all, let alone made time to engage properly with it. You show no awareness of the implications of the table of studies from Allen which is key to the argument I am making. I’ll quote from my piece again since it appears that you passed over it first time around:

          It should be immediately clear that the last three on the survey are by far the superior data sets.

          That’s the key brick in the argument which you have now twice ignored.

          Now if you want to keep running your usual “all Christians are liars” kind of schtick that you’ve used here before then your comments will end up in the moderation bin very quickly. On the other hand, you could take a breath and engage with the actual argument being made in the piece which is, in it’s essence, rather simple:
          1. The AMJ authors claim that the evidence of research is that there is no difference for kids from SS households.
          2. The research on which they rely is full of small sample, self-selecting, self-reporting or “soft” data.
          3. There are some pieces of research which are the opposite; large random sample, objective “hard” data (and therefore superior pieces of research) – but which show (to varying degrees) deleterious outcomes for children in SS households.
          4. Thus, the AMJ authors’ claim is palpably false.

  2. Wow! Based on the quoted statistics, children of gay and lesbian couples have lower grade retention rates than children of cohabiting heterosexual couples. If same-sex couples are allowed to marry, maybe their children will have even lower grade retention rates, similar to married heterosexual couples! ? Also, if you are quoting statistics from the 2010 Rosenfeld (US) study, it would be good to include a link to the abstract, which puts these in context. It says: ‘The results show that children of same-sex couples are as likely to make normal progress through school as the children of most other family structures. Heterosexual married couples are the family type whose children have the lowest rates of grade retention, but the advantage of heterosexual married couples is mostly due to their higher socioeconomic status.’ (https://www.ncbi.nlm.nih.gov/pubmed/20879687)

    • Maybe, maybe not. The point of this piece is simply to respond to the claim laid out in the last quotation; ie that, according to the research, there is no difference between outcomes for kids raised by homosexual parents. That claim is palpably not supported by the evidence.

    • Hi David, I posted another comment after the comment above but it didn’t seem to come through? Thanks for including the link to the Rosenfield study (sorry I missed this before!) Although the Rosenfeld study seems to be dated 2010, it is based on data from the US 2000 census. In his introduction, Rosenfeld notes that at that time same-sex couples could not marry (although some same-sex couples ‘self-identified’ as married). Among other factors, he acknowledges the benefits of legal marriage and that heterosexual cohabiting couples are a second logical comparison group for same-sex cohabiting couples (along with heterosexual married couples) in that they are both two-parent families living without the rights and benefits of marriage.
      In talking about the relevant comparison sets, he states: ‘The comparison between children of same-sex cohabiting couples and children of heterosexual cohabiting couples allows for a more specific test of the effect of same-sex parenthood on children, while holding constant legal rights and the number of parents.’ Based on this test, the statistics from the Rosenfeld study show that children of same-sex cohabiting couples with 9.5% and 9.7% retention rates definitely did ‘as well as’ (or in this case, better than) children of cohabiting heterosexual couples with 11.7% retention rates.
      This would seem to support the statement in the AMJ paper that children of same-sex couples ‘develop as their peers in heterosexual families do’ and ‘do as well … as children of heterosexual couple parents’. From the quotes included, the AMJ paper doesn’t seem to differentiate between married or cohabiting parents, so the comparison with heterosexual cohabiting couples would seem to be as relevant as the comparison with heterosexual married couples. Also, I couldn’t see the phrase ‘no difference between outcomes’ in the AMJ quotes, which has a slightly different meaning in my mind than ‘do as well as’ or ‘develop as’ their peers.
      Because Rosenfeld uses heterosexual married couples as the standard comparison group for all other groups in his study, it would be interesting to see an updated study with more recent US census data, which could include a comparison between children of heterosexual married couples and children of same-sex married couples.

      • Hi Ros, thanks for commenting. I’m afraid I’ve not seen any other comments from you ( I do seek to approve comments as soon as possible!).
        I think you’re right – an updated study would be very helpful given the changes we’ve seen in the past 10 years.

  3. Hi David, I remember when you first made mention of this study on your blog and your comment that this study would be a good one to put in your back pocket, presumably to pull out to counter studies which say the opposite. If your approach is to collect studies which support your preconceptions so that you can ignore studies that deny them then what right do you have to point the finger at the doctors. Everyone has an agenda. I personally the people who’s beliefs aren’t written in stone and who have the freedom to change it when the evidence suggests otherwise. Thank God that the knowledge of our doctors keeps growing thanks to their reliance on peer-reviewed, evidence based studies.
    This is a complex issue where multiple variables are in play (gender, societal expectations, discrimination, a history of parental breakups…). Even with all of this, the study you quote says that children of gay parents are doing better than the children of cohabiting heterosexuals. I sincerely believe a yes vote is a vote for better conditions for children currently living in same-sex households and I think the evidence backs me up in this.

  4. If your approach is to collect studies which support your preconceptions so that you can ignore studies that deny them then what right do you have to point the finger at the doctors.

    Agreed. Just as well, then, that I don’t collect studies for this reason. The only people here “ignoring studies” are those making the claim that all the research tells us that kids in SS households do just as well, or better, as kids in married households. As the OP makes clear, the summary table of data from Allen shows otherwise.

    I agree that this is a complex issue. My point in this piece is that the AMJ article authors’ claims are false and easily refuted. No more, no less.

    For you make this claim: “I sincerely believe a yes vote is a vote for better conditions for children currently living in same-sex households and I think the evidence backs me up in this.” you would have to provide us with a longitudinal study that measures “success” for such kids both before and after SSM is introduced in a particular jurisdiction. Do you have such a study you can point us to?

    • OK David, let’s try this line of reasoning… Suppose I am a chocolate lover and I read a peer reviewed study that chocolate is good for me. In fact, their study shows a link between consuming chocolate and fighting cancer. Now suppose that I put that study in my back pocket and pull it out to refute every study that says that chocolate consumption is harmful. I have my one study which shows it is helpful therefore the study which shows it is harmful has just been falsified, right?
      Separately, I don’t agree with your statement that “you would have to provide us with a longitudinal study that measures “success” for such kids both before and after SSM is introduced in a particular jurisdiction”. That is akin to me saying that for you to believe in the resurrection you need to provide video evidence of an angel sitting on a rock saying “He is risen”. What you are asking for doesn’t exist and cannot exist due to the limitations of the studies which you have already pointed out. How would anyone have had the foresight to begin longitudinal studies on these children and the nature of longitudinal studies is that they are extremely small samples (when compared with sampling methodologies which do not attempt to track change over time). A small sample size of a small section of the population is going to make it almost impossible to achieve a 95% confidence interval.
      What I am saying is simply that I believe that the evidence backs me up. I don’t go anywhere near saying that the evidence proves this person right or that one wrong. On my weighing of the evidence, however, I believe that I am doing the right thing by children in voting yes. I sincerely hope that for all the negative points of the plebiscite, a yes outcome will put us on a path to destigmatising this portion of society and their families.

  5. OK David, let’s try this line of reasoning… Suppose I am a chocolate lover and I read a peer reviewed study that chocolate is good for me. In fact, their study shows a link between consuming chocolate and fighting cancer. Now suppose that I put that study in my back pocket and pull it out to refute every study that says that chocolate consumption is harmful. I have my one study which shows it is helpful therefore the study which shows it is harmful has just been falsified, right?

    No, because it’s just one study. More importantly we’ve made no assessment of the study itself. We could, for example, use the criteria that commonly help us work out how good a study is: sample size, randomness of sample, hard/soft evidence.
    Without that, we really don’t know how good the study is at all.

    Besides, as I’ve already stated in the comment that you are replying to, I haven’t “kept Allen in the back pocket”.
    I turned to Allen because it’s the best comparison we’ve got to the claim being made by the AMJ article authors. They have take other surveys of research and combined them to give us a meta-survey. Having produced this meta-survey they then claim that the evidence is overwhelmingly in favour of SS-household kids having equal (or even better) outcomes.

    So to test this claim we just look at the survey from Allen and we see that actually the claim is not substantiated. As I state in my OP,

    The evidence is not clear. The largest random samples using “hard” measures say otherwise. It’s not conclusive either way and there’s much reason to suggest that perhaps the strong claims made by the authors in the media need some examination.

    Now I’ve made this point 3 times. Once in the OP. Once in the comment that you reply to above and now here one third time. I note that you have not dealt with this main thrust of the piece, but instead bring up peripheral stuff. If you can’t actually engage with the main argument being made (and the way to do so would be to demonstrate that Allen’s survey is massively incomplete or that there is a serious flaw in the argument being made (which would itself mean that small self-selecting non-random self-reporting “soft” samples were superior)) then readers of this thread can draw their conclusions.

    What I am saying is simply that I believe that the evidence backs me up

    Sure, as long as you discount the evidence from the large random hard research pieces in favour of the evidence from the small non-random soft research.
    But then they are really helpful to have in the back pocket.

  6. Thinking on this further (before I give up and watch TV instead), I don’t think “soft” data is all that bad; I’m happy to believe someone who tells me they feel terrible without first making them fill in a diagnostic screen as proof. Your article seems to miss what I took as the key message from the Knight et al. paper: that stigma and discrimination hurt people, and that it’s this debate (especially arguments from those who say they’re concerned for the children) that’s causing suffering to kids of same-sex parents. The response to this message doesn’t require a search for hard data. Just ears to hear, and some compassion.

    • I don’t see anyone here lacking in compassion. I think stigmatisation of people is an awful thing.
      But it’s presence doesn’t discount the critiques of “soft” data. It’s hard to quantify and the plural of anecdote is not anecdota.

      • On behalf of all qualitative researchers, I’m kinda bummed that you think the only data worth listening to is quantifiable. Faith isn’t quantifiable! You wouldn’t tell a group of congregants/friends who approached you with their struggles that you couldn’t take their perspectives into account because of sample size!

        There’s so much to learn from hearing people’s stories and trusting them when they say, “We’re not okay right now” OR (and this is particularly relevant, given you’re discounting the self-reported data used in many of the studies included in the reviews/meta-analyses which show no difference in the children of same- vs. different-sex couples) “We’re actually fine.”

        “Hard” research may be quantifiable, but that’s not the only thing to consider when analysing research or data. It has its disadvantages too, just as “soft” research has advantages. Both quantitative and qualitative approaches are useful ways of gathering and analysing information.

        • On behalf of all qualitative researchers, I’m kinda bummed that you think the only data worth listening to is quantifiable.

          Well that’s a shame, since I never made such a claim.

          Faith isn’t quantifiable! You wouldn’t tell a group of congregants/friends who approached you with their struggles that you couldn’t take their perspectives into account because of sample size!

          I’m not sure how you even begin to suggest they’re the same thing. A church member coming to me with their experience is not the same thing at all as a research study seeking to establish something.

          … given you’re discounting the self-reported data used in many of the studies…

          again, you’re putting words in my mouth. My critique of the AMJ paper is that it makes extraordinary claims based on only that sort of data and effectively ignores some very large scale random samples. The self-reported data is helpful for showing us some of what is going on, but it’s simply not going to help us when we’re looking to do the large-scale analysis of causal links (or non-links, as the case may be).

          Both quantitative and qualitative approaches are useful ways of gathering and analysing information.

          Yes, but only the quantitative approach can give us the hard data we need to establish correlations etc.

          And again, you miss the main point here. The AMJ paper appears to utterly discount those large-scale studies and then make enormous claims.

          • The AMJ paper takes into account the results from many studies, one of which is the Regnerus one which found that children with same-sex parents fare poorly – this doesn’t seem doesn’t seem like they’re discounting information, or making unfounded claims. Overwhelmingly it seems as though research has found little to no significant differences between kids of same- or different-sex parents.

            Is your point that they didn’t refer to Allen’s research specifically? Was Allen’s paper peer-reviewed? I know this is at least one of the criteria used to choose which research can be included in reviews and meta-analyses (but I can’t find info on Allen’s article in particular). It’d be interesting to know the reasons behind its exclusion; given the Regnerus research is pointed out in the AMJ article, Allen’s doesn’t seem to have been left out purely because of its conclusions…

    • Your point in the OP seemed to be that these particular doctors couldn’t be trusted because the reviews and meta-analyses they referred to were mostly based on qualitative data and smaller sample sizes. I’ve pointed out both that there are significant methodological limitations in the Allen paper (you can search for the same info on the Regnerus one, if you’re actually interested in doing some research here) and that qualitative research isn’t as bad as you seem to suggest. Smaller sample sizes are also expected when researching a minority population. I’m not sure I did miss your point after all.

      Also, if you want proof of anything, you’ll need more than a correlation. I’m happy to trust those who know what they’re talking about on this one.

      • Smaller sample sizes are also expected when researching a minority population.

        Yes, and they’re also what you expect when you have convenience sampling with all the issues that go with it. What’s fascinating about this exchange is that you appear (but correct me if I’m wrong) to be happy to talk about all the possible issues with Regnerus/Allen etc (which I’m more than delighted to entertain – no study is perfect) but don’t appear so critical about all the other types of studies.
        I say all studies to be critiqued and don’t ask them to bear a greater weight than they can. The AMJ conclusion asks the studies to bear far too much weight.

        I’m happy to trust those who know what they’re talking about on this one.

        As am I. A trained statistician that I share the same gene pool with described Rosenfeld and Allen as “the gold standard here”.

        • I was thinking the same thing about you! It’s always hard to hear the strengths of studies that disagree with your (/my) point of view, and easy to see the limitations of the ones you (/I) agree with! That’s how confirmation bias works.

          Your relative is wrong; removing confounds so that you’re actually measuring what you think you are is what makes a gold standard design. Allen’s making guesses about his sample (as pointed out in the link I posted earlier), which means that his conclusions may not even apply (and would certainly not be generalisable) to children of same-sex parents. We can’t know if he’s right or not, because confounds.

          As for the AMJ study, it’s nothing profound. I know from experience that people talking about me as if I’m inferior is painful (I was a woman in the Anglican church for quite some time), so it doesn’t surprise me at all to hear that LGBTQI people may be suffering as a result of the survey, as well as blog posts like this one. Why make the conversation about hard data rather than acknowledging those who are hurting, keeping the focus on them rather than looking anywhere else for distractions? That’s what I mean by compassion: where’s the compassion in actively campaigning against existing families who are hurt by your words? It makes no sense to me.

  7. Why make the conversation about hard data rather than acknowledging those who are hurting, keeping the focus on them rather than looking anywhere else for distractions?

    Why not make it about both?

    That’s what I mean by compassion: where’s the compassion in actively campaigning against existing families who are hurt by your words?

    I’m not “actively campaigning against” families. And the fact that they may or may not be hurt by my words does not invalidate the argument which should stand on it’s own terms. That doesn’t mean I don’t care. On the contrary, anybody with an ounce of compassion wants people to have the best outcomes in life. The question is, how will that come about? I know that making the avoidance of hurt the be all and end all isn’t the answer.

  8. Thanks David, I’ve not commented on blogs much so may have mucked it up the first time around! As other people have noted, concerns have been raised about the samples and methodology of the Regnerus and Allen studies.

    Regnerus:
    https://www.washingtonpost.com/news/volokh-conspiracy/wp/2015/05/10/new-criticism-of-regnerus-study-on-parenting-study/?utm_term=.fe56a7d6a29a

    https://newrepublic.com/article/116741/regnerus-study-same-sex-marriage-michigan-must-defeat-bad-science

    Allen:
    https://familyinequality.wordpress.com/2013/10/11/the-douglas-allen-study-of-canadian-children-of-gaylesbian-parents-is-worthless/

    https://www.skepticink.com/humanisticas/2013/10/14/no-children-of-same-sex-parents-do-not-have-lower-graduation-rates/

    The first critique of the Allen study (which Annelise also references below) suggests that while this study was promoted as involving a big, random, national sample, in the case of same-sex households ‘…Allen ends up with a tiny sample of people from gay- or lesbian-parent households, and makes bad decisions to increase its size.’

    So while the overall sample may have been large in relation to other groups, the actual sample size for same-sex households seems to have been small.

    These issues should be taken into account when considering the findings of the Regnerus and Allen studies as should the factors mentioned for other studies (e.g. hard/soft data, sample size, random).

    • No problem Ros.
      FWIW I wouldn’t want to avoid the limitations of any study. Nevertheless, even with the limitations of Regnerus’ definition of “gay parent” it’s still a powerful study given it’s breadth of analysis of different parent types. The homosexual parent aspect is just one facet of the dataset.

      More research needed, and less over-reaching assertions about conclusions.

  9. I don’t know why you need make ad hominem that I “keep running your usual “all Christians are liars””. I never have said that nor do I think so. If you have even a single evidence that I think “all Christians are liars” please let me know. Otherwise please redact that comment.

    You asked for a “second” opinion:
    http://whatweknow.law.columbia.edu/topics/lgbt-equality/what-does-the-scholarly-research-say-about-the-wellbeing-of-children-with-gay-or-lesbian-parents/

    It is more resent and has more studies. It also have your two darling studies among the four at the bottom. Maybe you should write an article why Allen and Regnerus methodology is criticised and/or how they lump their data.

    • hi Jon. You regularly accuse me (and Christians more generally) of dishonesty on this very site. I’m happy to make the charge again.

      As for the survey that you link to, readers can go and see what they think of the selection criteria and assessment of results and come to their own conclusions about whether it is persuasive or not. I’ve already written several times on the limitations of Allen and Regnerus.

Leave a Comment - but please pay careful attention to the commenting rules