Systematic reviews have been at the forefront of the campaign to reform pediatric gender medicine. A systematic review is a process in which the reviewers collect all of the available studies on a research question and summarize the results using methods designed to minimize bias and errors. Systematic reviews of the use of puberty blockers and cross-sex hormones in young people have consistently found that the evidence of benefit is low or very low certainty. This contradicts the claims of gender affirming clinicians that these treatments are “life saving” and, when combined with evidence of risk, raises questions about whether they should be offered at all.
The process began in Finland where, after conducting a systematic review which found the evidence for the benefits of pediatric medical transition uncertain, the national health authority issued new guidelines which recommended psychotherapy as the first line of treatment. Sweden followed, with a systematic review that found the evidence of benefit to be of low certainty, and new guidelines that described gender transition of minors as an experimental practice. The National Institute for Care and Excellence in the United Kingdom published systematic reviews of puberty blockers and cross-sex hormones which found the evidence of benefit to be of low certainty. In the United Kingdom, the Cass Review commissioned a series of systematic reviews from the University of York. Researchers at McMaster University have published systematic reviews of puberty blockers, cross-sex hormones and mastectomy.
Umbrella reviews synthesize the results of multiple systematic reviews. The Florida Board of Medicine and the Department of Health and Human Services evidence review both conducted umbrella reviews.
Challenging Systematic Reviews
Advocates for gender medicine have responded to the challenge to their preferred model of care posed by the results of systematic reviews in several ways.
One is to attempt to suppress unfavourable systematic reviews. WPATH tried this with the reviews it commissioned for its SOC8, but it faced a serious backlash when evidence of the suppression was disclosed on discovery in the Boe v. Marshall case. Undeterred, activists have mounted a campaign to suppress publication of the systematic reviews which the Society for Evidence Based Gender Medicine commissioned from McMaster University.
Another tactic for mitigating the effect of unfavourable systematic reviews is to add editorial comments to the published review which skew its conclusions. The review of hormonal therapy by Baker et al., which was commissioned by WPATH and published only after WPATH approval, provides an example. The analysis in the review found that the strength of evidence for improvement in quality of life, depression and anxiety was low. There was insufficient evidence to draw any conclusions on death by suicide. However, the concluding sentences of the study read:
Despite the limitations of the available evidence, however, our review indicates that gender-affirming hormone therapy is likely associated with improvements in QOL, depression, and anxiety. No studies showed that hormone therapy harms mental health or quality of life among transgender people. These benefits make hormone therapy an essential component of care that promotes the health and well-being of transgender people.
Activist criticisms of systematic reviews often misrepresent the review process. One is to point out that authors of systematic reviews usually have no experience in gender medicine. However, subject matter expertise is not required for a systematic review. Subject matter experts are usually involved in framing review questions, but the process of evaluating research methodology and evidence synthesis is a separate field of expertise.
Another common criticism, raised in the aftermath of the Cass Review, is that systematic reviews require randomized controlled trials which are often not ethical or feasible in gender medicine. This is a misreading of systematic reviews, which do consider other types of research. Well-designed and conducted observational studies can provide at least moderate quality evidence that a treatment is effective. The problem is that there have been very few well-designed studies in pediatric gender medicine.
A more promising line of attack has been to raise issues about the application of systematic reviews to treatment decisions and guidelines. Advocates for pediatric gender medicine often point out that many treatment decisions, particularly in pediatrics, must be made based on low certainty evidence. A counterargument, found in the Department of Health and Human Services review, is that evidence from systematic reviews must be considered in the context of certainty of diagnosis, the natural history of the condition and the risks of treatment. Treatments supported by low or very low certainty evidence may be justified when the risk of harm from a treatment is low and / or the likely consequence of failing to treat is death or serious disability. Pediatric gender medicine is exceptional because it involves not only low certainty of evidence of benefit, but a condition which is diagnosed based on self-reported symptoms, a natural course that tends towards resolution in many cases, and substantial risks to treatment, including sterility and early onset osteoporosis.
The findings of systematic reviews must be integrated with patient values and preferences in both clinical decision making and guideline development. Advocates for gender affirming treatments claim that patient preferences in gender clinics generally favour providing treatment. This raises an ethical debate over the extent to which respect for patient autonomy can be used justify treatments which are supported only by low certainty evidence.
Some activists push the autonomy argument even further and claim that the evidence of mental health benefits from treatment which the existing systematic review have evaluated is not relevant as treatments should be provided on the basis of self-determined embodiment goals.
The Utah Review
Another tactic which affirming care advocates use to counter unfavourable systematic reviews is to promote aggressively favourable pseudo-systematic reviews. These are documents that have a superficial appearance of a systematic review but fail to meet the essential criteria for a genuine review.
A review conducted for the State of Utah (the “Utah Review”) is the latest example of a pseudo-systematic review that has received wide public attention. In 2023, the Utah state legislature banned gender reassignment surgeries for minors and imposed a moratorium on puberty blockers and cross-sex hormones. It also ordered the Department of Health and Human Services to conduct a systematic review of the provision of transgender medical treatments to minors. The review, which was conducted by the Drug Regimen Center of the University of Utah College of Pharmacy, was completed in August, 2024 and was made public in May 2025.
Unfortunately, the government was not careful in defining the terms of reference for the Utah Review and the result was a major embarrassment for the Republican legislators who were promoting the treatment ban.
The Salt Lake City Tribune published an article with the headline, “Utah lawmakers’ own study found gender affirming care benefits trans youth. Will they lift the treatment ban?” It reported that the 1,000 page report found that, “Overall there were positive mental health and psychosocial functioning outcomes” as a result of gender-affirming care. There were similar reports by NBC News, Mother Jones, Assigned Media and others. The review has also been cited in a peer reviewed article in The New England Journal of Medicine .
What the supporters of the Utah Review failed to see was that, despite its length, it lacks the scientific rigour that underpins the Cass Review and the Department of Health and Human Services Review. The shortcomings of the Utah Review have been described by the Society for Evidence Based Gender Medicine and Do No Harm Medicine. Journalist Ben Ryan wrote an X/Twitter thread criticizing the review. The authors of the DHHS review included a formal methodological appraisal of the Utah Review as Section 2.4 of Appendix 4 to the revised version of the DHHS Review and found it to be at high risk of bias in every respect.
The Utah Review fails to satisfy any of the fundamental requirements for a reliable systematic review.
The first step in a systematic review is to develop a research question as a series of questions using the Population, Intervention, Comparator, Outcome (PICO) framework. In gender medicine, a framework might be, “In people with gender dysphoria under the age of 25 (population), what is the effect of cross-sex hormones and psychological support (intervention) as compared to psychological support alone (comparator) on mental health (outcome)?” This question is then used to develop eligibility criteria for studies.
The Utah Review never clearly defined its research questions. It collects studies on puberty blockers and cross-sex hormones but does not identify the specific outcomes from each intervention it wants to evaluate.
The next step is to search for relevant studies in the online databases of medical literature. The search queries should be registered in advance in PROSPERO or some other service. This ensures that the search will be consistent and transparent. The retrieved studies are then screened to identify the ones that are eligible for consideration.
The lack of clearly defined research questions compromised the search for studies by the Utah Review. Furthermore, the search criteria were not registered in advance. The DHHS critique notes that, “The review includes 134 primary clinical studies involving over 28,000 youth with gender dysphoria from diverse global settings, yet provided no transparent rationale for how such a wide range of study types—all with vastly different designs, populations, and methodological quality—fit within a coherent framework for evidence review.”
The reviewers then abstract the relevant data, such as study design, participants, control group (if any) and outcomes from the selected studies. The next, critical step, is the assessment of the eligible studies for risk of bias. If a study has a high risk of bias, the results from the study may not be representative of the entire patient population. There are various tools for assessing risk of bias, such as the Risk of Bias in Nonrandomized Studies of Interventions (ROBINS-I), the Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-2 tool, and the Quality in Prognosis Studies (QUIPS) tool. Best practice is for two reviewers to conduct the data abstraction and risk of bias assessment independently and then compare results. A third reviewer may be called in to resolve conflicts. This minimizes the risk of error and bias by the reviewers.
The data extraction was not performed in a clear and consistent manner. The review explained that due to “time constraints the review extracted data for 58 observational / experimental studies and 32 single-arm clinical trials/longitudinal, pre-post descriptive studies, while other descriptive studies were relegated to the “bibliography only.” The DHHS critique responded that “This ad hoc approach undermines the reliability of the review.”
There were also problems with the risk of bias assessment. The reviewers used two different assessment tools for different types of studies The DHHS critique objects that this use of different risk of bias tools was not justified and prevented the results from being presented in a clear and consistent manner. The review also failed to assess all eligible studies.
Once the evaluation of the studies is completed, the reviewers proceed to the evidence synthesis. Once the review of the primary studies is complete, the reviewers summarize the data from all the studies for each outcome defined in the PICO questions. Where possible, the reviewers conduct a meta-analysis, which uses statistical methods to combine the results from each study for a given outcome. Where a meta-analysis is not possible, other methods are available, but these are less reliable.
The final step is to assess the certainty of evidence and draw conclusions. This is usually done with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. The evidence summary will describe an estimate effect of treatment (eg. puberty blockers lead to an improvement in quality of life). The GRADE assessment describes how confident the reviewers are that this effect represents the true effect. GRADE uses four levels:
· High: Very confident that the true effect lies close to the estimate of effect;
· Moderate: Moderately confident in the effect estimate but there is a possibility that it is substantially different from the true effect;
· Low: Limited confidence in the effect estimate; it may be substantially different from the true effect;
· Very low: The true effect is likely to be substantially different from the effect estimate.
The lack of an evidence synthesis is the critical failure of the Utah Review. . A casual reader might be misled on this point because the table of contents has chapters entitled “Evidence Synthesis Methods” and “Results of Evidence Synthesis.” However, at page 90, the authors of the review say, “We were not contracted to include a synthesis of the evidence that we found: only to assess ROB [Risk of Bias] and provide evidence tables summarizing safety and efficacy findings.”
What this means is that while the Utah Review contains hundreds of pages of tables describing the risk of bias assessment and results of individual studies, there is no place in the review that summarizes the evidence on all studies on any given outcome in a way that meets the requirements of a systematic review. The body of the review breaks down studies into experimental, observational and descriptive studies and provides narrative descriptions of results for various outcomes. For example, at p.82 the review offers “limited takeaways” based on the findings of pre-post descriptive studies. Under mental health, the review states, “Scores for anxiety and depressive symptoms tended to either have no change, or decrease while on treatment.”
Compare this analysis of depression in a genuine systematic review such as Miroshnychenko et al. (2025) on gender affirming hormones:
Unlike the Utah Review, the authors provide details of the studies considered, the measurement tool used, the size of the effect and an assessment of the certainty of the evidence.
On page 91 of the Utah Review there is a statement which clearly shows that the review is not a systematic review and, further, that the authors do not seem to understand the hierarchy of evidence in evidence based medicine:
Based on the reviewed evidence included in this report, it is our expert opinion that policies to prevent access to and use of GAHT for treatment of GD in pediatric patients cannot be justified based on the quantity or quality of medical science findings or concerns about potential regret in the future, and that high-quality guidelines are available to guide qualified providers in treating pediatric patients who meet diagnostic criteria.
In an earlier paragraph they explain that this opinion is a general impression from reading large numbers of studies. This misses the whole point of a systematic review. The general impression gained from reading a large number of studies can be highly misleading. The certainty of the evidence supporting a treatment depends not on the quantity of studies but on their quality, which can only be reliably assessed through an evidence synthesis.
The authors of the Utah Review claim that they included a larger number of primary studies than other systematic reviews. According to Assigned Media, “The authors found a total of 277 studies that included data on over 28,000 young people with gender dysphoria.”
It is a fundamental error to equate volume of studies with quality of evidence. The large volume of studies includes many case reports on one or two patients. The large number of young people in the studies includes cross-sectional studies which survey a large number of respondents at a single point in time. Neither of these types of studies is capable of providing high certainty evidence of the effectiveness of a treatment.
In his X thread on the Utah Review, Ben Ryan pointed out that expert opinion ranks at the bottom of the pyramid of evidence. Even as expert opinion, the Utah Review is suspect. In medicine, an expert is normally a doctor or other clinical practitioner who has both research experience and clinical experience working with the condition in question. The authors of the Utah review are pharmacists or graduate students, and Google searches of their backgrounds did not show that any of them have published research on young people with gender dysphoria or any relevant clinical experience. As noted above, lack of subject matter expertise is not a factor in a true systematic review which depends on rigorous application of assessment methodology. A review which is based on neither rigorous methodology nor deep subject matter expertise must be at the very bottom of the bottom of the evidence pyramid.
In Utah itself, the review does not seem to be having the desired effect. Deseret News reports that Republican law makers are not impressed with the Review and are moving ahead on legislation to ban permanently pediatric medical transition. Nevertheless, it has been cited in at least two peer reviewed publications on the HHS Review and appears likely to join the growing number of “zombie studies” which continue to be cited no matter how often and how thoroughly they are refuted.
Good explanation. Clear, concise, and a killer review. Thank you.
interesting. thanks