To discuss reliability of specific sources, please go to Wikipedia:Reliable sources/Noticeboard or the Wikiproject talk pages of WT:MED or WT:PHARM.

Shortcut: WT:MEDRS

	This is the talk page for discussing improvements to the Reliable sources (medicine-related articles) page.
	Put new text under old text. Click here to start a new topic. Please sign and date your posts by typing four tildes (~~~~). New to Wikipedia? Welcome! Ask questions, get answers.	Be polite Assume good faith Avoid personal attacks Be welcoming
Archives: 1, 2, 3

Contents 1 What "assess the evidence" means 2 New template 3 Scope of this guideline 4 Aspartame controversy 5 Talk:Thiomersal controversy#Introduction Neutrality Dispute 6 Medicinal clay 7 Hair analysis (alternative medicine) 8 Need for WP:MEDRS/N noticeboard 9 Theories of dyslexia

[edit] What "assess the evidence" means

I've been looking through some conversations from earlier this year, and I have run across an area that WP:MEDRS#Assess_evidence_quality may need to provide a little more clarity. It seems that this section is being occasionally (mis)understood as requiring editors to decide whether a study (or review) has sufficient data to support the conclusions that it draws.

So what we mean -- and what I thought we said -- was that if you're faced with contradictory papers, or trying to decide whether something's important enough to include, then you prefer reviews and meta-analyses to original RCT reports, and that you prefer RCTs to case studies, and so forth.

I'm seeing this invoked as a justification for WP:OR: I didn't think that their data sample was big enough to justify that conclusion... and the review cites Smith's trial for this fact, so that's just a single original paper... and Smith's paper is a little old, anyway, even though the review is recent... and probably the reviewer is twisting Smith's conclusions... -- even though we're citing the review instead of Smith.

I think that the problem might actually be in the section's title. Could we consider renaming it to something less open-ended, like "Prefer well-designed studies"? WhatamIdoing (talk) 19:58, 19 October 2009 (UTC)

Wikipedia should be a tertiary resource and so if it comes down to determining whether a single paper is reasonable or not or which of a pair of contradictory papers Wikipedia should be citing, it is always best to go with secondary sources. In general, we should be encouraging editors to use WP:PSTS and avoid primary sources completely. The purpose of the encyclopedia is not to stay on the cutting-edge of medical wisdom but to provide a dispassionate and removed summary of the most conventional and staid wisdom about notable subjects. To that end, I'd say what we should retitle the section Use consensus summaries or something of that nature. ScienceApologist (talk) 21:52, 19 October 2009 (UTC)

I very much agree with ScienceApologist. Regards, —mattisse (Talk) 22:20, 19 October 2009 (UTC)

Yes, of course. But that point is fully covered in WP:MEDRS#Respect_secondary_sources, and is not the point that I'm talking about. My question involves the section that compares meta-analyses to individual trials, and individual trials to case studies, and case studies to pure speculation. The title of this section seems to be misunderstood as encouraging editors to pass judgment on whether a secondary source has accurately represented the primary works that it discusses. WhatamIdoing (talk) 22:30, 19 October 2009 (UTC)

Not sure that the point is fully covered there, actually. Essentially all journal articles can be considered to be primary sources whether they are meta-analyses, RCTs, case studies, etc. A secondary source would review the state of the topic and report the consensus opinion. Those are the best sources to use. The issue I have with this section is that it does not go far enough in making sure that primary sources are avoided whenever possible. I guess you can say that I'm a fan of, "When in doubt, ask for an expert summary." ScienceApologist (talk) 22:47, 19 October 2009 (UTC)

Again, I agree with ScienceApologistm having dealt with many medical/psychiatric articles recently where primary sources are considered ok. Not only primary sources, but old primary sources. Regards, —mattisse (Talk) 22:56, 19 October 2009 (UTC)

Again, encouraging the use of secondary sources is not the point of this subsection. No matter how fervently we all agree that secondary sources are wonderful -- and I believe that we do agree on that point -- our agreement does not change the content of this particular subsection to be about secondary sources instead of about study designs. WhatamIdoing (talk) 23:35, 19 October 2009 (UTC)

Why not just rename the section to "Use high-quality sources"? That matches the section's contents better anyway, and would help avoid the possible misunderstandings noted above. Eubulides (talk) 23:25, 19 October 2009 (UTC)

Because an elderly self-published source is not a high-quality source even if it is a meta-analysis. Study design is not the only factor that determines high-quality sources. It is, however, the only factor addressed in this particular subsection. WhatamIdoing (talk) 23:32, 19 October 2009 (UTC)

But the section is about more than study design. It also emphasizes reviews and other secondary sources that are not research studies in the usual sense. I'd rather not have the section header limit itself to "studies", which sounds too much like primary research studies. Eubulides (talk) 23:44, 19 October 2009 (UTC)

Agree. We should not be int the business of "evaluating" primary research studies. Regards, —mattisse (Talk) 00:06, 20 October 2009 (UTC)

Eubulides, I take your point about "studies", although the major point of this section is clearly just that. I'm open to alternative titles.

Mattisse, with respect, we should be in the business of evaluating primary studies, because this guideline instructs us to do that. If you do not stop to figure out what kind of paper you're looking at, you'll treat an outdated narrative review of three case studies as being far more reliable than the a recent very large randomized controlled trial on the same point. WP:PSTS is not the only valid consideration, and this guideline has always encouraged appropriate use of primary sources. WhatamIdoing (talk) 00:19, 20 October 2009 (UTC)

How about Consider study design? I think asking editors to conduct assessments of studies is wildly inappropriate, and maybe even insisting on "preferences" can be construed problematically, but exhorting people to consider the designs of studies when writing prose is okay in my book. ScienceApologist (talk) 03:35, 20 October 2009 (UTC)

The "Consider" is fine, but as mentioned above the section is about more than just study design. How about "Consider the type of source"? Also, in the 1st sentence the phrase "assessing whether a particular viewpoint" should be changed to "considering whether a particular viewpoint". Eubulides (talk) 16:33, 20 October 2009 (UTC)

"Consider the type of source" will be misunderstood as "Classify it according to WP:PSTS," which would make this redundant with the previous section (which is all about PSTS issues).

I don't think that there's really very much in this section that isn't directly or indirectly about study design. WhatamIdoing (talk) 02:53, 1 November 2009 (UTC)

A lot of it is about study design, but some of it is about other kinds of sources as well (meta-analyses, reviews). How about "Critically appraise sources"? This title was inspired by the cited source Young & Solomon 2009 (PMID 19153565), "How to critically appraise an article". The section's lead sentence should be altered to match the new title: "Several systems exist for assessing the quality of available evidence on medical subjects, and these Sources for biomedical topics often vary widely in quality, and this should be kept in mind ...". Eubulides (talk) 02:42, 3 November 2009 (UTC)

Too vague, and misleading. Source has a technical definition on Wikipedia (see the footnotes at WP:V), and what this section is about has nothing to do with that. The main point of this section is: A meta-analysis trumps a case study. I want a title that specifically deals with the main point. WhatamIdoing (talk) 04:35, 3 November 2009 (UTC)

That's an important point of the Assess evidence quality section, but it's not that much more important than the section's other points. The section also mentions systematic reviews (at the same level as meta-analyses), narrative reviews, and clinical experience, and none of these fit into the narrow rubric of "study". As for the technical definition of source in WP:V, although the section focuses on articles, it also briefly talks about publishers and authors, so the word "sources" is not out of place here. Eubulides (talk) 04:55, 3 November 2009 (UTC)

Perhaps I'm missing something obvious, but I can't find the words "publisher" or "author" anywhere in this section. I see nothing in these two paragraphs that is unrelated to figuring out whether or not a given claim is based on high-quality data -- from "meta-analyses" at the top of the scale to "speculative proposals" at the very bottom.

I agree that a narrative review isn't technically a 'study'. However, a systematic review generally is taken to be a study (of the previously published literature). (It's not an "experiment," but neither is a case report.) WhatamIdoing (talk) 19:29, 11 November 2009 (UTC)

The words "publisher" and "author" are not in the section, but the topic of publishers is mentioned when the section talks about "popular press or a peer-reviewed medical journal", and the topic of authors is raised when the section talks about "expert committee" and "respected authorities". I take your point that the term "study" sometimes includes reviews; but often it doesn't, and it'd be better to avoid ambiguity here. How about "Consider reviews and study design"? Eubulides (talk) 21:16, 11 November 2009 (UTC)

That phrase means two unrelated things to me.

When we tell an editor to "consider reviews", we mean "consider relying on them -- they're usually pretty good sources for all sorts of purposes". When we tell an editor to "consider study design", we mean "This paper is just a short case series, and that's pretty weak evidence" (or "That's a meta-analysis, which is strong evidence").

IMO, the point of this section is the second, not the first. WhatamIdoing (talk) 22:27, 11 November 2009 (UTC)

[edit] New template

The new template is interesting, on the one hand, but based on its sole use to date,[1] I suspect that it may be no more useful than Template:Verify credibility, and may be misused. MEDRS doesn't presume to provide standards for strictly social facts, such as whether scientists "should" publicly oppose quackery, or whether a country devotes sufficient resources to health care for a given marginalized population, or what the polite term is for a given condition, or that sort of thing. Instead, it cares mostly about physical facts, of the type that you can determine with a well-designed trial or three. The other RS-related guidelines are quite sufficient for these non-scientific statements.

Does anyone else have any thoughts about this new template? WhatamIdoing (talk) 02:52, 1 November 2009 (UTC)

I think I'll use it a lot, and it will help shorten edit summaries and explanations ! SandyGeorgia (Talk) 02:54, 1 November 2009 (UTC)

Its contents "MEDRS violation?" were too cryptic; almost nobody outside our little circle would know what that means. I changed it to "Reliable medical source?". Like SandyGeorgia, I think it could be useful; this is true regardless of whether it was used well or poorly in its first use. Also, I noticed that {{fix}} says "please do not use this template to create a new sub-template without discussing and justifying the need for it at Wikipedia talk:WikiProject Inline Templates", so I just now created a thread Wikipedia talk:WikiProject Inline Templates #New template MEDRS to give that project a heads-up. Eubulides (talk) 06:19, 1 November 2009 (UTC)

[edit] Scope of this guideline

This guideline implicitly defines its scope as all medical articles. While reasonable in principle this is probably not precisely what is intended, or at least should not be. I believe the situation is parallel to WP:BLP, which generally talks about biographic articles but really means biographic information in any article.

Here are two real examples of the problem.

• Thalidomide doesn't have a section "Thalidomide in popular culture", but it does have something that comes close: Thalidomide#Notable children affected.
• List of common misconceptions#Human body and health contains medical advice about sugar/hyperactivity, wet weather/common cold, sleepwalking, vitamin A/eyesight and duct tape/warts.

The first problem is addressed by a paragraph in WP:MEDRS#Popular press. It makes clear that for non-medical sections in medical articles there is a relaxed standard. It does not try to define this standard; clearly such an attempt would be out of proportion in this guideline. It would be more efficient to delegate this to WP:RS.

I can't see that the more serious second problem is addressed anywhere: Medical information in List of common misconceptions seems to be outside the scope of this guideline! Therefore I propose adding a new section at the end of the lead:

"For simplicity this guideline refers to medical articles. This is not meant to restrict or define its scope: This guideline applies to medical information in all articles.

Hans Adler 11:12, 2 November 2009 (UTC)

You're absolutely right that the guideline is intended to apply to all medical facts and figures, regardless of whether the overall article topic is exclusively medical; this is already indicated by the guideline's title of medicine-related articles). But rather than introduce that disclaimer, I think it'd be better to systematically fix the language in the guideline to make this clearer; that way readers won't have to know about the disclaimer in order to understand the rest of the article. I did that (it was a surprisingly small change). Eubulides (talk) 18:10, 2 November 2009 (UTC)

Not sure if this adds anything, but the WP:FA category is called "Health and medicine", and there's also the "Psychology" and "Biology" categories. SandyGeorgia (Talk) 18:27, 2 November 2009 (UTC)

Hans, is there an active dispute that centers around this distinction, or is this purely hypothetical? WhatamIdoing (talk) 03:27, 3 November 2009 (UTC)

Neither. I am afraid there may be a dispute at List of common misconceptions sooner or later. I feel rather strongly about us not distributing medical misinformation based on random newspaper reports. And as you know people tried to argue that MEDRS must be applied to our reporting of a political controversy in the past, at Aspartame controversy. Hans Adler 07:53, 16 November 2009 (UTC)

Excellent thread Hans, and thanks to Eubulides for the simple fix. (I notice a couple places where more of this type of fix can be done. I'll take a whack at it.) MEDRS doesn't replace RS, but supplements it: "These guidelines supplement the general guidelines at Wikipedia:Reliable sources..." In any given medical article or article that contains biomedical information, MEDRS applies to the scientific nitty gritty details, not to the political, controversial, historical and cultural details. That is still covered by RS. That which is falsifiable will often come under the scope of this guideline. -- Brangifer (talk) 14:59, 16 November 2009 (UTC)

[edit] Aspartame controversy

I am in doubt about a reference that has been added. To me it sounds like dubious woo-woo advice by pushers of dietary supplements that isn't backed up with good evidence. The ref is being used to justify mentioning that aspartame can produce migraines. While there is a subset of individuals that are sensitive to it, the ref is being used to imply that it causes migraines, without qualifying the subset it applies to:

• The diff

• The referenced article

The edit seems to be pushing an unjustified and POV use of the source. Brangifer (talk) 04:43, 11 November 2009 (UTC)

Sheesh. Why not simply reword the statement to read

In some people, aspartame may act as a migraine trigger, as may many other food additives and naturally occurring substances.^[1]

—Fvasconcellos (t·c) 13:12, 11 November 2009 (UTC)

Thanks, I added wording along those lines, and followed up at Talk:Aspartame controversy #Association with migraine. Eubulides (talk) 19:03, 11 November 2009 (UTC)

Excellent, glad I could help :) Fvasconcellos (t·c) 19:42, 13 November 2009 (UTC)

[edit] Talk:Thiomersal controversy#Introduction Neutrality Dispute

An IP editor (131.215.40.141 and 131.215.6.110) seems to need advice regarding the importance of not depending on primary sources, on the difference between in vitro and in vivo, and on how the dose makes the poison. -- Brangifer (talk) 04:51, 17 November 2009 (UTC)

You might want to move this thread to WT:MED, which is watched by about four times the number of editors. WhatamIdoing (talk) 06:02, 17 November 2009 (UTC)

OK. -- Brangifer (talk) 06:29, 17 November 2009 (UTC)

[edit] Medicinal clay

This article needs to be gone through. I suspect there are many false claims backed up by sources that appear to be official medical journals, but are really fringe journals not even listed in PubMed. -- Brangifer (talk) 15:23, 22 November 2009 (UTC)

This is the talk page for discussing improvements to the WP:Reliable sources (medicine-related articles) page. When you want to let people know that specific articles are in need of responses, you will get more responses at WT:MED.

I would be surprised if this Alternative medicine topic didn't have some non-mainstream sources. AltMed topics are like that. WhatamIdoing (talk) 17:12, 22 November 2009 (UTC)

They don't seem to appreciate me coming there instead of here. It seems that this is the closest thing to an "WP:RS/N" for MEDRS topics, so that's why I'm starting here. (We really should have a section of WP:RS/N for such matters: WP:RS/MEDRS) I expect alt med topics to contain non-mainstream sources, but they shouldn't have undue weight and shouldn't be misleading. If they stand alone, then they mislead readers. MEDRS guidelines are supposed to correct such imbalances. -- Brangifer (talk) 22:55, 22 November 2009 (UTC)

[edit] Hair analysis (alternative medicine)

This section seems to be one huge special pleading section with peacock language and undue weight given to alt med advocates like Haley and Blaxil from SafeMinds, and ancient claims made by Bowen. (Yes, believe it or not, Bowen does rant about Satan, Zionism, and the Illuminati, and in his letter to Martini explains why his medical license was revoked.) While the fringe position should be stated clearly, it shouldn't be allowed to deceive the public by making their claims seem plausible through a presentation marred by undue weight. -- Brangifer (talk) 00:10, 23 November 2009 (UTC)

[edit] Need for WP:MEDRS/N noticeboard

I have started a thread and hope readers here will chime in there:

We need a parallel noticeboard to deal with medical sourcing: WP:MEDRS/N. Currently such matters end up at Wikipedia talk:Reliable sources (medicine-related articles) and Wikipedia talk:WikiProject Medicine, but that's unsatisfactory. We need a separate noticeboard which is watched by medical professionals who understand the complexities of medical matters and medical research, understand our RS policy, and understand the MEDRS guideline. What think ye?

Please discuss it there, not here. -- Brangifer (talk) 00:50, 23 November 2009 (UTC)

[edit] Theories of dyslexia

Could someone have a glance at activity at Theories of dyslexia? It seems to be going wildly uncompliant with WP:MEDRS, with editors working by collecting multiple primary sources to support statements. Gordonofcartoon (talk) 22:40, 23 November 2009 (UTC)