16  Inspecting post-publication notices ✎ Draft

Publication is not the end of an article’s history. After an article is published, its journal can correct it, warn readers about it, or retract it. These post-publication notices are the simplest thing to check in any trustworthiness assessment, and the first: they are checks 1.1 and 1.2 of INSPECT-SR (Section 24.2). A retracted article should usually not be relied on at all, whatever its methods and results look like.

This chapter explains what each kind of notice means, who decides to issue one, what a notice can and cannot tell you, and how to find them. The last of these is harder than it should be.

16.1 Types of post-publication notice

The Committee on Publication Ethics (COPE) is an organisation whose members include most major journals and publishers. Its guidelines are the closest thing to a standard for how journals handle problems with published articles, although they are advice rather than rules, and journals follow them unevenly. COPE describes three main kinds of notice, in increasing order of seriousness.

16.1.1 Corrections

A correction (sometimes called an erratum when the error was the publisher’s, or a corrigendum when it was the authors’) fixes an error in an article that is otherwise reliable: a wrong number in a table, a mislabelled figure, a missing funding statement. The article stays in the literature, and the correction is published as a separate notice linked to it. Many articles have corrections, and most are minor. But read them: a correction that changes a main result, or a series of corrections to the same article, can be a concern in itself.

16.1.2 Expressions of concern

An expression of concern warns readers that there are serious, credible doubts about an article’s reliability, but that the journal cannot yet decide whether to correct or retract it (COPE Council, 2025a). Typical reasons are that an investigation by the authors’ institution is still ongoing, that the journal is waiting for the authors to provide information such as their data, or that the authors have refused to provide it. An expression of concern can be a temporary notice, later replaced by a correction or a retraction, or it can be the final outcome if the case is never resolved. For a trustworthiness assessment, it is a strong warning: the journal itself is saying that the results may not be reliable.

In practice, journals rarely use them. Grey et al. (2026) extracted all the expressions of concern and retractions in the Retraction Watch Database up to the end of 2024.

3.3%

of retracted articles had been given an expression of concern before they were retracted (Grey et al., 2026).

Even among authors with more than five retractions, only 35% had any expression of concern on any of their articles. For most retracted articles, then, readers received no warning while the journal investigated, which can take years (Nature, 2014). Throughout that time the articles looked like any other published article and continued to be read, cited, and included in systematic reviews. The absence of an expression of concern therefore says very little about whether a journal has concerns about an article.

16.1.3 Retractions

A retraction is the most serious notice. It tells readers that the article’s findings and conclusions cannot be relied on (COPE Council, 2025b). Retraction is often described as “unpublishing” an article, but the article does not usually disappear. It normally stays online, marked as retracted (often with a “RETRACTED” watermark on every page) and linked to a retraction notice that explains, in principle, why. This keeps the record intact, so that readers who come across the article, or across citations of it, can find out that it has been retracted. Only rarely is an article removed entirely, for example for legal reasons or because following it could put people’s health at risk.

A retraction does not necessarily mean misconduct. COPE’s guidelines describe retraction as a way to correct the literature, not to punish the authors, and unreliable findings can come from honest error as well as from fabrication, falsification, plagiarism, or paper mills. Some retractions are requested by the authors themselves after they find a serious mistake. For a trustworthiness assessment this distinction matters less than it might seem: either way, the article’s results should not be used.

Retracted articles continue to be cited, often without any sign that the citing authors know about the retraction. This is easy to do. Most researchers have little reason to check whether an article they already know has been retracted, or to revisit its page on the publisher’s website, which is usually the only place the retraction is shown. Once an article is in your reference manager, nothing in a typical workflow removes it from the articles you can cite, so it can be cited for years after its retraction. Zotero helps with this: since 2019, it flags retracted articles in your library, shows the details of the retraction, and warns you if you try to cite one, or if a document already cites one that is later retracted (Stillman, 2019). It does this by checking the DOIs and PubMed IDs in your library against the Retraction Watch Database, the same database used by the checker in Section 16.6, without sending your library to Zotero’s servers. For this reason, I recommend Zotero as a reference manager.

Section 17.4 covers how to check an article’s references for retracted work.

16.2 Who decides?

Under COPE’s guidelines, the decision to correct, issue an expression of concern about, or retract an article belongs to the journal’s editor, because the editor is responsible for the journal’s content (COPE Council, 2025b). The publisher can support the editor, advise on how to handle a case, and issue the notice jointly, but should not make the decision, so that commercial interests do not influence it. While many (but not all) journals are COPE signatories, there is still a great deal of heterogeneity in processes. This may be due in part to editors not being aware of their own power. For example, editors sometimes ask authors’ institution to investigate, or defer to publisher’s research integrity offices. Institutional investigations can take years, some never happen or are never reported, and many people feel that they suffer from an inherent conflict of interest (e.g., an institution has an incentive to minimise reputation damage due to its employees behavior). Different entities also make judgement about different things: COPE states that an editors role is to retract if they have lost confidence in the work, not to adjuicate whether research integrity violations took place. These are often conflated - for example, sufficient ambiguity can lead to an editor losing confidence in a specific article, even if there is not sufficient evidence for a university research integrity committee to assert that research misconduct occurred. The point of this section is not to overly editorialise about who should make a decision here, only to note that despite clarity about who can make a decision, about what, on what timeline, and in conjunction with what other entities.

The result is that notices are slow, and their absence means little. Many articles with serious problems have no notice at all, and many that are eventually retracted were cited for years before the retraction.

16.3 What retraction notices do and do not tell you

A retraction notice is not a full account of what happened and should not be read as one. It is the most conservative statement the journal is confident it can defend - possibly in court.

Notices are often (partially) negotiated with the authors, and checked by the publisher’s lawyers before they are published. Because an author who is named in a notice as having committed misconduct may sue, this is rarely stated outright. In a 2014 editorial, Nature explained that “for legal reasons” it could not usually use retraction statements to highlight wrongdoing (Nature, 2014). The result is notices that say little: that “concerns were raised about the data”, that the editor “no longer has confidence in the findings”, or that the authors “could not provide the raw data”. In an analysis of 7,318 retraction notices, 73.7% did not say whether any institutional investigation had led to the retraction (Xu et al., 2023).

Notices can also show signs of negotiation directly. The retraction notice for the article described in Section 17.1 ends by stating that the author “agrees with this retraction but for different reasons than those stated here” (American Psychological Association, 2026).

So a notice that does not mention misconduct is not evidence that there was none, and a vague reason such as “error” may cover anything from a typo to fabrication. For more context, look for other sources: PubPeer comments, coverage by Retraction Watch, and published reports of institutional investigations. For a trustworthiness assessment, though, what matters most is usually the fact of the retraction rather than its stated reason. A retraction for any reason means the article’s results should not be used, and a retraction for misconduct also raises questions about the authors’ other work (INSPECT-SR check 1.3).

Relatedly, whereas there is a normative requirement in science that authors of research be given fair credit by being named as authors of articles, there is no comparable norm in retraction notices to give fair credit to the people who detected the problems. Most notices mention that issues were raised by “a reader”, for example, but not by name. While I understand there may be legal reasons for doing so, this contributes to the perverse incentive structures in science.

16.4 There is no single database of retractions

It would be reasonable to expect a single, complete list of retracted articles. Surprisingly, there is none. Retractions are recorded in several places, and they often disagree.

Publishers are supposed to register notices with Crossref, the organisation that registers the DOIs of most journal articles, but many do not, or do so incompletely. PubMed and commercial indexes such as Web of Science and Scopus each record retractions in their own way, for the journals they cover. The Retraction Watch Database, compiled by hand by Retraction Watch from publishers’ websites and other sources, is the most comprehensive, and Crossref acquired it in 2023 and now makes it openly available.

Schneider et al. (2023) combined the retractions recorded by Crossref, the Retraction Watch Database, Scopus, and Web of Science, giving 49,924 publications that at least one source recorded as retracted.

3%

of publications recorded as retracted by at least one of four major sources were recorded as retracted by all four (Schneider et al., 2023).

For almost 30% of the publications, one source recorded a retraction that another, which indexed the same publication, did not. Of the retracted publications that Scopus indexed, it failed to mark 32% as retracted, compared with 7% for Crossref and 4% for Web of Science.

The practical consequence is that checking one source is not enough. The checker below therefore combines four:

1. notices that publishers have registered with Crossref;
2. the Retraction Watch Database, which Crossref now includes alongside the publishers’ own records;
3. PubMed, which links articles to retraction notices and expressions of concern, and tags retracted articles;
4. the article’s title, because some publishers mark a retraction only by changing the title (e.g., “RETRACTED: …”).

This approach follows, and was inspired by, the “Annulled” and “Concerning” detectors of the Problematic Paper Screener (Cabanac et al., 2022), which combine the same sources. The difference is that this checker queries the sources live, whereas the Problematic Paper Screener’s lists are updated periodically.

16.5 PubPeer and its browser extension

Formal notices are only part of an article’s post-publication history. PubPeer is a website where anyone can comment on published articles, and it has become the main place where problems with articles are reported and discussed. Comments are moderated, may be posted anonymously, and are often detailed: they report duplicated or manipulated images, impossible statistics, inconsistencies with the trial registration, nonexistent references, and the other problems covered in this book. Authors can reply, and commenters often post updates when a correction, expression of concern, or retraction is published, as in the example in Section 17.1. Because journals are slow to act, and because most retractions are not preceded by an expression of concern, a PubPeer thread is often the first, and sometimes the only, public sign that an article has problems.

PubPeer comments are not verdicts. Read the thread, check the claims it makes, and treat it as information to assess and verify, not as a conclusion. When reading and writing PubPeer comments, remember that a good comment is one that provides verifiable information that might adjust our trust in the work.

Many of the arguments made against PubPeer are straw men that neglect the principled reasons it works the way it does.

Some argue that PubPeer comments are not legitimate because they have not been peer reviewed, although PubPeer has never claimed that they are. Its moderators check comments against its rules, not their scientific merit, and readers are expected to judge each comment on the evidence it presents (PubPeer FAQ). There are also good reasons why science needs a way to verify published work other than critiques published by journals, because journals make such critiques difficult. Of 330 top-ranked journals across 22 disciplines, only 63% accepted post-publication critique at all, and those that did often limited it (Hardwicke et al., 2022).

23%

of a random sample of psychology journals offered an explicit way to submit a critique of a published article (Whamond et al., 2025).

1,000 words

was the median length limit for critiques of published articles at top-ranked journals (Hardwicke et al., 2022).

12 weeks

was the median time allowed after the original article’s publication to submit a critique (Hardwicke et al., 2022).

Even among the most prominent psychology journals, only 38% did, and almost no articles in either sample had a published critique (Whamond et al., 2025). Concerns about data integrity often take far longer than a few weeks to find, and need more than 400 words to document. Journal limits were found to be as strict as 175 words and two weeks (Hardwicke et al., 2022). These represent severe misalignment between publishing processes and the putative goal of the scientific record, which is to pursue truth. PubPeer is, arguably, a symptom of the very flawed way that critiques of published work are treated by the publishing system.

Others argue that PubPeer comments are mostly about trivial points such as typos, or are unprofessional personal attacks. The large-scale evidence says otherwise. In a classification of 17,244 articles commented on PubPeer, the comments on 82.9% reported manipulation, potential publishing fraud, methodological flaws, or honest but serious errors. The largest single category was manipulation of data or images (Ortega & Delgado-Quirós, 2023; see also Ortega, 2022). PubPeer’s own rules require comments to be based on “publicly verifiable information” and polite, neutral, and factual. They forbid allegations of misconduct and personal comments about authors, and moderators do not publish comments that break them (PubPeer FAQ).

Some authors object to anonymous PubPeer comments, without asking why anonymity is allowed. Researchers who report serious concerns about published work have repeatedly faced legal action. In 2014, the cancer researcher Fazlul Sarkar sued to make PubPeer reveal the identities of anonymous commenters who had pointed out problems with images in his papers. PubPeer, represented by the American Civil Liberties Union, won on appeal in 2016 (The Scientist), and Sarkar has since had dozens of papers retracted (Retraction Watch). In 2023, Francesca Gino sued the three researchers behind the Data Colada blog, who had reported evidence of data fabrication in her work, for $25 million for defamation. The claims against them were dismissed in 2024 (Poets & Quants), and Harvard revoked her tenure in 2025. Even unsuccessful lawsuits cost the people who are sued time, money, and stress, and anonymity protects people, especially early-career researchers, who could not otherwise afford to report what they find. In the Gino/Data Colada case, the researchers’ universities initially said they would not support their legal defense as critique of others’ research was not part of their official duties. After public outcry and a GoFundMe campaign raised more than $250k, the universities eventually agreed to cover some of the legal costs - but likely for PR reasons more than any commitment to the idea that the lawsuit was a threat to academic freedom.

PubPeer’s browser extension, available for Chrome, Firefox, Edge, and Safari, checks every article you read for PubPeer comments and shows a banner at the top of the page if it finds any. This turns PubPeer from a site you have to remember to search into one that alerts you as you read. I highly recommend installing it.

16.6 Checking for notices

Enter a DOI below to check for post-publication notices (retractions, expressions of concern, and corrections). The checker can be used for INSPECT-SR checks 1.1 and 1.2 for a single article, for check 1.3 by entering the DOIs of the research team’s other articles, and for checking the references of an article (Section 17.4).

Even four sources do not give complete coverage, so a clean result does not rule out notices that were never registered. Also check the publisher’s page and PubPeer, using the links in the result.

viewof doi = {
  const initial = "10.1177/0269215520923135";
  const examples = [
    {label: "Retracted", doi: "10.1177/0269215520923135"},
    {label: "Not retracted", doi: "10.1177/25152459241287123"}
  ];

  const input = html`<input class="form-control font-monospace" type="text" spellcheck="false" autocomplete="off"
    aria-label="DOI" placeholder="e.g., 10.1177/0269215520923135 or https://doi.org/...">`;
  input.value = initial;
  // Only update the result when the form is submitted, not on every keystroke
  input.addEventListener("input", e => e.stopPropagation());

  const form = html`<form class="input-group">
    <span class="input-group-text">DOI</span>
    ${input}
    <button class="btn btn-primary" type="submit"><i class="bi bi-search"></i> Check</button>
  </form>`;

  const exampleRow = ({label, doi}) => {
    const useBtn = html`<button type="button" class="btn btn-outline-primary" title="Check this DOI">Try it</button>`;
    const copyBtn = html`<button type="button" class="btn btn-outline-secondary" title="Copy DOI"><i class="bi bi-clipboard"></i></button>`;
    useBtn.onclick = () => { input.value = doi; run(); };
    copyBtn.onclick = async () => {
      try {
        await navigator.clipboard.writeText(doi);
        copyBtn.innerHTML = `<i class="bi bi-check2"></i>`;
      } catch (e) {
        copyBtn.innerHTML = `<i class="bi bi-x"></i>`;
      }
      setTimeout(() => (copyBtn.innerHTML = `<i class="bi bi-clipboard"></i>`), 1500);
    };
    return html`<div class="example"><span class="example-label">${label}</span><code>${doi}</code>${useBtn}${copyBtn}</div>`;
  };

  const root = html`<div class="doi-checker">
    ${form}
    <div class="examples">
      <div class="examples-title">Examples</div>
      ${examples.map(exampleRow)}
    </div>
  </div>`;

  root.value = initial;
  function run() {
    root.value = input.value;
    root.dispatchEvent(new Event("input", {bubbles: true}));
  }
  form.onsubmit = e => { e.preventDefault(); run(); };
  return root;
}

async function lookupDoi(d) {
  const [cr, pm] = await Promise.all([lookupCrossref(d), lookupPubmed(d)]);
  if (cr.status !== "ok" && pm.status !== "ok") {
    return {status: "error", message: cr.message, pubmed: pm};
  }
  // Use Crossref for the reference where possible, otherwise PubMed
  const ref = cr.status === "ok" ? cr : pm;
  const notices = [
    ...(cr.status === "ok" ? cr.notices : []),
    ...(pm.status === "ok" ? pm.notices : []),
    ...titleNotices(ref.title)
  ];
  const types = new Set(notices.map(n => n.type));
  const kind = types.has("retraction") ? "retracted"
    : types.has("concern") ? "concern"
    : notices.length ? "other"
    : "clean";
  return {
    status: "ok", authors: ref.authors, year: ref.year, title: ref.title, journal: ref.journal,
    notices, kind, isNotice: cr.status === "ok" && cr.isNotice, crossref: cr, pubmed: pm
  };
}

cleanText = x => (x ?? "").replace(/<[^>]+>/g, "").replace(/\s+/g, " ").trim();

function formatAuthors(names) {
  return names.length === 0 ? "Unknown authors"
    : names.length === 1 ? names[0]
    : names.slice(0, -1).join(", ") + (names.length > 2 ? "," : "") + " & " + names.at(-1);
}

async function lookupCrossref(d) {
  let r;
  try {
    r = await fetch(`https://api.crossref.org/works/${encodeURIComponent(d)}`);
  } catch (e) {
    return {status: "error", message: "Could not reach Crossref. Check your connection and try again."};
  }
  if (r.status === 404) return {status: "notfound", message: "This DOI was not found in Crossref or PubMed. It may be mistyped, or registered with another agency (e.g., DataCite)."};
  if (!r.ok) return {status: "error", message: `Crossref returned an error (${r.status}). Try again later.`};
  const m = (await r.json()).message;

  const typeOf = t => ["retraction", "withdrawal", "removal"].includes(t) ? "retraction"
    : t === "expression_of_concern" ? "concern"
    : ["correction", "erratum", "corrigendum"].includes(t) ? "correction"
    : "other";
  // Deduplicate: Crossref can list the same notice more than once
  const seen = new Set();
  const notices = (m["updated-by"] ?? []).filter(u => {
    const key = `${u.type}|${(u.DOI ?? "").toLowerCase()}|${u.source}`;
    return seen.has(key) ? false : (seen.add(key), true);
  }).map(u => ({
    type: typeOf(u.type),
    label: u.label ?? u.type,
    date: u.updated?.["date-parts"]?.[0]?.map(x => String(x).padStart(2, "0")).join("-"),
    source: u.source === "retraction-watch" ? "Crossref (Retraction Watch)" : "Crossref (publisher)",
    url: u.DOI && u.DOI.toLowerCase() !== d.toLowerCase() ? `https://doi.org/${encodeURI(u.DOI)}` : null
  }));

  return {
    status: "ok", notices,
    authors: formatAuthors((m.author ?? []).map(a => a.family ?? a.name).filter(Boolean)),
    year: (m.issued ?? m.published ?? m.created)?.["date-parts"]?.[0]?.[0] ?? "n.d.",
    title: cleanText(m.title?.[0]) || "(untitled)",
    journal: cleanText(m["container-title"]?.[0]),
    isNotice: (m["update-to"] ?? []).some(u => (u.DOI ?? "").toLowerCase() !== d.toLowerCase())
  };
}

async function lookupPubmed(d) {
  const eutils = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils";
  try {
    const s = await fetch(`${eutils}/esearch.fcgi?db=pubmed&retmode=json&tool=trustworthyresearch&term=${encodeURIComponent(d + "[doi]")}`);
    if (!s.ok) return {status: "error"};
    const ids = (await s.json()).esearchresult?.idlist ?? [];
    if (!ids.length) return {status: "notfound"};
    const r = await fetch(`${eutils}/esummary.fcgi?db=pubmed&retmode=json&tool=trustworthyresearch&id=${ids.slice(0, 5).join(",")}`);
    if (!r.ok) return {status: "error"};
    const result = (await r.json()).result ?? {};
    // PubMed's DOI search is fuzzy, so keep only a record whose DOI matches exactly
    const rec = (result.uids ?? []).map(u => result[u]).find(x =>
      (x.articleids ?? []).some(a => a.idtype === "doi" && a.value.toLowerCase() === d.toLowerCase()));
    if (!rec) return {status: "notfound"};

    const refTypes = {
      "retraction in": ["retraction", "Retraction"],
      "retracted and republished in": ["retraction", "Retracted and republished"],
      "partial retraction in": ["other", "Partial retraction"],
      "expression of concern in": ["concern", "Expression of concern"],
      "erratum in": ["correction", "Erratum"]
    };
    const notices = (rec.references ?? [])
      .filter(x => refTypes[(x.reftype ?? "").toLowerCase()])
      .map(x => {
        const [type, label] = refTypes[x.reftype.toLowerCase()];
        return {
          type, label, date: null, citation: (x.refsource ?? "").replace(/\s*doi:.*$/i, ""),
          source: "PubMed",
          url: x.pmid ? `https://pubmed.ncbi.nlm.nih.gov/${x.pmid}/` : null
        };
      });
    // Retracted publications are also tagged with a publication type, even without a linked notice
    if ((rec.pubtype ?? []).includes("Retracted Publication") && !notices.some(n => n.type === "retraction")) {
      notices.push({type: "retraction", label: "Indexed as a retracted publication", date: null, source: "PubMed", url: null});
    }

    return {
      status: "ok", pmid: rec.uid, notices,
      authors: formatAuthors((rec.authors ?? []).filter(a => a.authtype === "Author").map(a => a.name.replace(/\s+\S+$/, ""))),
      year: (rec.pubdate ?? "").slice(0, 4) || "n.d.",
      title: cleanText(rec.title) || "(untitled)",
      journal: cleanText(rec.fulljournalname)
    };
  } catch (e) {
    return {status: "error"};
  }
}

// Some publishers mark a retraction only by changing the article's title
function titleNotices(title) {
  const marked = /^\W*(retracted|withdrawn|removed)\b/i.test(title ?? "") || /\((retracted|withdrawn)( article| paper)?\)/i.test(title ?? "");
  return marked ? [{type: "retraction", label: "Title marks the article as retracted or withdrawn", date: null, source: "Article title", url: null}] : [];
}

{
  const d = doi.trim().replace(/^(https?:\/\/(dx\.)?doi\.org\/|doi:\s*)/i, "").trim();
  if (!d) return html``;

  const doiUrl = x => `https://doi.org/${encodeURI(x)}`;
  const makeLinks = pmid => html`<div class="actions">
    <a class="btn btn-outline-primary btn-sm" href="${doiUrl(d)}" target="_blank" rel="noopener">
      <i class="bi bi-box-arrow-up-right"></i> Publisher's page</a>
    <a class="btn btn-outline-primary btn-sm" href="https://pubpeer.com/search?q=${encodeURIComponent(d)}" target="_blank" rel="noopener">
      <i class="bi bi-chat-square-text"></i> PubPeer comments</a>
    ${pmid ? html`<a class="btn btn-outline-primary btn-sm" href="https://pubmed.ncbi.nlm.nih.gov/${pmid}/" target="_blank" rel="noopener">
      <i class="bi bi-journal-medical"></i> PubMed record</a>` : ""}
  </div>`;

  yield html`<div class="doi-result loading">
    <span class="spinner-border spinner-border-sm" role="status"></span> Checking Crossref and PubMed for <span class="doi-text">${d}</span>...
  </div>`;

  const res = await lookupDoi(d);

  if (res.status === "error") {
    yield html`<div class="doi-result error">
      <span class="badge text-bg-secondary status"><i class="bi bi-exclamation-triangle-fill"></i> Could not check</span>
      <div class="citation">${res.message}</div>
      <div class="doi-text">${d}</div>
      ${makeLinks(null)}
    </div>`;
    return;
  }

  const badges = {
    retracted: html`<span class="badge text-bg-danger status"><i class="bi bi-x-octagon-fill"></i> Retracted</span>`,
    concern: html`<span class="badge text-bg-warning status"><i class="bi bi-exclamation-triangle-fill"></i> Expression of concern</span>`,
    other: html`<span class="badge text-bg-info status"><i class="bi bi-info-circle-fill"></i> Other notices (e.g., corrections)</span>`,
    clean: html`<span class="badge text-bg-success status"><i class="bi bi-check-circle-fill"></i> No notices found</span>`
  };
  const icons = {retraction: "bi-x-octagon", concern: "bi-exclamation-triangle", correction: "bi-pencil-square", other: "bi-file-earmark-text"};
  const noticeItem = n => html`<li>
    <i class="bi ${icons[n.type]}"></i>
    <strong>${n.label}</strong>
    <span class="meta"> · ${n.date ?? n.citation ?? ""}${n.date || n.citation ? " · " : ""}source: ${n.source}</span>
    ${n.url ? html` · <a href="${n.url}" target="_blank" rel="noopener">view notice</a>` : ""}
  </li>`;

  // Which sources could be checked
  const status = (name, s, found) => s.status === "ok" ? html`<span><i class="bi bi-check2"></i> ${name}${found ?? ""}</span>`
    : s.status === "notfound" ? html`<span><i class="bi bi-dash"></i> ${name}: not indexed</span>`
    : html`<span class="unavailable"><i class="bi bi-exclamation-triangle"></i> ${name}: could not be checked</span>`;
  const checked = html`<div class="checked">Checked:
    ${status("Crossref and Retraction Watch", res.crossref)}
    ${status("PubMed", res.pubmed, res.pubmed.pmid ? ` (PMID ${res.pubmed.pmid})` : "")}
    <span><i class="bi bi-check2"></i> article title</span>
  </div>`;

  yield html`<div class="doi-result ${res.kind}">
    ${badges[res.kind]}
    <div class="citation">
      <span class="who">${res.authors} (${res.year})</span>
      ${/[.?!]$/.test(res.title) ? res.title : res.title + "."}
      ${res.journal ? html`<em>${res.journal}</em>.` : ""}
      <a href="${doiUrl(d)}" target="_blank" rel="noopener">https://doi.org/${d}</a>
    </div>
    ${res.notices.length ? html`<ul class="notices">${res.notices.map(noticeItem)}</ul>` : ""}
    ${res.isNotice ? html`<div class="note">This DOI is itself a notice that updates another article.</div>` : ""}
    ${checked}
    ${res.kind === "retracted" ? "" : html`<div class="coverage"><i class="bi bi-info-circle"></i>
      <span>Crossref, Retraction Watch, and PubMed do not have complete coverage. Even if no retraction or other notice is listed here, check the publisher's page for notices, and PubPeer for comments, using the links below.</span></div>`}
    ${makeLinks(res.pubmed.pmid)}
  </div>`;
}

16.7 Test yourself

Answer all the questions, then click the button below the quiz to see why each answer is right.

Types of notice

1. A journal has serious, credible doubts about an article, but is still waiting for the authors’ institution to finish its investigation. Which notice should it publish? A correctionAn expression of concernA retractionNone until the investigation ends
2. When an article is retracted, it is usually deleted from the journal’s website. TRUEFALSE
3. A retraction shows that the authors committed misconduct. TRUEFALSE
4. For a trustworthiness assessment, an article retracted because of an honest error, with no misconduct involved, can still be used. TRUEFALSE
5. Roughly what share of retracted articles had an expression of concern before they were retracted? 3%25%50%90%
6. An article with no expression of concern is unlikely to be under investigation by its journal. TRUEFALSE

Who decides, and what notices say

7. Under COPE’s guidelines, who should make the final decision to retract an article? The journal’s editorThe publisherThe authors’ institutionCOPE
8. A retraction notice that gives “an error in the analysis” as its reason is good evidence that there was no misconduct. TRUEFALSE
9. Why do retraction notices often say little about what happened? They are written to be legally defensible, and are often negotiated with the authorsCOPE forbids notices from giving reasonsJournals rarely know why an article is being retractedNotices have strict word limits

Finding notices

10. Checking the Retraction Watch Database alone is enough to find every retraction. TRUEFALSE
11. Of the publications that at least one of Crossref, the Retraction Watch Database, Scopus, and Web of Science recorded as retracted, most were recorded as retracted by all four. TRUEFALSE
12. Why does the checker in this chapter also look at the article’s title? Some publishers mark a retraction only by changing the titleTitles are more reliable identifiers than DOIsTo detect hallucinated referencesPubMed indexes only titles
13. Zotero warns you if you try to cite an article in your library that has been retracted. TRUEFALSE

PubPeer

14. PubPeer’s moderators check each comment’s scientific merit before publishing it. TRUEFALSE
15. Large-scale analyses of PubPeer show that most comments are about trivial issues such as typos. TRUEFALSE
16. A PubPeer thread raising concerns about an article is enough on its own to conclude that the article is untrustworthy. TRUEFALSE

1. An expression of concern. Expressions of concern are for cases where there are serious, credible doubts but the journal cannot yet decide whether to correct or retract the article, for example while an institutional investigation is ongoing. COPE’s guidance is that editors should not wait silently for a decision (Section 16.1).
2. False. Retracted articles normally stay online, marked as retracted and linked to a retraction notice, so that readers who come across them, or across citations of them, can find out. Only rarely is an article removed entirely (Section 16.1).
3. False. Retraction corrects the literature; it is not a finding of misconduct. Unreliable results can come from honest error as well as from fabrication, falsification, plagiarism, or paper mills, and some retractions are requested by the authors themselves (Section 16.1).
4. False. A retraction for any reason means the journal no longer considers the findings reliable, so they should not be used. The reason matters more for what it implies about the authors’ other work (Section 16.3).
5. 3%. In the Retraction Watch Database up to the end of 2024, only 3.3% of retracted articles had a preceding expression of concern (Grey et al., 2026).
6. False. Because expressions of concern are so rarely used, most articles that are later retracted carried no warning while they were being investigated. The absence of an expression of concern says very little (Section 16.1).
7. The journal’s editor. The editor is responsible for the journal’s content. The publisher can support and advise, and issue the notice jointly, but should not make the decision (Section 16.2).
8. False. Notices are the most conservative statement a journal can defend, and are often negotiated with the authors and checked by lawyers. A vague reason can cover anything from a typo to fabrication (Section 16.3).
9. They are written to be legally defensible, and are often negotiated with the authors. An author named in a notice as having committed misconduct may sue, so notices rarely say so outright. Most do not even say whether an institutional investigation took place (Section 16.3).
10. False. There is no single complete database of retractions. The Retraction Watch Database is the most comprehensive, but the checker combines it with publishers’ notices in Crossref, PubMed, and the article’s title (Section 16.4).
11. False. Only 3% were recorded as retracted by all four sources, and for almost 30% one source recorded a retraction that another, which indexed the same publication, did not (Section 16.4).
12. Some publishers mark a retraction only by changing the title, for example by adding “RETRACTED:” to it, without registering a notice anywhere else (Section 16.4).
13. True. Since 2019, Zotero has checked the DOIs and PubMed IDs in your library against the Retraction Watch Database. It flags retracted articles, and warns you when you try to cite one, or when a document already cites one that is later retracted (Section 16.1).
14. False. Moderators check comments against PubPeer’s rules, such as being based on publicly verifiable information and avoiding personal comments, not against their scientific merit. Readers are expected to judge each comment on its evidence (Section 16.5).
15. False. In a classification of 17,244 articles commented on PubPeer, the comments on 82.9% reported manipulation, publishing fraud, methodological flaws, or honest errors (Section 16.5).
16. False. PubPeer comments are not verdicts. Read the thread, check the claims it makes, and treat it as information to verify, not as a conclusion (Section 16.5).

American Psychological Association. (2026). Retraction of “Generative artificial intelligence reliance and executive function attenuation: Behavioral evidence of cognitive offload in high-use adults” by Baldeo (2026). Technology, Mind, and Behavior. https://doi.org/10.1037/tmb0000217

Cabanac, G., Labbé, C., & Magazinov, A. (2022). The ’Problematic Paper Screener’ automatically selects suspect publications for post-publication (re)assessment. arXiv. https://doi.org/10.48550/arXiv.2210.04895

COPE Council. (2025a). COPE guidelines: Expressions of concern. Committee on Publication Ethics. https://doi.org/10.24318/mw2J3661

COPE Council. (2025b). COPE guidelines: Retraction guidelines. Committee on Publication Ethics. https://doi.org/10.24318/cope.2019.1.4

Grey, A., Avenell, A., & Bolland, M. (2026). Editorial expressions of concern are infrequent even for researchers with multiple retracted publications: An observational study. Accountability in Research, 33, 2671154. https://doi.org/10.1080/08989621.2026.2671154

Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Tzavella, L., Bendixen, T., Handcock, S. A., Köneke, V. E., & Ioannidis, J. P. A. (2022). Post-publication critique at top-ranked journals across scientific disciplines: A cross-sectional assessment of policies and practice. Royal Society Open Science, 9(8), 220139. https://doi.org/10.1098/rsos.220139

Nature. (2014). Retraction challenges. Nature, 514, 5. https://doi.org/10.1038/514005a

Ortega, J. L. (2022). Classification and analysis of PubPeer comments: How a web journal club is used. Journal of the Association for Information Science and Technology, 73(5), 655–670. https://doi.org/10.1002/asi.24568

Ortega, J.-L., & Delgado-Quirós, L. (2023). How do journals deal with problematic articles. Editorial response of journals to articles commented in PubPeer. El Profesional de La Información. https://doi.org/10.3145/epi.2023.ene.18

Schneider, J., Lee, J., Zheng, H., & Salami, M. (2023). Assessing the agreement in retraction indexing across 4 multidisciplinary sources: Crossref, Retraction Watch, Scopus, and Web of Science. 27th International Conference on Science, Technology and Innovation Indicators (STI 2023). https://doi.org/10.55835/6441e5cae04dbe5586d06a5f

Stillman, D. (2019). Retracted item notifications with Retraction Watch integration. Zotero Blog. https://www.zotero.org/blog/retracted-item-notifications/

Whamond, A., Vazire, S., Clarke, B., Moodie, N., Schiavone, S., Thibault, R. T., & Hardwicke, T. E. (2025). How do psychology journals handle postpublication critique? A cross-sectional study of policy and practice. Advances in Methods and Practices in Psychological Science, 8(3). https://doi.org/10.1177/25152459251366102

Xu, S. B., Evans, N., Hu, G., & Bouter, L. (2023). What do retraction notices reveal about institutional investigations into allegations underlying retractions? Science and Engineering Ethics, 29(4), 25. https://doi.org/10.1007/s11948-023-00442-4