Survey humility

Herding Cats and other silly ventures

A lighthearted note on why theology surveys are hard to word well, easy to critique, and still worth doing carefully.

The 2026 weekly survey project is closing.

We are deeply sorry to announce that the 2026 Christian Thought Survey weekly survey project is closing. The email platform we used experienced a bug that caused some participants to receive multiple emails, effectively spamming them. That was never intended, and we deeply apologize for the inconvenience, frustration, and loss of trust this caused.

The response from participants made clear that continuing would not be responsible. We are disappointed that we cannot carry the weekly survey project forward, and we are grateful to everyone who participated, suggested items, read the reports, or otherwise helped up to this point.

Existing public reports and archives from the weekly survey project will remain available, but no further weekly surveys, invitations, reminders, participant requests, or newsletter requests are planned.

Survey wording is where the trouble starts.

Writing surveys for theological leaders is a little like herding cats, except the cats have read the footnotes, spotted three possible meanings of authority, and would like to know why the question did not distinguish between at least five denominational contexts.

CTS tries to write items that are clear enough to answer, fair enough to trust, and short enough to finish before the participant's coffee gets cold. That is a worthy aim. It is also, on many days, a comedy of theological calibration.

The survey designer's dilemmaPrecise enough to be fair. Plain enough to answer. Brief enough to survive.

The neutral sentence problem

The same survey item can sound suspicious for opposite reasons. Conservative theologians may hear a progressive assumption tucked into the framing. Progressive theologians may hear a conservative assumption doing quiet work in the categories. Both reactions can be sincere, thoughtful, and inconveniently plausible.

Pastor-theologians add another layer of useful difficulty. They notice when a phrase sounds tidy on a form but lands awkwardly in the life of a congregation. They ask whether the wording is too broad, too narrow, too abstract, too loaded, too soft, too sharp, or missing the pastoral qualifier that would keep the whole thing from wobbling.

From one angle

Too progressive

The wording seems to smuggle in cultural accommodation before the participant has even touched the slider.

From another

Too conservative

The same wording seems to carry inherited assumptions as if they were simply the neutral starting point.

From the pulpit

Too tidy

The sentence may be technically clear and still miss the pastoral messiness people actually carry.

From the respondent

Too long

Every helpful caveat makes the item fairer, right up until the question becomes a paragraph in formalwear.

Why the work was worth trying

The goal is not to produce perfect sentences that float above every tradition untouched. Those sentences do not appear to exist, and if they do, they are probably too long for a 0-100 slider.

The more modest goal is to ask disciplined, transparent, revisable questions that help reveal where thoughtful Christian leaders agree, where they differ, and where the wording itself needs another pass.

  • Precision has to be balanced against readability.
  • Neutrality has to be balanced against recognizable theological language.
  • Pastoral nuance has to be balanced against respondent fatigue.
  • Breadth has to be balanced against the fact that every tradition names the terrain differently.

Thank you to CTS participants. Your patience made the project possible. Thank you for answering imperfect questions carefully, suggesting better wording when something felt off, and accepting that some limitations are built into this kind of work.

The weekly survey project can no longer collect corrections, better phrasing, or participant-nominated survey items for future weeks, but we remain grateful for the thoughtful help participants offered while it was active.