The Devil and the Dome: LLM Response Sampling

Imagine the whole of humanity (yourself included) has been captured under a giant dome by the devil.

He gives you a single challenge:

“Present me with an amazing novel, and I will let everyone out.”

You, thankfully, are not alone. Under the dome are brilliant novelists, poets, editors, critics, weird experimentalists, obsessive plotters, dialogue savants, and millions of people with taste. Collectively, humanity has read everything, written everything, argued about everything, and developed every possible instinct about what makes a story work.

There is just one catch.

You only get to submit one attempt.

The issue then, is not whether someone under the dome is capable of writing something amazing. Of course they are. The issue is whether you can reliably produce an amazing submission under uncertainty, with limited time.

Strategy #1: Council

After the devil leaves, you immediately form a council of carefully selected specialists to support this effort.

One cares about structure. One about character. One about style. One about emotional force. One about logic. One about risk. One about clarity. One about what the reader will actually feel.

At first, each of them writes their own independent submission.

They then read one another’s work. They attack weak spots. They defend what is worth saving. They point out what feels false, what feels thin, what breaks under pressure, and what still has life in it.

At the end, you appoint a chair to read the submissions, read the arguments, and choose what survives into a final manuscript.

However, despite the complex effort, the story while technically solid, reads emotionally flat. The too-many-hands approach has collapsed toward a smoothed, over-defended, least-offensive answer. This cannot go to the devil.

You could probably pull the council back into alignment – remind them that their mission is to sharpen a strong vision, not edit toward consensus… but time is running out, so you try a new approach.

Strategy #2: Fusion

Next, you assemble every skilled author under the dome, and ask them to create a story. You find the best editor available to humanity and assign them the task of reading all the submissions, and fusing together the best parts of every story into one epic novella.

At first glance, the story has everything. It feels almost unbeatable.

One author wrote the strongest opening. Another found the sharpest dialogue. Another invented the most memorable villain. Another delivered the best ending.

The editor now has an apparent superpower: to harvest the best parts from all of them and assemble a final work from humanity’s greatest fragments.

However, stories are not just collections of individually excellent parts. They are coherent wholes. A brilliant opening in one voice may not belong with a middle written in another. A character’s motivation may stop making sense once they are dropped into somebody else’s plot. A twist borrowed from one draft can introduce contradictions or plot holes into an ending borrowed from another. What looked like “the best parts” in isolation can become strangely awkward when stitched together.

So fusion offers a seductive promise: keep every strength, discard every weakness. But, for a story at least, the result is failure.

Quickly running out of time, you try a final approach.

Strategy #3: Best-of

Instead of forcing humanity to collaborate, you ask each author under the dome to write their own story independently. No sharing drafts. No debate. No compromise. Just each person’s vision, unmodified.

One writes a tragic literary epic. Another writes a dark fantasy. Another produces a strange philosophical fable. Another leans into suspense. Another into beauty. Another into raw emotional power.

When they are finished, you task your famed editor to gather every manuscript, come up with objective criteria for scoring, read them side by side, and score them all.

Most are flawed. Some are forgettable. A few are wildly ambitious and collapse under their own weight. But one of them is undeniable.

It has a voice. A shape. A kind of internal conviction the others lack. It is not the average of many good ideas, nor the compromise of a committee. It is a complete thing – coherent, distinctive, alive.

The brute-force approach, which seemed so inelegant as to be a final resort, was in fact the most elegant.

Its strength is that it preserves vision. No strong idea is sanded down by consensus, and no singular voice is diluted by committee pressure.

Under the dome, Best-of succeeds because it gives humanity many independent shots at brilliance – and then the wisdom to choose.

With that lesson learned, you hand the pages to the devil.

What this was really about

At this point, the analogy should be clear.

The devil is the user of an AI system, demanding an answer.

Each person under the dome represents an individual large language model. The way those people are coordinated, compared, and combined creates the multi-agent system. Some architectures create councils. Some fuse outputs. Some rely on many independent attempts and then select the strongest.

So while this may have looked like a story about humanity trying to impress the devil, it was really a story about sampling and orchestration.

Without realising it, you have just walked through three response-level sampling strategies:

  • Council-of-N, where N specialist agents each produce an independent draft, critique the others, and a chair synthesizes the final response
  • Fusion-of-N, where N outputs are merged into one
  • Best-of-N, where N independent attempts are generated and the strongest is chosen

There are of-course, many more of these strategies. And importantly, none are universally best.

Best-of-N saved humanity here because the devil asked for a novel, a task that rewards voice, coherence, and a singular vision.

But imagine if the devil had asked for something else.

“Draft a peace treaty between two countries.”

That might have rewarded a Council-of-N approach, where many perspectives pressure-test answers for weaknesses, blind spots, and unintended consequences.

“Write a practical guide to all the best pubs in Britain.”

That might have rewarded Fusion-of-N, where different writers contribute knowledge of different regions, styles, and experiences, and an editor combines them into one complete guide.

The lesson is simple: the task determines the strategy.

If you want to experiment with these kinds of sampling strategies yourself, you can do so at https://getnchat.com, which was built using a combination of them.

-hiburn8


Leave a comment