Mechanism design for large language models

For example, in the situation illustrated in the figure above, the shared sequence of tokens might be “Mechanism Design for”. The distributions might be [(“Large”, 0.8), (“Generative”, 0.2)] for the LLM of Agent 1, (“Large”, 1.0) for the LLM of Agent 2, and (“Generative”, 1.0) for the LLM of Agent 3. The bids might be 1, 2, and 2, respectively. A possible aggregated distribution would be the bid-weighted average of the distributions, namely [(“Large”, 0.56), (“Generative”, 0.44)]. A possible choice for the payments would be to ask each agent to pay their bid, which would have the agents commit 1, 2, and 2, respectively.

For our theoretical analysis of this model (and possible choices of distribution aggregation functions and payment functions), we assume that the agents truthfully report their distributions, but may be strategic about their bids. We believe this is a realistic assumption, as LLMs encode preferences over output text in a succinct and non-obvious way. Moreover, in order for the token auction to be able to aggregate distributions, we need to have (at least) some (minimal) information about agent’s preferences away from their “preferred” distributions. Our approach here is to assume that the agents have (known) partial preference orders over distributions. That is, we assume that agents may be able to rank some, but not all, pairs of distributions.