magic you decide to perform, concierge experiments are less risky than building out the entire product and then seeing what happens. WIZARD OF OZING IT Digital product simulation actually dates back to a 1983 experiment done at IBM by J.F. Kelley. In his whitepaper titled “An Empirical Methodology for Writing User-Friendly Natural Language Computer [54] Applications,” he unveils a methodology he calls the “OZ Paradigm.” Kelley describes his test as “an experimental simulation in which participants are given the impression that they are interacting with a program that understands English as well as another human would. In fact, at least in the earlier stages of development, the program is limping along, only partly implemented. The experimenter, acting as ‘Wizard,’ surreptitiously intercepts communications between participant and program, supplying answers and new inputs as needed.” We all know this famous “wizard” character that Kelley was referring to from The Wonderful Wizard of Oz. In this classic children’s story, the wizard convinces Dorothy that he is “great and powerful” by using magic tricks and props. His incredible simulations lead her to believe that he is the only man [55] capable of solving her problem and those of her companions. The concept of creating simulations of new technologies to test out innovative concepts is core to gaining early validation. Essentially, J.F. Kelley was doing Build-Measure-Learn feedback loops, exposing many users to his product who had never even used a computer before without completely building out the entire, complicated interface. He gained validated user research by using humans instead of machines to simulate just enough artificial intelligence to make his system seem real.