The second visit of the Analytics Genie was fully as unexpected as the first. The sudden appearance of mist billowing out of nowhere in the corner of my study and the image of an imposing figure in a turban materializing within it is not the sort of thing one easily gets used to.
The genie stepped forward, touching his hand to his forehead in greeting – a little less theatrical than the sweeping bow he had executed on his first visit – and addressed me in a tone as if he had only left my presence moments before. "Have you made progress in choosing your second wish?"
As I struggled to pull my memories back into focus, the genie prodded me by saying "Or would you like to revise your first wish?"
Then it came back to me in a rush. The Analytics Genie had stipulated that my three wishes were to be about features I wanted to have in my models and methods for analytics problems. My first wish for such a feature had been
" … a simulation optimization capability with the best possible design."
As I reflected again on the critical elements that analytics models and methods should have, I was led back to my initial realization that an ability to handle uncertainty, risk and complexity was at the top of the list. And simulation optimization was a pre-eminent candidate for achieving this, by combining the modeling flexibility of simulation with the capacity to uncover high quality decisions for meeting the goals of the model. The more advanced forms simulation optimization also had the advantage of employing metaheuristic search in addition to classical optimization, making them superior to approaches that only relied on traditional optimization processes.
Still, asking for a "best possible design" of a simulation optimization capability was not enough to fully convey what I really was looking for. There was immense variability in the quality of simulation optimization methods. I needed to clarify the factors that define this "quality" by pinpointing the characteristics that lay at the heart of the better methods.
My reflections since the Analytics Genie’s first visit had produced the realization that a conspicuously desirable characteristic of good simulation optimization methods is to go beyond producing a single good solution. In other words, good methods should generate a strategically varied collection of solutions. The value of such a collection was apparent in practice. It was no secret that managers and decision makers are often unable to envision all of the intricacies of the problems they face when they first undertake to formulate them. The argument I had traced in my reflections went something along the following lines. A repository of high quality candidate solutions avoids being saddled with a "best" solution that is defective – due to unanticipated limitations in the model and the difficulty of accounting for all relevant problem characteristics. Instead, when a given solution turns out to have shortcomings, the decision maker can select among repository members that are better suited to practical implementation. The process of reviewing alternative candidate solutions builds insights that can lead to better models and, in turn, to better solutions.
All well and good, but that still wasn't enough to differentiate "high quality" systems from others. Another hallmark of good simulation optimization methods is an ability to handle multiple objectives. Practical problems often occur in a context where the preferred policy is not only to maximize profit or minimize expense, for example, but simultaneously to maximize resource utilization and customer satisfaction, or to minimize idle production or detrimental environmental impact, and so forth. Tradeoffs typically encountered in these settings render it impossible to find a solution that is best across all relevant criteria simultaneously. A simulation optimization approach that can effectively handle multiple objectives offers a bonus, not only by accounting for such tradeoffs, but by also giving a platform for responding to uncertainty and risk. Taking advantage of such an approach assures that the CEO and her team will not wake up the next morning and discover that an unforeseen event has wiped out their capital – as a result of a strategy that was tailored to achieve one goal at the expense of another, or that seemed highly profitable at the moment, but that fell apart when conditions changed.
In fact, the ability to deal with multiple objectives, if done intelligently, will automatically include the feature I had previously singled out as important – the feature of maintaining a repository of strategically generated solutions. Such solutions can be relevant in a multiple objective setting even if not all of them reside on the efficient frontier of so-called "Pareto optimal" solutions, because the objectives as presently formulated may not encompass all the elements that may emerge as important.
In short, my second wish followed naturally on the first, and I did not need to keep the genie waiting to hear what I had decided. Again, to avoid the chance of getting something less than I might hope for, it would be preferable to formulate my wish by asking for the "best possible" alternative. I turned to the Analytics Genie who was patiently awaiting my response and said:
"My second wish for a feature to include in an analytics system is to have the best possible capability to handle multi-criteria optimization."
The genie raised an eyebrow. But he did not seem dissatisfied, and I chose to interpret his gesture as not so much questioning my second wish as signaling his acknowledgement that it fit with the first.
"I will give you the same opportunity as before," he said. "You have the chance to change your second wish as you go through the process of selecting the third."
I thought I knew what was coming. But instead of stepping back into the mists, the genie remained in place and the mists came to him. As they enveloped him and he began to fade from view, he flashed a curious smile in my direction. Something about his expression gave me the distinct impression he had not told me everything. I resolved that the next time I saw him I would ask more questions and try to piece together more fully what was going on.