Some experts question industrywide popularity of Monte Carlo simulations

When it comes to using Monte Carlo simulations, Shawn Brayman says he’s heard it all. “Every extreme, from don’t touch it at all to if you don’t do it your clients will surely suffer,” said Mr. Brayman, president of PlanPlus Inc., a Lindsay, Ontario-based company specializing in investment and financial planning software. He was one of several experts who presented research about Monte Carlo-simulation usage at last week’s Financial Planning Association conference in Seattle. Mr. Brayman discussed his research paper, titled “Beyond Monte Carlo, A Replacement for a Misunderstood Practice.” Its main tenet is that despite the efforts of academics over the years to quell usage of Monte Carlo simulations in financial forecasting except as a last resort, the current state of affairs means “advisers, software manufacturers and even some academics continue to promote the technology,” he said. A more sound approach to financial planning, Mr. Brayman explained, should be based on what he calls a “reliability forecast” approach. This method, he said, provides all the analytical results available using Monte Carlo simulation but with a fraction of the calculations, using calculations that are based on more common sense and realistic assumptions that also happen to ultimately provide higher precision. “Don’t get me wrong, [Monte Carlo simulation] is a simple statistical procedure and as such cannot be wrong. It is the way it is applied and the assumptions made when using it that I’ve examined,” Mr. Brayman told conference goers. There are perfectly valid applications for Monte Carlo simulations in specific instances, he added. All this has left some financial advisers and planners scratching their heads. Financial planner David Johnson said he had never grasped the full depth of the simulation’s shortcomings. Mr. Johnson, a certified public accountant for 30 years and a financial planner for the last three years, is the principal of Rainy Day Planning LLC of Renton, Wash., with $10 million in assets under management. “The presentation was really eye-opening. To see what variables are in there and to see what I thought it was doing that in actuality it isn’t is really food for thought,” Mr. Johnson said. Since his clients comprise engineers from The Boeing Co. and Microsoft Corp., it’s always best to pursue anything with as much accuracy as possible, he said. Jason Hall, a financial adviser with Key Private Bank in Tacoma, Wash., which is a branch of Cleveland-based KeyCorp, said, “Over the last few days, I think this is the third presentation on Monte Carlo simulations I’ve attended. I’m going to go home and really pore over the notes and presentations to grasp it all fully and digest it.” “It’s a garbage-in and garbage-out process,” Mr. Hall said. “The gut feeling I’ve taken away from this is that you’ve got to get your initial assumptions right, or it could all be for nothing.” Mr. Brayman was joined by other presenters wary of the industrywide popularity of Monte Carlo simulations, including Richard Ferri, chief executive of Portfolio Solutions LLC, an investment management firm based in Troy, Mich., that manages $1 billion in separately managed accounts, and Somnath Basu, a professor of finance at California Lutheran University in Thousand Oaks. Advisers should keep in the back of their minds the connection between gambling, games of chance and Monte Carlo simulation, he said. “At worst, overly optimistic assumptions in retirement planning can be catastrophic,” Mr. Basu added. He also led financial planners through a series of advanced techniques using available off-the-shelf software. The products included Crystal Ball risk analysis software and OptQuest algorithmic-optimization software. Crystal Ball is a spreadsheet add-in for carrying out Monte Carlo simulations in Microsoft Excel made by Decisioneering Inc. in Denver. OptQuest is from OptTek Systems Inc. of Boulder, Colo., and is meant to run primarily as a value-added module imbedded in simulation software. Mr. Ferri, the author of four books on asset allocation and investing, summarized his presentation by warning advisers not to depend on modern portfolio theory for calculating either reduced risk or increased returns, because models are most useful for learning and educational purposes. “You’ve got to focus on creating portfolios applying fundamentally different asset classes that have real expected returns,” Mr. Ferri said. “Your greatest value as an adviser is in achieving consistent results.” Davis D. Janowski can be reached at djanowski@crain.com.