It’s amazing how little we know after all these years.….Myst
In the 8 March 1999 issue of Simulation Technology Magazine I wrote a short article that started out “One of the nice things about making a job change is that it gives you the opportunity to reflect on what you have been doing and where you are headed”. At that time I was leaving DARPA where I had been the PM for Advanced Simulation Technology and was moving to the U.S. Army Model and Simulation Office (AMSO), initially as the Technical Director and soon thereafter promoted to the ranks of the Senior Executive Service (SES) and made the Director. I now find myself in the position once again of moving on, retiring from civil service after 30 great years, so the opportunity comes once again to sit back and reflect on where the M&S community is and how far it has come in the intervening 5 years since I wrote the 8 March article. I will start with this:
Statement: M&S is not a standalone domain; it is a support tool for others. Success is not measured in M&S domain space; it is measured by how well it supports someone else in doing their job.
Question (and the only important question): Is our User happy? This article does not try to answer that question directly, it is more of a reflection of our status as a maturing technology. However, the underlying issue is always one of value to the User.
Our status is IMHO both a good news and a not so good news story. It is definitely not a bad story (even though some would have you think otherwise).
(1) Good news: M&S as an industry as a whole has matured greatly over the past few years.
(2) Good news: The institutionalization of M&S within and outside of DoD is well underway. Most DoD acquisition programs have a M&S component now, simulation support plans are becoming a necessary program manager’s tool; more undergraduate and graduate programs in simulation are coming on line; we have our first Ph.D.’s in simulation; there more places for simulationists to publish than ever before; we have the new IEEE 1516 standard approved (yea SISO!); new doors opening up every day. If we don’t have ‘irreversible momentum’ we are close.
(3) Good News: Every day I see M&S being used in new and often novel ways. New communities are becoming M&S converts, old communities are finding more depth and breath in their use of M&S.
(4) Not so good news: JSIMS. Lot’s of lessons learned, some good products did result, but the end goal was not reached. It isn’t BTW our only example here, but it is the biggest and most visible. I often use bridge building technology as an analogy. Bridge technology is now very mature; bridges don’t fall down anymore, but that wasn’t always the case. We should think of our ‘failures’ as part of maturing a technology. Some day we will be as mature as the bridge building business. Until then, build, learn, mature.
Note: JSIMS was what it was for a lot of reasons, not all of them about M&S technology. We violated all fundamentals, not just M&S fundamentals!
(5) Not so good news: Cost of developing, maintaining and executing M&S programs/projects isn’t coming down. A recent report said we just can’t afford to continue down the path of creating major exercises (such as MC02) as our primary method of deploying M&S (paraphrased);
(6) Not so good news: Our simulation workforce is still basically On-the-Job (OJT) trained. This isn’t all bad but the narrow experience that comes from OJT often does not provide a workforce that is well grounded in fundamentals. It is harder for them to understand the why vs. the what, which tends to foster a formula of doing the same thing over and over.
I would also add: Good News: SISO presents it’s first Simulation Life Time Achievement Award! That might be a personal bias of course, but thank you again SISO for that great honor.
The really good news is that I have seen us come a long way in the 30 years I’ve been a DoD Simulationist. I’m not a scientific historian, so I do not have any feel for either the pace or the process new technology goes through on its way to maturity. 30 years isn’t that long, however, keep in mind that only 66 years separated Horatio Jackson (the first man to drive across the U.S.) and mankind’s first trip across space (Apollo 11). Using these two dates as very arbitrary metric, that puts M&S a little less than half way to the Moon, so is it no surprise we have a balanced portfolio of successes and failures? The last 30 years has obviously been ‘…an excellent adventure’. (I will leave it to the reader to decide who our equivalents of Bill and Ted are) and in looking back now, the two questions that come to mind are:
(1) Have we been asked to do more than the technology is ready for?
(2) Are we learning and maturing from this balanced portfolio, or are we walking in circles like people lost in the woods often do?
Based on my 30 years as a senior Simulationist (approximately 20 years with the Navy and the last 10 with the Army) here are my thoughts on the above two questions.
Question (1): We did reach too far too fast (and to some extent have paid the price). I often use the creation of DMSO (’91) as the timeframe were the DoD entered the modern age of M&S. DMSO was an artifact however, not the event. We were coming out of the Cold War, the DoD needed to transform significantly. At the same time personal/micro computers and the Internet were coming on strong. We had a vision: Take the potential of M&S, combine it with the modern power of the information technology revolution and become the foundation for leading the DoD through it’s cold war to peace time transformation. It was a grand vision in every sense of the word. Senior leaders bought the vision and we believed in it. Because of this, a great burden was placed on M&S by the DoD, which definitely pushed all the technology, application, skills and resources of the M&S community. We sold, and the DoD bought that M&S will help you refine your requirements, make your design process quicker, cheaper and less risky, will solve all your testing problems, etc. etc. etc. It of course couldn’t. It could help in many ways, but it wasn’t/isn’t THE ANSWER TO ALL THINGS GREAT AND SMALL (with apologies to James Herriot). As a technologist, I don’t think it’s bad to push to a vision. I do believe;however, that it is unwise to let a vision naively drive the train. When you do that, you end up doing yourself more harm than good. The question for the reader, which I have posed before, is: Did we fall into the same trap the Artificial Intelligence community fell into during the 70/80’s? Over promise, under deliver, to the point where no matter how much progress we eventually make, we are tainted for a long, long time to come? And if we have fallen victim to this trap, what responsibility do we now have as we continue to move from the entrepreneurial phase to the institutionalization phase of M&S? BTW, I use the word technology here is a loose way. The question isn’t just about the technology of M&S alone, it’s also about the ‘techniques, tactics and procedures’ we wrap around this technology. A good topic of discussion is to ask, if we have not lived up to User expectations, then what has failed them?
Question (2): I get very frustrated when I hear people make negative statements about M&S and it’s abilities because I know the problem isn’t the technology, it’s the hype and the grand expectations that are at fault. Or is it? How much is it a question of what we sell and then don’t deliver vice senior leaders hoping beyond hope that M&S will make problems they create for themselves go away? Since our destiny is mostly in our hands, I’m prepared to argue it’s more our fault than theirs…We over-sell and under-deliver. We do that because we believe in the technology, but we also do it because in many ways, as a community, we are M&S technology neophytes and we believe our own rhetoric perhaps more than we should. Do we really understand the limitations of our technology, and perhaps the biggest failing, do we understand how to apply it properly? Are we Simulationists, professionals that do professional work, or are we a rag-tag group of people living off a vision that was good for the entrepreneurial phase, but has limits to its usefulness in the institutionalization phase?
From my perspective, here is a perspective. I recently was asked to be the “leap ahead technology” team lead for the effort that made recommendations for what to do post JSIMS. Here are a couple of personal observations I made in my out brief:
•Simulation technology isn’t a ‘big system’ friendly technology
• Has all the challenges of software development, multiplied
•The more complex the simulation the harder it is to develop/deploy
• Risk increases non-linearly
• Flexibility to update/change/add functionality decreases
• VV&A becomes more complex
• Development and life-cycle costs increase non-linearly
• Number of operators/pucksters increases
• Understanding the results becomes harder
•Simulation does not have to be complex to get the job done, actually KISS applies more than we realize (or to quote Albert Einstein: “Everything should be made as simple as possible, but not simpler.”)
•Simulation technology is stuck in a rut
• We still build them like we did 15 years ago
• We still concentrate on the ‘easy’ stuff, that is, applications that are physics based or things that we have done before and understand and ignore the hard applications that we need just as much, if not more. Why? Because it’s easier to polish the bowling ball than it is to invent new games. Let’s not make the next major M&S battleground what should replace HLA, let’s assume between DIS and HLA that bowling ball is shiny enough and move on.
Where should we go from here? Once again, IMHO, I offer the following observations. Each one deserves it’s own article, so please forgive the bumper sticker nature of the thoughts.
(1) Quit trying to duplicate the world in a single simulation. Under the idea of ‘simulation technology isn’t big system friendly’ let’s move away from the idea of a single simulation being all things to all people. This is what JSIMS tried to do and it paid the price.
(2) Build them cheap, easy to use and even easier to understand the results. We will never turn my mother into an analyst, but we can go a long way in giving people useable tools that don’t require a huge infrastructure to use.
(3) Algorithms, the ‘M’ in M&S. We have done a lot to mature our algorithm base over the past few years, but not nearly enough effort is going into this part of the M&S problem. It takes good algorithms to make good simulations.
(4) Don’t forget the support tools. A good simulation isn’t just a duplication of the world, it’s an appropriate abstraction of the world AND a set of tools that allows you to create a scenario based on objectives, tools that allow you to monitor and evaluate those objectives, and tools that allow you to provide feedback on how well you performed against the objectives.
(5) Did I mention KISS? Not to say there isn’t room for a few large, complex simulations, but they should be the exception, not the rule. M&S is a tool that should be in the hands of the person that needs it. You should not have to, in most cases, call someone else that owns large complex simulations and ask them to do something for you. Put the power in the hands of the User. (OK. so I am still a student of the 60’s.) There is a difference, BTW, between building the wrong thing and building the thing wrong, but KISS applies in either case.
(6) Get out of the rut.
Don’t get me wrong here, there isn’t anyone reading this that is more of a supporter of M&S than I am. My comments should be considered technology area self-examination. As a simulationist, your job shouldn’t be just to create simulations, you must also understand your customer, because after all it is the customer that you are here to support. And considering that the customer doesn’t always know what he/she needs from your technology, then you must understand their needs and provide tools and services that meet their needs…not yours! …With that, I will close with this final thought. On occasion when I speak to ‘organized’ M&S groups I present them with the ‘always coveted, often feared’ M&S Snow Globe. I tell them the globe has the characteristic of being pretty; you can shake it up and for a few seconds it will completely grab your attention as you watch the beauty of a light winter snowfall. I then tell them to put it on the table when they meet as a reminder that if they are not careful they will be nothing more than the organizational equivalent of a snow globe. The last snow globe I will give out as the Director of AMSO is in the mail to the SISO EXCOM Chairman. I hope he puts it on the podium during each SIW as a reminder that to rise to the vision, you need to be more than a pretty face.
Dell
P.S. As for the currency of the 8 March 1999 paper….I’ll leave that to the reader to evaluate.
Link to the 8 March SimTech Article: www.sisostds.org/webletter/siso/Iss_5/art_97.htm
Google will find numerous references for the following, each one below picked more or less at random:
Horatio Jackson article: http://americanhistory.si.edu/onthemove/exhibition/exhibition_7_2.html
Bill and Ted’s Excellent Adventure: www.chucksconnection.com/billandted.html
All Things Great and Small: www.mouthshut.com/readreview/43661-1.html