Article by: Ernest H. Page (epage@mitre.org )
Introduction
The recent hubbub about “simulation-based acquisition” has evolved in the U.S. Army into an initiative called SMART – Simulation and Modeling for Acquisition, Requirements and Training. SMART, as the acronym implies, extends beyond what we typically think of as acquisition. The use of modeling and simulation (M&S) proliferates the Army, its organizations and missions. Historically, the Army M&S community has been divided into three so-called domains: (1) Advanced Concepts and Requirements (ACR); (2) Research, Development, and Acquisition (RDA); and (3) Training, Exercises and Military Operations (TEMO). This separation tends to reflect the different types of M&S typically employed within the domains. For example, TEMO simulations include manned simulators such as CCTT. RDA models include CAD/CAM models for engineering-level system design and analysis. ACR models, e.g. CASTFOREM, are used to develop future doctrine and tactics.
The differentiation of the Army M&S community into the ACR, RDA and TEMO domains is useful in the sense that it reflects meaningful differences in modeling objectives and techniques within these activities. However, when the Army is viewed at the highest level—the so-called “enterprise level”—it is clear that the domains must collaborate to produce an end-product, be it a piece of equipment, new doctrine and tactics, or both.
While there is certainly a significant degree of collaboration across the Army M&S domains already, the premise of the SMART initiative is to enhance and support this collaboration through modified business practices and targeted technology investments. The U.S. Army Model and Simulation Office (AMSO) has been designated the executive agent for the SMART initiative. Part of that responsibility includes the requirement to define an “architecture” for SMART. In this article, I provide a few philosophical observations regarding this task. The opinions stated below are the author’s and do not necessarily reflect the opinions and policies of AMSO, the U.S. Army, or the U.S. Department of Defense (DoD). And, as always, the author reserves the right to modify these opinions without notice, and accepts the possibility that he might occasionally get it wrong.
The SMART Execution Plan
In Fall 2000, the SMART Execution Plan was drafted and accepted by the Army Model and Simulation Executive Council (AMSEC). Noteworthy is the fact that the plan does not contain language relating to the regimented design, development and acquisition of an Army-wide information technology (IT) infrastructure to support SMART. Generally, the philosophy accepted by the AMSEC is that IT aspects of SMART—which are not the key elements of SMART, the organizational and cultural changes are—should be emergent rather than rigorously prescribed. As such the SMART Execution Plan contains a number of tasks related to the derivation of a SMART architecture: - Develop and promulgate SMART architectural reference models
- Identify target opportunities for technology insertion
- Identify and resolve technical issues associated with moving toward a digital (model-based) Request for Proposals (RFP) system.
- Identify and/or develop standards for data and information interchange
- Formulate strategic directions in simulation research and direct funding to address those challenges relevant to SMART
In the following, I briefly relate a three of the philosophies that undergird the diffuse tasks comprising the architecture portion of the SMART Execution Plan. Further philosophies will be related in subsequent articles and/or as they occur to me.
SMART Architecture Principle #1: Generalization is not free nor is it always the best answer
This is an obvious point, but I’ve been around the DoD M&S community long enough now that I’m starting to feel it is a point worth making and re-making. The understandable push toward cost-effectiveness in the government has produced an environment where decision makers are loathe to “pay for the same thing twice.” “Why am I paying ten different contractors to develop ten separate scenario generation tools?” is a hypothetical, but plausible, example. The implied mathematics of that statement is that:
Developing one system that satisfies each of the ten needs is cheaper than developing ten separate systems.
This seems like a compelling argument on the surface, but I suspect that the issue is far more complicated and goes to the heart of the tension between “general-purpose” and “special-purpose” solutions. Clearly, neither approach (general-purpose nor special purpose) is inherently superior to the other. Further, the problem is very difficult to define—one man’s specialization is another man’s generality. It seems that in almost any domain you see a mixture of general-purpose and special-purpose solutions—programming languages, hand tools, vitamins, medical doctors, and so forth. So, appealing to proof-by-existence, a marketplace for both types of solutions must exist.
Returning to the specific example of the scenario generation tool above, it would still seem that the commonality of function would justify a general-purpose solution here, right? Perhaps; and perhaps not. Certainly, I believe commonality of function is a factor to be considered, but there is also the need to consider the costs associated with defining and constructing a general solution. And while commonalities can be easy to spot, the costs of generalization may be extremely difficult to quantify. How many meetings must take place between the interested parties to specify the general solution? How much time does this coordination add to the delivery schedule? How is maintenance and evolution of the product to be handled? The hidden (and not-so-hidden, but certainly under-appreciated) costs of developing general-purpose solutions make finding the optimal solution elusive. If I can build and maintain ten nearly identical pieces of software for $100 each, and the cost of constructing and maintaining a single general-purpose solution is $100,000, then unless I can reduce the total cost of the general-purpose solution, “paying for the same thing twice” (in this case, ten times) is the better approach.
Within the SMART architecture, we accept the possibility of redundancy across special-purpose solutions and we reject the argument that such redundancy is inherently inefficient.
SMART Architecture Principle #2: The worst thing you can do is spend a lot of money on technology
SMART is about making an enterprise-level activity more efficient and effective: the “better, faster, cheaper” mantra. Technology is simply a means to that end. Information Technology (IT) is not, and should not become, the focus of SMART. We observe, however, that once you invest a significant amount (in dollars or time) on something, there is an implied level of importance, and hence focus, on that item. We further observe that large investments typically equate to longer delivery cycles. Delays in delivery of any technology can very well mean that the technology, once delivered, is obsolete. The government has faced this dilemma for many years in the acquisition of large systems like battleships. But while battleships can probably never be a disposable economy, information technology can. How often do you replace your desktop computer or laptop or palm device? How often do you upgrade your web browser or anti-virus software? With software, these updates can even be automatically, and transparently effected!
Consider a hypothetical simulation system that costs $1 billion dollars to build. How long do you think that system will be used? Probably a long time; one billion dollars is a lot of money. How long before the system is really obsolete? And how long before it is delivered in the first place? Presumably, the system serves an analytical or training need. The need should be the focus. But once a large enough investment in a tool is made, there may be a strong tendency to confuse tool with solution—potentially even redefining the needs to suit the capabilities of the tool…
But what if you held the cost of the system down? Discounting the effects of inflation, for $1 billion you could deliver and throw away a $1 million system each year for 1000 years, or a $10 million system each year for 100 years. For sake of argument let’s say that the hypothetical system takes seven years to develop and has an operational life of 20 years. Using the throw-away economy, and assuming a $10 million per year investment level, you save a little over $700 million over those 27 years plus you have a system to use during the first seven years that you waited for the $1 billion system to be delivered!
But is it possible to shift costs that dramatically? I don’t know, but I believe it is a notion worth careful consideration. Perhaps one way is simply to pick a reasonable spending limit and avoid the temptation to seek general-purpose solutions. As Brooks observes (The Mythical Man Month, Anniversary Edition, p. 259):
Paradoxically, it is much more difficult to design a general-purpose tool than it is to design a special-purpose tool, precisely because one has to assign weights to the differing needs of the diverse users. … The besetting temptation of the architect of a general-purpose tool … is to overload the product with features of marginal utility, at the expense of performance and even ease of use. The appeal of proposed features is evident at the outset; the performance penalty is evident only as system testing proceeds.
Information technology represents an economy in which dollars spent in the future buy more than dollars spent today. Certainly Moore’s Law indicates the increasing processing power of computing systems. And while processing speed tends to double every 18 months, the cost of these systems does not rise that steeply, and in many instances tends to decrease. This is another reason to discipline ourselves in IT expenditures.
Within the SMART architecture, we recognize that organizational and cultural behaviors are the focus of SMART, information technology is merely an enabler for these behaviors.
SMART Architecture Principle #3: There is no right answer
I am reasonably certain that finding optimal solutions to complex problems is generally an exercise in intractability and/or undecidability. In any context, the analysis of complex systems requires an appeal to models of these systems (mental or otherwise). Depending on the factors one considers within a model, and how heavily one weighs each, one man’s optimal solution may be criticized within another man’s model(s). You need only look at the debates over global economic or climatic conditions to see this phenomenon. The question, “are you better off today than you were x years ago”, for example, is completely relative.
Within the SMART architecture, we recognize that achieving optimality is beyond reasonable expectation, and further, that consensus regarding the quality of any given architectural solution is unlikely.
Summary
The SMART initiative is first and foremost about effecting behavioral change, at both the individual and organizational levels, within the Army enterprise—engendering an environment for effective and widespread collaboration within the system life cycle. SMART is not a technology demonstration, nor should SMART become one. On the other hand, the behaviors targeted by SMART will be enabled through the application of science and technology. Broadly speaking, the infrastructure for SMART will consist of a vast array of information technologies—computers, databases, languages, protocols, standards, and so forth. The nature of these technologies, and the means through which they may be effectively introduced into, accepted by, and evolved within the Army enterprise is the subject of the architecture and infrastructure sections of the SMART Execution Plan.
With regard to information technology, we assert the following:
|