M&S Knowledge Repository

An Interview with Philip J. Kiviat

On 19 December 1997, Ernie Page and John Tufarolo interrupted the holiday luncheon at Sterling Software in Vienna, Virgina. They sat down with Mr. Phil Kiviat and discussed the topic of simulation. As expected, they found Phil to be engaging, insightful, and occasionally, controversial.

A record of the conversation follows. Click Text to view a text transcript of the question and response. Click Video to view a video transcript of the response.

Question 1: What brought you to the field of simulation?

That in fact is an interesting question, because it was a total accident. In 1959, I think it was, I was a student at Cornell and I couldn't find a summer job because there was something of a depression and there weren't any engineering jobs around. So I got a job that summer working for Professor Bob Bechhofer in the IE department doing, essentially, Monte Carlo sampling simulation for some of his research projects. So I spent a summer learning what random numbers were and all about Monte Carlo sampling, and almost by default I then became the simulation expert. That is hardly the same kind of simulation but at that point in time nobody knew the difference, at least most people didn't know the difference. And then it happened that when U.S. Steel came by looking to hire an employee to do some simulation model building for them, I was put forth as the simulation student. So I took a job with U.S. Steel Research in Monroeville, Pennsylvania, building a simulation model of a steel making facility. And from then on I just learned what I had to learn by myself. But it all started because of a job shortage, and if there hadn't been a shortage of jobs I might very well be designing steam turbines someplace, and I never would have gotten into the simulation business. It was all an accident.

Question 2: How did you become involved with SIMSCRIPT?

When it became necessary to build this simulation model of these open-hearth steel making factories, I was told I had to work with a group of industrial engineers ñ none of whom knew a whole lot about programming or modeling. And I had to work with the computers that they had available which were some IBM 7040/7044 data processing machines. So a couple of things became clear: one was the job had to be done in FORTRAN, and the other was that I had to find some way to communicate with the industrial engineers as to what modeling meant and how to translate a model into a computer program. So I started doing research and thinking about what you had to do. And that is how I first got in touch with Tocher and his work in England, and found out what IBM was doing -- what Jeff Gordon and his group were doing with GPSS -- and heard something about what Markowitz was doing with SIMSCRIPT out at RAND. I never came into contact with him directly, but I talked to somebody who talked to somebody. And finally put together my idea of how to do discrete event modeling which was the work that culminated in the GASP (General Activity Simulation Program) FORTRAN package. I worked on that; presented a couple of papers. I had a friend from Cornell who was at RAND and eventually they approached me, would I like to go to RAND and work with Harry Markowitz on doing SIMSCRIPT II? I said yes. And that's how I wound up working with SIMSCRIPT.

Question 3: What was the climate among SPL designers in the 1960's and 1970's? Were there rivalries, for example, between Norwegians (Simula), RAND (SIMSCRIPT) and IBM (GPSS)? Or was the climate congenial?

It was a very active time. Jeff Gordon and IBM were working full-blown on GPSS and developing new versions of it. The Brits were working on their activity-oriented languages, and they had several going there: CSL and other derivatives of that, as well as Tocher's GSP. We were working on SIMSCRIPT. There were other SIMSCRIPT-like languages that were developed in the states, one of which, MILITRAN, I think was a TRW product I think at that time. Then of course [there was] the Simula people. And from about the mid 1960s to the early 1970s, all the people that were doing simulation language research and development, we all used to meet several times a year at conferences and symposia, either at universities or major public meetings. We used to talk about these things all the time. It was a very friendly, collegial kind of thing. They were all very bright, very exciting people. They were also very intellectually sure of themselves. And while you had four people in a room, each was sure that their world view was the right world view, and, "Yes, what you've done is wonderful and I respect you for it, and your very ingenious in doing it, but no thank you. I prefer to do it my way." And so we just got along fine. It was a very exciting time to work. I met people that I had ... probably the best group of people that I have ever met, intellectually, in my life as a group working together. You had Tony Hoare, who was involved in simulation almost before anything else. Dave Parnas was a participant. Nygaard and Dahl, from Norway, who were the Simula people. Of course, K.D. Tocher from England. John Buxton, who after his simulation work became a very, very important person in the Ada world. Chris Strachey from England. A bunch of other people. It was very exciting and a very good time to live, and I suspect that a time like that doesn't exist very often and I was very fortunate to be a part of it all.

Question 4: Of the many simulation professionals you have known and worked with, and you just mentioned several notables, who most impressed you? Was there anyone you would like to have met (worked with) but never got the chance?

Well of people that I worked with, and worked closely with, absolutely nobody could compare with Harry Markowitz. He had a kind of clarity of vision and a creativity without parallel. I truly enjoyed working with him, also with George Fishman, because while I was working on language development, it was very obvious that interpreting the outcomes that you got from simulation models was as important as the models themselves -- it has to do with that old statement about lying with statistics. And so George and I worked very closely at the RAND Corporation and I think George did a very important amount of work in the early days on spectral analysis and how one made sense of simulation results. I learned a lot from George and truly appreciated what he brought to that part of the job. People I would have liked to have spent more time with: John Buxton, I think; Tocher in England, who was a brilliant man; and Dave Parnas who I had a great deal of respect for. I say had because he left the simulation world and went into the more general software engineering things. But when you count people like Nygaard and Dahl and Tony Hoare in that group, I'd say working with any of those people would have been a pleasure and would have been something I personally would have benefited from. But if you put them all in a bottle and you had to pull one straw out, I'd say Harry Markowitz.

Question 5: Turning to a somewhat more technical question, do you think that historically a tendency has existed for the programming language research community to ignore developments in simulation programming languages? Was this true in the 1960's and 1970's? Do you think it is true for other application domains, such as artificial intelligence?

I don't know enough about other application domains to make any kind of definitive statement. I suspect it's true. I know it's true in the simulation language domain because it's very clear that the Simula people were way ahead of the power curve in object-oriented thinking. I believe they are the first ones that discuss inheritance, classes and encapsulation -- all those things that became important in defining what objects were. And for a long time you just didn't see anything that they were doing mentioned in the general software/computer science literature. The whole idea of a process was neglected. Some of the ideas that we put in SIMSCRIPT II having to do with programs that responded well when errors were made, such as data formats that were too small to report a number that was attempted to be output. We went to every end we could to print out something that was meaningful so that one wouldn't just have a program that aborted. [We had] certain ways of dealing with input and output -- free-form input and output -- and the easy production of reports. I/O was never something that was paid a lot of attention to. And of course, look what happened with Ada -- it was not recognized as being important there either. And I think the simulation language community was overlooked in that respect also. Like I say, that's probably true of other domains, but I'm not familiar enough with them to make a statement.

Question 6: Why do you think that was? Was simulation viewed as an operations research technique maybe moreso than a computer science technique, and that computer scientists were doing the programming language development and they didn't respect the simulation programming language development?

I don't know about the word respect -- that may be too strong a word. And there were some people, like Tony Hoare and David Parnas, who kind of moved into the computer science community. But I think it was looked at as application area and not part of computer science. It was something specialized, and therefore probably not essential. And with the amount of work that was being produced, this just wasn't something that people looked at first, and many people never looked at it at all. And the people that were in the simulation language business, except for the ones like Tony Hoare, had their futures in simulation not in computer science, so they weren't out proselytizing. Harry Markowitz, for example, could have cared less about whether anybody picked his ideas up in the general world. He did what he did because he felt it was right and good for what he was trying to do. So a lot of it just was that.

Question 7: We are all familiar with the fact that Simula 67 was the first object-oriented programming language. SIMSCRIPT II gives some flavor of the object oriented description, e.g. entities described in terms of attributes, entity categorization (classification) as "permanent" or "temporary," the strong reliance on the set and set relations ("member," "owner," etc.) Do you think the correspondence with the object- oriented paradigm was accidental or intentional?

Oh, accidental, in the sense that there was no communication between the two groups for a long time and certainly not until the two languages had been essentially defined. Things wound up the way they did, I think, because of what you have to do to put together a language. On the one hand you have to define what the real world looks like. Harry, being an economist, looked at it one way. Nygaard and Dahl, I don't know what their backgrounds are, but they probably were mathematicians, or more computer scientists than Harry ever was, because he was not a computer scientist. And so you look at the world and the world consists of things which Harry called entities and things have qualities that he called attributes. And they have relationships, economists are very big on relationships, that he called sets. And they were dynamic things, temporary things, and permanent things. And I think a lot of it came from Harry's background as an economist and the purpose that he felt the simulation language had. Nygaard and Dahl looked as the world, saw the same world – there's only one world – but they looked at it in a much more formal, mathematical way in terms of objects and defined things differently. Then there's the dynamics of the world. There's nothing right or wrong about an activity scanning view, a process view, a transaction flow view or an event view. It's just how you choose to paint that picture. I can't tell you whether Harry's idea of an event came from his background as an economist. I came up with the same idea and I'm an engineer. And I can't explain why the Simula people picked up the idea of a process. But it probably has something to do with our cultural background. I'll tell you a little story about that. When I went over to England for the first time in the 1960s to talk about SIMSCRIPT, I gave a series of lectures: one at Manchester, one at Edinburgh, one at Cambridge. I was accosted for the first time with a question I had never been asked before. After I finished the SIMSCRIPT II presentation, somebody asked me what the mathematical formalism was behind it all. And I said, what? There was none. It was what it was because we thought of actions that had to take place and a formalism for describing the world in terms of entities, attributes, and sets and each statement pretty much stood on its own. The Europeans were looking for the mathematical model behind it all, the rigor behind it all. And that's the way they looked at the world and we didn't look at the world that way. So, when we said the world consisted of entities, attributes, and sets, we didn't construct set manipulations such as you normally find in algebra. It was called a set – It could have been called a collection. When I did GASP, I didn't call it a set, I called it a file. But you did the same things. So a lot of it has to do, I think, with the European's interest in formalism and rigor and our interests, maybe the practicality of the Americans: let's get something that works, and if there are some pieces that don't hang together properly, we'll work it out in the second version. So it was accidental in a sense because they were developed independently that they arrived at the same kind of basic collections of things but then again we both represent the same world so it had to wind up in the same place.

Question 8: There certainly seems to be a dichotomy in the M&S techniques and approaches adopted in the U.S. and Eurpoe. Simula, for example, never really caught on as an SPL in the U.S., and the activity scanning world view is clearly under-represented in the U.S.-based simulation literature. Are the causes for this split also cultural?

When you posed this question to me, I tried to remember my thinking about this when this came up 30-odd years ago. I don't recall the answer if I had an answer. I do suspect if you could trace it down that it was a cultural difference. If you wanted to get very philosophical you might say it had to do with one's view of resources. We were much more resource rich than the British at that time. When one is resource poor, one is very concerned about utilizing resources properly and the activity point of view is wrapped around that. We didn't concentrate on resource utilization as much but on defining the dynamics of the process. It could come from that. I don't have a good explanation for it. I do know that once all the cards were on the table, as I said before, each group tended to say, "Thank you. I understand what you did, but I'm going to stay with what I have." Now, the economics of it was, in the United States, IBM put its full support behind GPSS and there were by far more GPSS users [here] than anywhere else. And at that time IBM was completely bundled and people got all the support in the world so most people just stayed there. Also, anybody who wanted to use a European language, a UK language, had no support in this country. So those people who had a choice between, say, CSL and SIMSCRIPT, would have a choice between staying in the United States for support and going to Great Britain. And so there was certainly an economic justification there. Simula didn't become popular, because it was Algol based and it was too complex for many people. The world at that time was a COBOL/FORTRAN world, and I know a lot people that I dealt with at that time just said it was just too complicated. "I don't want to learn about these strange things like encapsulation."

Semantically too complicated, not syntactically.

It was semantically too complicated. People were interested in building models, not in becoming programmers. And they viewed Simula as requiring too much that way. I don't think that's true, but that's the way people viewed it. Now IBM threw it's weight behind Simula in the sense that they put a team together that developed SIM/PL, which was a Simula-type language written in PL/1. Neither language survived, neither SIM/PL nor PL/1. But even with IBM's power behind it, it did not succeed in gaining ground over SIMSCRIPT. And I think again that says something about the difficulty of the programming approach. People found I guess more comfort, more directness, with the SIMSCRIPT idea.

Question 9: You wonder if the same kind of thing would happen today with a Microsoft being the "800-pound gorilla" overcoming what some people believe are better models of computing and computing support. SIMSCRIPT would be a success story, for example, for Apple to latch onto and say, you don't have to be overcome just by the weight of your opposition if you have a product that's better suited in certain arenas than perhaps theirs is.

People who are problem oriented, and I think simulation people are problem oriented – they're solution oriented – [to them] the question is "I have to build a simulation model of some thing, now what's going to be the most direct path to take to that?" When they look at the choice of tools they map that pretty much right away into what they have to do. If you're selecting a general programming language or general set of tools you probably don't have that same mapping. So I'm sure that the influence of the problem solving orientation has something to do with it. Now, Microsoft doesn't do a particularly good job in supporting its customers. Anybody who has tried to dial into a help line or use windows, on their own, knows that. So it could very well be that given two different approaches, one supported by a big company like Microsoft and one by a small, but available, firm, people would choose the support rather than the comfort of the 800-pound gorilla. I don't know.

Question 10: It appears that discrete event simulation tools and techniques available today are graphically enhanced versions of tools that have been available for years. Aside from a better GUI for users, are there simulation techniques that are not represented in commercial tools? At the 1996 Winter Simulation Conference, the keynote speaker from Intel commented on the inability of commercial tools to support really complex problems in the design and business decisions for a new chip factory. How can tool makers take another step toward more powerful tools?

Well, first of all, I agree with the initial statement that most of what seems to have been done is in the GUI area -- where things are dramatically more user friendly and easier to use. The second part of the question, I don't know. And the third part, without knowing what the keynote speaker was referring to, it's kind of hard to say why he said what he did. If he said the languages don't allow you model these complex environments, the question is why? What's deficient? If you look at what the elements you need to be present in tools are: (1) you have to model the physical world, (2) you have to model the logical world, the relationships between things, (3) you have to model the dynamics of the world, (4) you have the model the decision processes that go on in the world, and (5) you have to model the control system and the reporting system that allows you to make it all work and figure out what's going on. If there are deficiencies in any of those areas, well tell us what they are and I'm sure they can be addressed. If there is something else, I don't know what it is. Now, on the other hand, he may have just said that life is too complicated, because when you get these infinitely complicated processes it becomes infinitely complicated to model them. Well, so what else is new? What we have found, certainly with object-oriented languages, you handle that by creating classes of things and changing your levels of abstraction. And it may be nothing more than a modeling problem rather than an inherent problem. I just don't know what the problem [the keynote speaker refers to] was. It may be a problem of scaleability and time rather than basic weakness.

There's very little, I guess, that tool manufacturers can do to solve the essential complexities of modeling complex systems.

Exactly.

Question 11: Given the potential for abuse of simulation results (like the case of the Federal Budget Summit you refer to in you 1990 Winter Simulation Conference keynote address), do you think that the field of ethics has been given an appropriate treatment in the modeling and simulation curricula. There certainly seems to be little, if any, treatment of the subject in most popular simulation texts.

In a way, this has to do with much more than simulation, it has to do with any analytical technique. And I don't know what the present curricula cover, so in one sense I can't say that they do or they don't cover it. But I suspect that they don't. If you look at a paper I wrote many, many years ago on modeling and management [Kiviat, P.J. "Model Centered Management," In: Proceedings of the Seventh Annual Virginia Computer Users Conference, Blacksburg, VA, September, 1977.], I think that the most important reason to use simulation is to study the consequences of various decisions -- such decisions as how do you employ resources, what number of resources do you employ, what decision rules do you use. And to use the models not to generate numbers, but to understand behavior, because we all know that any set of numbers that we put in is not terribly likely to be the set of numbers that result in the real world. So managing the consequences has to do with understanding the behavior and assessing the risk. And I think there is probably not enough time spent on those two elements: how do you use a simulation to assess risk, and how do you present what you've done to somebody, a manager, to have him understand that he shouldn't just look at the numbers he asked you for, but at the consequences of those numbers being right or wrong, near or far, from what is likely to happen. So in that sense I think it is probably under-represented in the curricula. Now what that has to do with ethics is something else again. When you think of ethics you think more of should I be studying something or not. Is it right for me to be even telling somebody how to do something, if I feel there is an ethical problem with doing that. That is really beyond simulation. But I think what I said still becomes very valid. I think it's still very important to teach people how to use models, and to be able to advise the people who ask you for the answers about how to use the models appropriately. I don't think that senior executives necessarily understand the value in a model. And so I think it may be an ethical problem of not discussing this with them, but it's not a problem in ethics in the other sense.

Question 12: Relating to this, currently there is a lot of pressure, technologically, to make simulation available to non-analysts so that simulation is going to be a desktop commodity in the 5,10,15 year timeframe. You throw icons together, you have a simulation. It runs. You make a decision. These people were not trained as analysts to use this tool -- this may be a trivial question -- do you think this is good or bad? How would you advise the simulation research community, or the simulation community in general, to fight the onslaught of the technological advance to build up the training necessary to allow people to use these things wisely?

Well it all goes back to that sign I saw above the door to the MIT simulation lab back in the early 1960s which was SINSFIT – Simulation is No Substitute For Intelligent Thinking. Every model is nothing more than an abstraction. And unless you understand that abstraction, and what has been left out and what has been included you don't understand the quality of the result. It has to do again with the consequences of risk. So to use a model without understanding it and without understanding the space in which it operates, you're picking up great risk in doing something entirely inappropriate or beyond the bounds of the model. So I think the worst thing that could happen is to make simulation be so simple that people could use it without understanding what it is they were really doing.

Question 13: What do you consider to be the three most significant ideas, concepts or contributions in discrete event simulation since 1965? Can you identify any wrong turns or wasteful diversions?

Well I think you may be surprised at what I'm going to say here. Again, I've been out of touch with things for some time, but in my view the only significant idea since 1965 is the incorporation of animation. Because again, in my view, the world is a communication problem. If you can communicate, you can understand. If you can understand and communicate, you can influence. And the whole essence of simulation languages is really creating a communication mechanism between the modeler and the programmer, and between the modeler and the world. Because you understand things in terms in which you describe them. Without language, it is said, you can't have thought. So without a good simulation language you're limited in your ability to describe things and understand your descriptions. Looking at tables of numbers as output is difficult, even for the best people. To look at an animation, which lets some people get a visceral understanding of what's going on, gets you way ahead of the power curve in communicating with people who are using these models. So I feel that was a very significant feature. Not to say that it can't be abused., but I think being able to communicate and show people the dynamics of a model and how its behavior changes, in a way that they can understand, is a very significant occurrence. I think that the basic concepts in all of the languages we've been talking about were in place by 1965. Some of the languages didn't come out until maybe 1969 or so, but people were talking about them as early as 1965. And I personally am not familiar with any other things. Now I may be off base, but that's my personal opinion. As for wasteful diversions, things like that, I would call the interest in putting together simulation packages in Ada, or in Pascal, or in C++ a wasteful diversion, because, in line with what I just said, the whole name of the game is communication.

That seems to be counter to something you wrote in one of your RAND Memoranda which was that [simulation packages in general purpose languages] was a perfectly acceptable thing to do. So you've since revisited that opinion?

Well, first of all I'm thirty-five years older.

I understand.

And second of all, I think the ability ... Well let me go back on that, I think I'm thirty-five years older, and I think I have every right to be able to change my mind.

Oh, absolutely.

I mean it is quite possible to do these things. I think it is worth the time you spend to make a language more economical and more directly tuned to the domain in which it's going to be used because the most important thing is the communication between the modeler and the real world. And we're getting away, more and more and more, from programming. We're more user oriented. Take a look at what's going on in the general community. Programmers are very important, but as a percentage of the workforce they are probably dwindling – more and more end-user computing, and end-users are the ones who will use the better languages. And to eliminate the difficulty they have of learning language-oriented things as opposed to modeling-oriented things is what you want to do. So I think while it's possible to do these things and if you're running some exotic, strange computer, as you were in the old days, and there wasn't a simulation programming language, well then build a package set. And that's what I did with GASP. We ended up running on, at one point, 20 or 30 different computers. But nowadays the number of machines is limited and that's not a consideration anymore.

Question 14: Departing from simulation briefly, what are the motivations and inspirations for the graphical technique that bears your name?

Well back around 1972 I think it was, 1972 or 1973, a number of professional organizations, the National Bureau of Standards, and some other Government organizations got together for a two-day meeting in San Diego to try and come up with a single figure of merit for measuring the performance of a computer. At that time we had five big, major mainframe manufacturers and the Government was trying to evaluate proposals. They wanted to be able to say that the IBM was a 6.3, and at that time the UNIVAC was a 6.4, and the CDC machine was a 9.3. Two days were spent trying to do this, with no result whatsoever, because everybody explained that their machine was different. And so you had multiple measures of performance and nobody knew any way to put them together. So I went away from that meeting thinking there has to be an answer here. And the answer appeared with the understanding that this is a multivariate problem and that the answer was how do you portray multivariate data in a way that would allow you discern a good situation from a bad situation, or a good situation from a better situation. And I, coincidentally, at the same time read an article about some people who were using this circular graphing technique to look at the physical state of a patient who had just undergone a heart attack, and how that patient's state changed over time as they were treated, the idea being that if you could take a picture of the person's state you could know whether they were in a severe heart attack problem or midway through recovery or they were perfectly fine. The idea was that this was kind of a wellness graph. And by playing around with ideas I came up with the idea of the Kiviat graph. That is you could take measurements of both hardware and software states on a computer just like you could take measurements of a person's bodily functions, and if you took the right measurements, and if you arranged them in the right way – and this took some playing around – you got a pattern. So a computer that was I/O bound had a certain pattern. A computer that was CPU bound had a certain pattern. The question then arose, well how can you validate a simulation model? People were doing the same thing there. They were coming up with lists of different variables and measuring them in the real world and then measuring them as they came out of the simulation model. They wanted to see if the model represented the real world. They were comparing numbers. But you can compare numbers forever. Does a 2% departure here mean it's good or bad? Is 2% here better or worse than 3% there? But if you took the pattern of measurement in the real world and compared it to the pattern of measurement in the simulation model, it was almost as though close enough was good enough. You had to see if the patterns were similar. If the patterns were similar that meant the model was behaving like the real world was behaving, and when you changed variables you could see if the behavior patterns changed. That was basically it. It was nothing more profound than that. It gets back to my basic thinking that a picture is where it all is. It's a communication problem. Man does not deal very well with multivariate numerical problems and if you figure out how to put it in a pattern then people can discern differences much more easily than they can with looking at columns of numbers. And it all went from there.

Question 15: Finally, what led you away from simulation? What are you doing today?

Well there were a couple of things. Having done GASP and having done SIMSCRIPT II, I'd kind of run through that interest. I wasn't interested in doing that all over again. I got more interested in higher level things like the uses of models. When I was with FEDSIM we funded Don Kosy at the RAND Corporation to build a language, ECSS, for modeling computer systems by making a subset of SIMSCRIPT II. So I was always interested in higher level things that would enable you to solve problems more easily and that took me away from, rather than closer to, the basic simulation language problem. And then over time I got into management and selling ...

More profitable pursuits.

One thing led to another.

Question 16: Thank you very much. Is there anything that I didn't have here that you wanted to say?

No I think you've captured almost everything. The one thing that is terribly important is that people not lose sight of the objective and treat the model as a thing in itself. It's only as good as the use for which it's intended and it needn't be any better than that. And that gets right down to the essence of modeling.

So you think it's wrong – not to put words in your mouth, so I'll let you comment – it's wrong for a manager to say, "Build me a model of x."The manager should say, "I need to understand how this system works." And if a model is involved in that, a model is involved in that, but it shouldn't be, "Build me a model."

Well, in my way of thinking, "Build me a model of x" is a meaningless statement. I think you understand what I mean by that.

Yes.

To say "I need an understanding of such and so" tells you exactly what you need to know.

But it's interesting that the Defense Department says, "We're going to build a model of x" all the time. It's an acquisition program now, to construct a model. That always seemed to me, standing on my soapbox, to be a misinterpretation of the use of models, or a misapplication.

It's a confusion of tool with solution. To build a model, that's a concrete thing to somebody. Build me a model. And they can buy that. And they probably even know how to buy that. But to say, "Help me understand the following," is a much more difficult question.

It's harder to right a contract to do that.

It's harder to write a contract for.

And maybe that's the only way to acquire large software systems, is to say, "Build me a model."

But, again, it's confusing the ends for the means. And it may turn out that an analytical model would be fine. You don't need a simulation model. In fact I had a practice when people would give me these directions, they'd say, "Build me a simulation model of such and so." I'd say, "What is it that you want to know?" They'd tell me something and so I'd say, "OK, I'll do that." And I'd go away and I'd come back and I gave them a graph and say, "If we change this then this is what will happen." But I made up the graph. And then they'd tell me, "Oh no. That's not really what I meant. What I really wanted to know was the following." And after we went through several of these totally phony starts and stops I found out what they really wanted and then I built the model. Because anybody who says "Build me a model" doesn't really know what they want. Now from a Government contractor's point of view – from anybody's contractor's point of view – that can be a great thing, you just build away, build away, build away. You can make a lot of money doing that. And that gets to ethics right? Ethically it's wrong. Good business ... But I think the real important question is what is it you need to know, and simulation may be the right way to get there.