Modeling and Simulation Slide Share Michael Zyda 1997

Calhoun: The NPS Institutional Archive Faculty and Researcher Publications Faculty and Researcher Publications 1997 Modeling and Simulation: Linking Entertainment & Defense (presentation) Zyda, Michael 28 April 97: "Modeling and Simulation: Linking Entertainment & Defense," at the 1997 Symposium on Interactive 3D Graphics, Providence, RI. http://hdl.handle.net/10945/41510  Modeling & Simulation: Competitiveness through Collaboration Michael Zyda Naval Postgraduate School zyda@siggraph.org Origin of the committee The Director of Defense Research & Engineering (DDR&E), Dr. Anita Jones, funded a study through the National Research Council Computer Science & Telecommunications Board (NRC CSTB). Page 1 Goal of Committee To explore how the Entertainment Industry (EI) and the Department of Defense (DoD) and its associated industries can develop a stronger technology base for modeling & simulation and profit from a closer working relationship. Committee Appointment The NRC CSTB appointed a committee consisting of nine leaders from academia, the computer and the entertainment industries. • That committee represented the virtual environment, computer graphics, networked videogame, film and entertainment communities. Page 2 The two communities are connected ... EI and DoD are connected but we often don’t think about it. Large amounts of government-funded research and infrastructure form the foundation of the EI industry from computing to computer graphics to the Internet... The path from technology to engineering ... We don’t often think about this relationship because the connectedness spans a long period of time, the time required to research and create a technology that can then, later, maybe 20 years later, be easily used... Page 3 Span of Time ... DoD Funding • Computer graphics - Geometry Engine ~1979. • Internet - ARPANET in late 1960’s. • SIMNET ~1984 EI Use • Computer graphics - Nintendo-64 1996. • Internet - Mosaic 1993, Netscape 1994. • Networked Doom 1995. Page 4 Technology has its own uncontrolled flow... Talented individuals from the entertainment communities have always found creative ways to use technology that the originators never considered... “The street finds its own uses for things.” William Gibson, Neuromancer. Which way does technology flow? EI --> DoD or DoD --> EI? This technological flow has been uncoordinated but it has been bi- directional. The DoD has benefited from EI’s constant attempts to push the price/performance ratios for image generation, networking technologies, and content development tools, to name a few areas. Show video? Page 5 Investments without questions or coordination ... In the past, the government has made large investments in fundamental research and infrastructure for its own purposes, some of which has then become the foundations upon which entrepreneurs have created whole new industries. Modeling & Simulation: the Overlap of DoD & EI Many of the future challenges that face the movie industry, the games industry and the DoD are the same. Multiplayer, simulations using real-time, 3D graphics (technology overlap in networking, low-cost graphics hardware, human modeling, autonomous agents, ...). Page 6 DoD - DIS - HLA - 500+ players Phone/Cable Companies - video to home - WAN-capable protocols PC Gaming Industry - Modems - 2 to 16 players - Internet games LBEs - 32 player LANs Networked, Multiplayer Games * Why have this committee? The task of the committee was to try and help the DoD and EI move forward in a more coordinated, and hence more efficient, manner, learning from each other’s successes and taking advantage of apparent commonalities. Page 7 A cultural mindmeld? This does not mean that the EI and DoD cultures have to merge. It is unlikely that the military will give up its love of acronyms, or that game programmers are likely to ever experience breakfast. Benefits for all ... We do not recommend that anyone change how they work, just that we discuss and explore how the work that each has done, and more importantly, will do, can benefit all the communities. Page 8 Shortening the long and winding road ... The DoD R&D effort is well-funded and looks 10-20 years ahead. EI can help shape the directions that DoD pushes in, and can reap benefits of that down the road, with a “very small amount” of effort now... For DoD “very small amount” means following what EI does and perhaps modifying direction when appropriate... Committee Workshop - Session Summary Introduction Electronic Storytelling Strategy & War Games Experiential Computing & VR Networked Simulation Low-Cost Graphics Hardware Finding Common Ground Page 9 Study Findings Outline of summary Research agenda. A discussion on how that research agenda can be carried out in collaboration. Page 10 Research Agenda Computer generated autonomous agents Tools for creating synthetic environments Virtual environment interfaces Standards for interoperability • Each of these areas has sufficient overlap in interest for the DoD and EI that some common work should be carried out. Research Agenda The collaborative research agenda is not fully-detailed due to study time constraints. The actual mechanisms for this collaboration are discussed separately from the research agenda. Page 11 Separate problem domains It is also important to point out that the technology development can certainly be common but the final applications will be most definitely separate. Realistic Intelligent Human Behavior Computer generated autonomous agents are a part of every major DoD simulation system and every videogame produced. The goal behind such agents is to reproduce realistic, intelligent human behavior such that humans interacting with the agents receive a compeling experience from that interaction. Page 12 Definition of terms Interchangeable terms • Human behavioral modeling, computer generated forces, autonomous agents, intelligent forces, automated forces ... The desire for autonomous characters The DoD wants computer generated forces that accurately mimic the behavior of humans and other battlefield entities. EI wants campaign engines and storyline engines that provide the driving force behind autonomous characters in networked games and interactive experiences. Page 13 Adaptable Behaviors Both DoD and EI want these autonomous agents to have adaptable behaviors, behaviors that cannot be “gamed”. Autonomous characters and storyline development Both DoD and EI need mechanisms for rapidly developing autonomous characters and integrating them into a coherent story. Page 14 Autonomous Agents - State of the Art The state-of-the-art with respect to autonomous agents in both DoD and EI is not high from the computer science standpoint. Non-reusable autonomy The EI community “hacks” intelligence into their interactive systems, with up to two-thirds of the development effort being spent on this non-reusable process. Page 15 Semi-autonomous forces While further down the road to autonomy, the DoD is hardly better off. In the modeling and simulation community, the largest part of defense dollars have been spent on semi- automated forces. Non-reusable autonomy Current SAF programs are not easy to add new behaviors to. Nor are they readily adaptable to new domains. Page 16 Programming autonomy The SAF systems are far from what computer science should provide as the interface to programming autonomy. There are few DoD programs much beyond this level. In addition, current SAF systems have no capability to model personalities or human character behaviors. What should be done/funded? Significant research investment has gone into AI technologies for autonomous agents for many years. Page 17 Higher-level autonomy specification Unfortunately, much of that work has gone to computer scientists who were tasked only to develop good autonomy technology. Good work was done but no one funded the development of a high-level interface to that autonomy. R & D - Run and Demo Rather, large systems were built for DoD demos and then no one spent any funds to transition that technology to a higher level of abstraction. Such a higher level of abstraction is necessary if we are to capitalize on our prior investments in autonomy technology. Page 18 Autonomy Re-use With such research done, it may then be possible to reuse prior work and to transition that work to the EI community. Broader-based autonomy development There is also the need to perhaps broaden the research agenda to include some members of the EI community in that effort in order to insure that the developed autonomy is usable for something other than tanks shooting at other tanks. Page 19 Human behavioral modeling Broadening this effort to include capturing and reproducing the entire panoply of human behavior with such systems may prove invaluable across many domains. Tools for Creating Synthetic Environments Low-cost, easy-to-use tools for creating interesting synthetic environments, terrain, buildings, 3D objects, dynamic features were described as key requirements by both the DoD and the EI. Page 20 Higher-Level SE CreationTools Tools that allow the rapid specification of polygonally-defined and textured worlds at a relatively high-level were mentioned repeatedly at the workshop. • The lack of such tools for virtual environments is also mentioned in a previous NRC study [the NRC VR report]. Bad CAD is the state-of-the-art The state-of-the-art today is extremely expensive tools, tools with bad interfaces and long-learning curves. • A typical tool is some $80,000 for a single license. • And the learning curve for such tools is measured in months. Page 21 Primitive CAD Tools And such tools are primitive, they are little more than bad CAD programs rapidly pushed out for the “money-is-no-object” DoD simulation community. • This same tool is also used by some portions of EI for video game development. Automatic synthetic environment creation There is a desire in DoD and EI for tools that work at a higher level than polygons. • There is a desire for tools that automate synthetic environment generation. • The NRC report on VR lists these same desires. Page 22 Better, faster, cheaper synthetic environments ... There is a strong feeling that these tools ought to be cheaper, better and faster than the marketplace is currently providing to both DoD and EI. Virtual Environment Interfaces Interfaces that allow players or participants to enter and interact with virtual environments are of great importance to the DoD and the EI. Page 23 Tracking the virtual soldier The DoD wishes to place a company of soldiers into unobtrusive bodysuits, on locomotion platforms, in order to track those soldiers’ movements and interactions in a networked virtual environment. Bodysuits & locomotion platforms ... The purpose of soldiers in VEs is to allow them to train for particular missions when access to the actual location is either hazardous or just not possible. The EI has many requirements for identical bodysuits and locomotion platforms. Page 24 I want my bodysuit! The EI needs bodysuits to build keyframes for animated characters for film and videogame production. Bodysuits are also needed for the development of full- body immersion, location- based entertainment systems. Where is the funding for the bodysuit? The need for research into lightweight, body tracking technology was fully described in the NRC VR report. Yet, since that report’s publication, little funding has been available for some of the hard problems involved. Page 25 Can you buy a “good” bodysuit? Many in the VR community reported that over and over again they were told by funding agencies to buy a bodysuit for their work on the commercial market. • The fact that commercial offerings did not provide the accuracy required and that research needed to be done to develop one that actually worked well fell on deaf ears. Concerted research required ... We still need a good bodysuit. We still need low-cost locomotion platforms. Some of the work necessary to carry out the full development of these devices will take longer than ninety days. Page 26 Standards for Interoperability Networking capabilities that allow multiple users to interact for large-scale virtual environments is familiar territory for the DoD and new territory for the EI. Large-Scale Virtual Environments (LSVEs) The DoD wishes to carry out theater size battle simulations across wide area networks. The EI wishes to build large-scale virtual environments as games. Page 27 Common protocols for scalability Both DoD and EI need common protocols for networking their scalable virtual environments. • Currently DoD goes its own way on this and EI slaps something together on a game-by-game basis. • It is clear that some common research into the technology required for this would be beneficial. Interoperability - What needs to be done? A careful, considered, joint research program needs to be put together that actually studies the issues involved (as opposed to slapping code together for rapid demo) in designing a common, scalable network software architecture capable of supporting large numbers of players across wide area networks. Page 28 Interoperability - Components Components of this research effort: • Architectures for “plug-and-play” interoperability. • Network software architectures for maintaining persistent universes. • The efficient use of multicasting via programmable area of interest managers. • Web-based interoperability similar to that proposed as the virtual reality transfer protocol. Interoperability - What is being done? No current DoD basic research program is looking into these issues as a contribution with wide ranging effect. There are programs that are hurriedly moving forward to meet demonstration deadlines. Page 29 Carrying Out the Collaboration The main human resource issue is that there is an apparent shortage of talented, high-quality, experienced people to develop virtual environments, modeling and simulation software, digital animation, design, and scripting of virtual worlds. The shortage of content developers ... This is a shortage that is seen by both the DOD and EI. Programmers with content development experience, programmers familiar with the technical problems of multiplayer/ multiprocessor games and simulations are just not to be found. Page 30 Cross-disciplinary skill-sets And the people sought are not just engineers and computer scientists. They are programmers and content developers with cross-disciplinary skills. • Such skills enhance the quality of virtual world development and the implementation of such cutting- edge technologies. Visually & cognitively literate These individuals are visually literate, in that they are facile with generating economical, high-quality displays. These individuals have a good cognitive understanding of the human so that the worlds they develop cause a change in the player’s mind, training the player to do the right thing in the simulated world. Page 31 Single-discipline mindsets & incomplete graduates ... Our universities are not turning out such cross-disciplinary individuals. Our universities are instead frozen in single-discipline mindsets, graduating students with only part of the skillset necessary. Interdisciplinary Infrastructures Interdisciplinary university infrastructures, with degrees we have never seen before, need to be constructed to solve this human shortfall. • We need people graduating with BS, MS and PhD degrees in subjects like modeling, virtual environments and simulation, electronic storytelling, ... Page 32 DoD & EI: Information Sharing & Technology Transfer The main information sharing and technology transfer issue between DoD & EI is that there basically isn’t any except what filters over after a large number of years. DoD & EI: Information Sharing & Technology Transfer From the presentations at the workshop, it was clear that things could have been developed a whole lot faster, industries could have been larger earlier if only open collaboration and cross-fertilization had been encouraged and facilitated between DoD and EI. Page 33 Establishing a clearinghouse BUT DoD and EI are two different cultures, with different languages, and separate circles of friends. The only way cross-fertilization is going to occur faster is if we establish some sort of clearinghouse for potentially common modeling, virtual environment and simulation information. Formal Information Exchange We know what informal mechanisms do for this today - not much. We need a more formal place for information exchange. We need a place free from DoD bureaucracy, a place not intimidating to either DoD or EI. Page 34 Create an M&S center? The purpose of this center would be as a clearinghouse and continuing education source for the EI and DoD modeling, virtual environment and simulation community. • If this center can be co-located with a relevant educational program, we have an enormous win. Carrying out the research agenda ... The research base of the United States has at its bottom government funding. In order to carry out the research charter between the EI and DoD, that funding may need to be delivered differently than such funding is now. Page 35 Setting research directions In the workshop, it was pointed out that research funding today is quite different than 20 years ago. Government funded projects are now more product-oriented than they used to be. • In the past, a general direction for research was set and no particular products were specified or delivered. Less-directed funding It is going to be very difficult to explore EI and DoD collaboration without a return to less-directed funding of smart researchers as was done in the past. Page 36 Universities should mostly produce students... Originally the product of university research was smart graduates, graduates who had gotten to cut their teeth on interesting, leading edge problems. Today, concommitent with the shortage of research dollars, we have a shortage of talented people. Cut backs by industry The lack of research dollars at the university research base is also due to cut backs by industry. The computer technology industry does not donate equipment and funding to universities as in the early days of IBM and HP. Page 37 Lack of donations ... Such lack of computer hardware, software technology, and personnel affects the availability and quality of research and affects the graduates of our universities. EI needs to pick up some of this ... A return to the days of yesteryear ... If we can return to the days of when we regarded university-trained people as the product of our funded programs, then our country will benefit tremendously. Funding which is less product-oriented and more open-ended will also allow a greater amount of creative and innovative research. Page 38 Revolution in the university ... Our technology revolution is producing whole new careers and vocations which cannot be ignored. Some of these technologies are taking us into unknown territory where crossing disciplines and university revolutions become necessary. Web site Michael Zyda http://www.cs.nps.navy.mil/people/faculty/zyda Page 39