Enhancing a Lightweight FIST2FAC System (Web Exclusive)

NRL Photo 1

By Ranjeev Mittu, Dr. Harold Hawkins, Glenn White, Dr. William F. Lawless, and Dr. Gavin Taylor

The Naval Research Laboratory (NRL) has partnered with the Office of Naval Research (ONR) to create a lightweight system that is part of the much larger Fleet Integrated Synthetic Test and Training Facility (FIST2FAC). The lightweight training system at NRL is focused on helping operators develop new tactics, techniques, procedures, and concepts of operation for engaging small-boat threats. NRL’s system is composed of the Joint Semi-Automated Forces simulation environment for representing the behavior of various naval entities within the training environment, a commercial agent-based simulation for developing and representing threat behaviors, and various training systems (shipboard bridge views, helicopter simulators, etc.) with which the user interacts. The FIST2FAC distributed system employs internet communications to allow participants to coordinate their actions and responses. The objective is to provide a realistic environment that allows the various personnel at the training stations to exercise distributed command and control.

New Areas of Research

To enhance the baseline lightweight system, NRL is exploring potential partnerships with industry, academia, and other Navy laboratories. Several topics of interest are being discussed, including: the development and/or integration of advanced command, control, communications, computers, and intelligence (C4I) decision support capabilities; proactive decision support to enable more effective human systems integration; team research within FIST2FAC; and development of machine learning techniques to enable the efficient mining of human performance data.

Advanced C4I Decision Support Capabilities: The FIST2FAC lightweight node at NRL is a scaled-down version of the primary system that focuses only on a narrow training domain: improving operator capabilities to identify and engage potential small-boat threats. The primary training activities within the lightweight system involve operator visual observations to detect and understand adversarial behaviors, coordination of actions, control of distributed assets, and weapons engagement from either a virtual ship bridge or helicopter. To train more complex decision making skills, however, advanced C4I decision support capabilities will need to be integrated into the system. These capabilities are primarily focused on a systems engineering solution to improve the technical infrastructure in order to provide a richer decision-making environment. These might include multi-intelligence sensor fusion systems and unmanned aerial vehicle simulations and systems.

Proactive Decision Support: The Navy is increasingly reliant on advanced and complex decision support systems. Most research indicates operational improvements with the use of such tools, but they also may introduce errors that affect performance. Some users rely on decision support tools more than is appropriate, while other users underutilize tools to the detriment of task performance. To help overcome these issues, the next generation of decision support systems must provide deeper intuitive insights for automated recommendations that are made, while understanding and predicting (with a certain degree of confidence) what information the users need in a given context to make decisions. Systems that exhibit this behavior are known as proactive decision support systems, and focus primarily on improving human systems integration by anticipating user needs. Specific areas of research might include the following:

  • Reasoning about information and command-and-control interaction patterns to infer decision making context
  • Creating formal models of decision making based on information interaction behaviors (potentially using cognitive architectures)
  • Leveraging research from the artificial intelligence community in plan recognition processes to infer which decision context (model) is active and which decision model should be active
  • Recognizing decision shift (which is change in the decision-making task sometimes caused by multitasking) based on work that has been done in the machine learning community with “concept drift,” and assessing how well this approach learns over time
  • Incorporating uncertainty and confidence metrics when fusing information and estimating information value in relation to decision utility
  • Using models of cognition and decision making (and task performance) to drive interface development
  • Addressing how command-and-control and simulation systems should adapt to user behaviors.

Team Research within FIST2FAC: Despite its introduction in game theory 70 years ago, the theory of teams lacks a valid mathematics of interdependence. With the approach of computational teams, the unsettled mathematics of interdependence has become a barrier to the science of teams, precluding effective models and efficient applications for computational hybrid teams composed of arbitrary numbers of humans, machines, and robots. We need computational metrics of performance to inform us how well teams are performing while in action, not afterwards.

Machine Learning to Mine Human Performance Data: One of the key research questions in measuring operator performance is the selection of the right features that are predictive of successful mission outcomes. Because training is limited, however, collecting sufficient data on an operator’s performance can be difficult. This is where we can draw on the work being done in the machine learning community, specifically on multitask learning. In multitask learning, similarities are identified between different tasks such that conclusions compiled from data in one domain can be applied to other domains.

Challenges and Opportunities

Elaborating upon our ideas, specifically with regard to proactive decision support, long-term research should address the following topics: decision models for goal-directed behavior, information extraction and valuation, decision assessment, and human systems integration.

With regard to decision models for goal-directed behavior, the most important research problem might be how to create prescriptive models for decision making, which integrate information recommendation engines that are context-aware. In addition, what are the best techniques for brokering across, generalizing, or aggregating individual decision models to enable application in broader mission contexts? Supporting areas of research might include the development of similarity metrics that enable the selection of the appropriate decision model for a given situation, and intuitive decision model visualizations.

Information extraction and valuation involves locating, assessing, and enabling the integration of high-value information within decision models, particularly in the big data realm. This is a research challenge because of heterogeneous data environments used by decision support systems. In addition, techniques that can effectively stage relevant information along the decision trajectory (while representing, reducing, and/or conveying information uncertainty) would enable a wealth of data to be maximally harvested. The decision trajectory is defined as the sequence of decisions that should be considered in support of a recommended course of action.

With decision assessment, research needs to address what are the most effective techniques for modeling decision “normalcy,” to identify decision trajectories that might be considered outliers and detrimental to achieving successful outcomes in a given mission context. Decision normalcy is defined as the sequence of decisions that optimizes the objective function for a given course of action. Techniques that induce the correct decision trajectory to achieve mission success are also necessary. Metrics for quantifying decision normalcy in a given context can be used to propose alternate sequences of decisions or help induce the best sequence of decisions. This would require prestaging of the appropriate information needed to support the evaluation of decisions, potentially improving the speed and accuracy of decision making.

The key challenges in human systems integration are in understanding, modeling, and integrating the human state (workload, fatigue, experience). Specific topics include: representing human decision-making behavior computationally, accounting for individual differences in ability and preferences, and assessing human state and performance in real-time (during a mission) to facilitate adaptive automation.

We have argued for the need to develop within FIST2FAC a suite of baseline decision support tools. This will permit us to assess the value of new proactive decision support capabilities against the baseline decision support by analyzing the collected human performance data across individuals and teams using novel machine learning techniques. NRL is interested in partnering with Navy laboratories, industry, and academia to integrate new technologies, and to extend the FIST2FAC lightweight system to support additional training domains.

About the authors:

Ranjeev Mittu is a researcher at NRL. Dr. Hawkins is a program officer at ONR. Glenn White is a maritime synthetic training specialist with Commander, Pacific Fleet. Dr. Lawless is a principal investigator at Paine College. Dr. Taylor is a professor at the U.S. Naval Academy.