A just released National Research Council report found that Department of Homeland Security methods for determining terrorism risk aren’t adequate for reliable decision making.

According to The Review of the Department of Homeland Security’s Approach to Risk Analysis, the department needs to develop an “understanding of the uncertainties in its terrorism-related risk analyses” and “strengthen its scientific practices, such as documentation, validation and peer review” by external experts. Taking such steps would also increase transparency, the report stated.

The 14-member review committee evaluated the quality of agency risk estimation methods and the ability of the methods to accurately represent risk and support decision making in the areas of terrorism and natural disasters. No problems were found with the department’s analysis in disaster situations.

“However, with the exception of risk analysis for natural disaster preparedness, the committee did not find any DHS risk analysis capabilities and methods that are yet adequate for supporting DHS decision making, because their validity and reliability are untested,” the report states. “Moreover, it is not yet clear that DHS is on a trajectory for development of methods and capability that is sufficient to ensure reliable risk analyses other than for natural disasters.”

Among the recommendations made by the committee are the formation of a well-funded research program to address the social and economic impacts of disaster, alternatives to current fixed-probability modeling, documented and repeatable analyses for infrastructure protection, and discontinuing the practice of integrating risk assessment across the entire range of DHS functions.