Efficacy of Forced Choice Testing in Detecting a Secret Cell Hiding Within a Group

Introduction

National security professionals have placed an emphasis on deception detection and have developed tools to help them determine whether individual is being deceptive, or conversely, telling the truth [1]. Because of the limitations of current deception detection methods, we believe innovations in the field are needed."[1]. Over the past 15 years, our research team has examined whether a non-coercive, probability-based method that has been used in medicine for detecting deception - known as Forced-Choice Testing (FCT) could be used effectively within contexts other of interest to national security [2].

Our previous research has demonstrated that FCT methods have been effective in detecting information that individuals are trying to conceal. The classification accuracies for this approach have been between 82-85% [3] which is higher than the demonstrated efficacy of other approaches to detecting deception - such as the polygraph [4] or voice stress analyzer [5]. While promising, previous studies did not assess how well FCT could be used to find a group of collaborating deceptive individuals hiding within a larger group. Such a methodology may provide new investigative techniques for national security professionals tasked with finding “terrorist cells” of individuals who have been identified as presenting a threat to U.S. national security.

The present study was designed to assess the efficacy of the FCT when administered to a group of people, among whom several individuals attempted to hide their membership in "a cell of deceptive individuals." As in our previous studies, we hypothesized that these deceptive individuals would, when asked about a "cell" hiding within the group, deny any knowledge of a cell and withhold information about its members. This avoidance behavior would, hypothetically, skew the distribution of FCT-acquired “voting” data for each of the cell members; this would result in each cell member receiving significantly fewer votes than innocent members of the group. We also hypothesized that this avoidance behavior would become visible when conducting a link analysis between members of the group.

Results

Figure 1 depicts the entire population of 64 participants in the study.

Click to enlarge

As shown in Figure 2, after calculating the “hit” rate the research team analyzed the results of the FCT and determined that 14 of 64 cadets had received ≤ 25 “votes.”

Click to enlarge

As shown in Figure 3, the link analysis conducted by the research team revealed patterns in the voting behaviors between suspected deceptive individuals.

Click to enlarge

As shown in Figure 4, the link analysis indicates the voting patterns of the 14 individuals who got ≤ 25 “votes”, 7 did not “vote” for each other. Those individuals are connected with maroon lines.

Click to enlarge

As shown in Figure 5, those 7 deceptive individuals that did not vote for each other among the 14 suspected individuals are depicted in black.

Click to enlarge

Discussion

The data from this study are consistent with previous observations that individuals attempting to deceive or conceal information will avoid incriminating information of interest when exposed to an FCT in order to give an impression of innocence [3]. The avoidance strategy exhibited by the deceptive members of the “cell” skewed the overall distribution of votes for the group as a whole and permitted identification of this hidden “cell.” Their behavior is consistent with the findings of FCT methods used in clinical settings where the avoidance strategies used by deceptive patients ultimately reveal the malingered nature of their claimed illnesses [7]. The present findings suggest the behavior of humans who are being deceptive behave similarly whether the FCT administration setting is clinical or non- clinical.

The present data also suggests that in settings associated with national security lengthy FCT presentations, which are common in the clinical setting, may not be necessary in order to be effective. The 32-card test used in this study permitted an accurate identification of the deceptive individuals in the “cell.” The avoidance behavior of the deceptive individuals resulted in response profiles that were patterns that were less frequent than one would predict by chance and from binomial calculations. These deviations from chance reduced the number of people of interest from 64 to 14. This suggests the FCT method may be effective in helping reduce the workload of investigators by identifying a smaller group “of interest.” The link analysis conducted on this smaller group of interest revealed the identity of the cell members.

There are several limitations to the present study. First, the utility of this method for “finding a cell” is dependent upon two critical things: the number of people in the cell who know each other cell member’s identity, and the number of people in the larger group. The current method is dependent on cell members knowing the identities of - and avoiding the identities of - other members of the “cell.” The avoidance behavior is what contributes to the skewed “hit” rates noted in the large group as a whole. If cell members did not know the identities of other members - as was the case in IRA “cells” [8]- their selection behavior would not likely result in a skewed “hit” distribution profile. As noted above, our decision to employ a design in which cell members knew each other was based on historical data about AQ cells. This suggests that this approach will be useful with respect to some terrorist organizations but not others.

When using FCT for an organization in which “cell members” do know each other, the efficacy of the method will depend upon the size of the larger group relative to the size of the “cell.” Binomial calculations indicate that the number of individuals who will receive fewer than the predicted “votes” or “hits” will increase as the larger group size increases. This means that secondary analyses (i.e. link analyses) will become necessary. In this study we found that several non-cell members of the group received fewer than the expected number of “hits” according to the binomial prediction. During a debriefing with the cadets about the study several participants made comments that their picking behavior was not only in response to the question on the test (about who might be in a cell) but also was influenced by a cadet’s reputation in the class (for being a prankster or being an “unknown” or something else). However, it is important to note that when dealing with larger populations it may be feasible to utilize methods, or intelligence data, to decrease the “group size” who will be given an FCT. Doing this may mitigate some of the issues with having a disproportionately large “group size” to “cell” ratio.

The information from the debriefing of participants indicated that none of the cadets was aware of the principle of the FCT during testing. The cadets in this study did not know how representative their responses would be for the group of cadets as a whole, they could not discern how their responses would skew the findings derived from the overall group. This is consistent with our previous FCT work in foreign nationals and suggests FCT may be useful operationally and avoid some of the problems that arise with the use of traditional deception detection methods. Methods such as the polygraph require explicit testing and instructions about the nature of the questions which alerts the subject that the information they are providing on the test may lead to the “cell” being detected. Therefore the individual is more likely to engage in defensive behavior designed to alter their physiologic or behavioral responses [9, 10, 11, 12, 13, 14, 15, 16, 17, 18].

The implication of these data underscores the usefulness of link analyses that extend from FCT data. Future studies might examine the utilization of Forced Choice Testing with a multinomial approach in which the relationship between the variables and the “Ground Truth” is not binomial. Future studies may also examine the efficacy of combining other interviewing techniques with FCT. This may assist in detecting not only the identities of individuals but also the type of information they may be attempting to conceal [2].