Report from Insider Summit (IS9), Monterey CA
Hey, everyone, I’m excited to let you know that there’s a new, improved version of the SOFIT (Sociotechnical and Organizational Factors for Insider Threat) knowledge base of potential insider risk indicators (PRIs). The original SOFIT framework was implemented in the .owl language as an ontology and also released as a PDF listing of PRIs, their definitions or descriptions, relevant research/case study references, and other information. SOFIT was originally developed under funding from the IARPA (Intelligence Advanced Research Projects Activity) SCITE program (2016-2019), and I would like to acknowledge the major contributions of my collaborators in that original work:
The SCITE project team was led by IDI Inc. (prime contractor) with collaborators from George Mason University (team led by Kathryn Laskey), HumRRO (Justin Purl, DE Becker, and Paul Sticha), Dr. Yung Mei Leong, and me (PsyberAnalytix): I led the task devoted to developing the SOFIT ontology. The .owl implementation of the ontology was developed by James Lee, who was a graduate student at GMU working on his doctorate at the time.
SOFIT was groundbreaking for several reasons, but I think its most significant contribution is its focus on behavioral contributing factors in addition to covering the customary technical (cybersecurity-focused workstation/network monitoring) data. I’m aware of numerous organizations that have used SOFIT as a foundation, or starting point, in developing their own PRI frameworks (in addition to other PRI taxonomies that became available around the same time).
One of the main reasons for my talk at the Insider Summit this year was to introduce the updated version of SOFIT, which I created myself (and therefore I bear all responsibility for any flaws that I might have introduced!). For this presentation at the Insider Summit (IS9) event, the organizers suggested that we use an “interview” format instead of the usual PowerPoint presentation, so the intent of this blog is to try and capture the gist of this conversation while also conveying the main points that I covered in the associated briefing material to introduce SOFIT 2.0. The following is not a transcript, but I have attempted to give you a sense of what was discussed. I had the pleasure of being interviewed by Paul Temple, one of the founders of Advanced Onion, Inc., which hosts the annual insider summit events.
Paul: Can you give us some background on why and how you came to develop the original SOFIT framework?
Frank: Although SOFIT was developed starting in 2016, it had origins in research that I conducted spanning the prior ten years. In a 2006 research project that I led for an intelligence community client, I developed a “shredded puzzle” graphic to convey a conceptual model for a threat assessment approach that integrates behavioral data and streaming cyber/network data (referred to as technical data) to yield a comprehensive, whole-person predictive analytics model.
For a description of why I used this particular metaphor – the “shredded puzzle graphic” depicting the assembling of a puzzle out of a combination of a huge number of puzzle-pieces that come from multiple, unknown puzzles (i.e., without benefit of knowing the pictures on the puzzle box tops), please see this Shredded Puzzle blog that I wrote on my PsyberAnalytix website. This hierarchical, pattern-based perspective still guides my thinking and development of advanced analytic models for insider threat assessment.
In particular, this conceptualization was the basis for the SOFIT framework and for my investigation of different modeling approaches aimed at increasing our understanding of the insider risk assessment process—and to provide more effective tools for supporting this difficult job.
This concept was most notable in its emphasis of an integrated framework for modeling insider risk based on behavioral, technical, and organizational factors. I enlisted the aid of my collaborators in the SCITE project—social/behavioral scientists, human factors engineers, a clinical psychologist, data scientists, and cybersecurity practitioners—and the PRI framework was further informed by expert researchers and practitioners in the broader insider threat community through survey studies.
At around the same time, following Executive Order 13587 in 2011 that required federal agencies and departments using classified material to establish insider threat programs, we saw the development of other frameworks or taxonomies (such as those for the DoD, Treasury Dept, etc.). There are strong similarities among these frameworks, but the SOFIT framework has a more elaborate set of behavioral factors and it takes an additional step of mapping the risk indicators to different insider threat types.
Paul: What prompted you to develop an update (SOFIT 2.0)?
Frank: SOFIT was built to facilitate the implementation of the whole-person insider risk mitigation approach, and I think it contributed to the gradual acceptance of this strategy. But the original version, built about 7-8 years ago, included over 300 risk indicators in a rather complex, deep hierarchy. In the last few months, I worked on an update that restructured the Individual Factor branch of the taxonomy to a 3-level hierarchy with a reduced set of risk indicators to just under 200 (the organizational factors branch comprising about 50 indicators was not changed). This diagram shows the new Individual Factors branch of the SOFIT 2.0 framework, which has nine major classes that are further subdivided into 32 sub-classes (the third/bottom level identifies individual PRIs, not shown in this diagram). The complete framework is downable here.
An organization might start with this knowledge base as a baseline, but then refine the set of PRIs based on its own mission and priorities.
Paul: What prompted you to develop an update (SOFIT 2.0)?
Frank: There are three main reasons for restructuring the original SOFIT.
- First, the original PRI list had many PRIs that were very similar to one another – so some pruning was in order. An example of the need for consolidation is the long list of PRIs in the original SOFIT for cyberloafing – there were ten different ways to observe this type of activity, even though they each are treated in much the same way by an insider risk model. They have similar definitions, similar detection means, and similar mappings to threat behaviors. Therefore, it was determined that these could readily be consolidated into a single PRI.
- Second, the concept of PRI decay was not considered in the original knowledge base—and an update was needed. PRI decay refers to the amount of time that a threat analyst will continue to weigh in on a PRI that was observed in the past. The concept of PRI decay is useful to a Whole Person approach to countering insider threats, but it has not been studied much or reported in the professional literature, other than a couple of my publications on the topic. What I have found in expert knowledge elicitation studies is that analysts do seem to assign varying rates of decay to different types of PRIs. My latest research suggests that we need to assign decay rates individually to PRIs because there are no general principles that can be applied: For example, personal predispositions (e.g. psychopathy and narcissism) have little or no decay, but so does the technical precursor, Introduction of Malicious Code: Egregious cyber acts are not soon forgotten, unlike lesser cyber events like Printing to Anomalous Location. Also, while some behavioral precursors (e.g., Associating with Extremist Groups) have low or no decay, others (e.g., Attendance Issues) have a moderate decay. Another result from these studies is that when the expert analysts were given choices of six decay rates ranging from NONE to VERY HIGH, the SMEs were very reluctant to assign high decay rates. Therefore, my current thinking on this topic leads me to use just three levels of decay rate: no decay, slow decay over 3 years, and a somewhat faster, moderate decay rate over one year. Using an exponential decay model, we may say that a PRI with a SLOW rate of decay will dissipate entirely in 3 years, with a half-life of 6 months (meaning that loses half of its strength or weight every six months). By contrast, PRIs with faster decay rates have a half-life of 2 months and disappear entirely in one year.
- Third, my research suggested that there are higher level patterns that should be addressed in more sophisticated models, which also led to a restructuring of the SOFIT framework.
Paul: Can you explain the concept of applying the pattern-based behavioral analytic approach with an example?
Frank: My research suggested that there was a need to address higher level patterns. Most models make the assumption (explicit or implicit) that all PRIs act independently… i.e., they do not interact. This is often assumed for expediency since it makes the math easier. However, in the most interesting cases, there is a pattern in the particular collection of observed PRIs – this helps “tell a story” and provides more context about the incident. The inferred risk of the collection of PRIs is more meaningful than what would be obtained by simply considering the independent contributions of each individual PRI. As shown below, the combination of multiple PRIs that come from certain subclasses in the SOFIT framework provides more insight into the nature of the incident such as motivation and intent. Thus, in other words, the whole is greater than the sum of its parts.
Here’s an illustrative example of how this works. Gabriel Romero was a Navy Machinist Mate Auxiliary Fireman who performed Dry-Dock patrol for Fast Attack Submarine USS Columbia at Pearl Harbor Naval Shipyard. In December 2019, without provocation, he began firing his M-4 rifle at civilians, killing two and wounding a third, before shooting himself with his M-9 pistol. The incident only lasted a few seconds from beginning to end.
Given the compressed timeline, one might conclude that this murder/suicide incident was unavoidable. But further consideration of behavioral and personal history “red flags” suggests otherwise, if only these indicators had been carefully examined. To be sure, Romero was qualified for armed Topside Roving Patrol and therefore had access to weapons. But consider this: He had been passed over for promotion and was formally disciplined for repeated tardiness and dereliction of duty. He complained to a shipmate that he was tired of work; and about people calling him stupid. In anger, he punched a locker and yelled at a shipmate who suggested that Romero should seek counseling to deal with stress. While not all these incidents were reported, there was sufficient concern about his behavior to refer him to a Force Psychologist, who assessed him as suffering from a “Phase of Life Problem” and problems relating to “psychosocial circumstances.” Clearly, Romero exhibited numerous PRIs—and while each individually may not have been significant enough to alert insider threat analysts, together they paint a serious picture of a troubled, at-risk individual. Cogility’s Cogynt model, which implements a hierarchical/pattern-based threat assessment approach informed by the SOFIT 2.0 framework, identifies this workplace violence threat at least one week before the incident occurred.
In a final question, Paul asked Frank to provide some background and results of his work in evaluating performance of several alternative counter-insider threat models. The method and results have already been described in Frank’s NITSIG presentation, which may be viewed here.
There was time for a few questions from the audience, including:
Question/Comment: Possible criticisms of the modeling approach and the methods used for performance evaluation are that (a) basing models on expert judgments is risky because experts are often wrong; and (b) “actuarial” approaches (presumably those based on statistical/empirical data) are flawed for similar reasons, and an alternative strategy is the use of Structured Professional Judgments. What’s your opinion about this?
Frank: Certainly, experts are not perfect and they do make mistakes. Models developed based on expert knowledge, such as Cogynt, do not rely on the judgment of only one expert, but rather are constructed based on input from multiple experts. This yields models and tools that reflect the expertise of many professionals with varied backgrounds and experiences, thus delivering solutions that are most often better than the output that would be expected from any single analyst. Further, and importantly, the contributions of these models should always be interpreted as “triage” – the starting point of the analysis rather than the end point. Cogynt excels at this sort of decision support because it provides complete provenance about the supporting material that led to its conclusions in a very transparent, effective visualization of pattern-based relationships that the analyst can examine, assess, and modify as needed to develop final recommendations. As a final (brief) note, I believe that models such as those I’ve examined (particularly Cogynt) can be augmented, with additional AI enhancements, to provide outputs, visualizations, and explanations of results in ways that promote and even strengthen the application of Structured Professional Judgment methods.
Question: We understand the importance of including behavioral indicators in a more complete/comprehensive “whole-person” C-InT program, but how can we convince management and stakeholders to allow us to acquire and use this type of non-technical data?
Frank: I would refer to a large body of research that now provides strong justification for this approach. This includes several of my papers (such as the one describing the “shredded puzzle” methaphor) and many esteemed publications that have been produced by Carnegie-Mellon’s Software Engineering Institute CERT division and reports published by PERSEREC, to name a few. As I and other thought leaders have argued many times, programs that rely solely on technical indicators will be “doomed” to only react to incidents and clean up the mess, in contrast to the “whole-person” and “wellness” strategies of more proactive, risk management strategies that use all types of data (especially behavioral/psychosocial) to identify and (whenever possible) help at-risk individuals find “offramps” from the critical pathway before they act to harm the organization.
Finally, I described a set of “take-aways” from this discussion:
- PRIs should include not only the most egregious violations, but also concerning events, behaviors, and characteristics that help to identify at-risk individuals and proactive opportunities for positive mitigation providing an “offramp” from critical pathway
- Map the PRIs to all threat behavior types of concern, not just a generalized concept of “insider threat.”
- Behavioral analytic models should reflect high-level patterns of PRIs to provide greater insights than would be derived from merely aggregating the PRI weights independently
- The SOFIT PRI ontology provides a solid framework for characterizing insider threat behavioral, technical, and organizational risk indicators and contributing factors
- Cogility’s Counter-Insider Threat (C-InT) solution leverages its Cogynt Continuous Intelligence Platform to apply AI and real-time streaming analytics - informed by SOFIT 2.0 - to augment insider threat decision making
