Learn How KDM Analytics Provides Automated and Continuous Risk Assessment For Complex Systems In This Active Cyber™ Interview

July 8, 2024

I remember making the case for automated and continuous risk assessment many years ago when the NIST Risk Management Framework (RMF) was first being drafted and put through some public review processes. Back then, the main focus of the RMF was enterprise IT systems. And back then, there were no tools that could support my vision of automated and continuous assessment. Today, I feel that many organizations are still struggling to get to the goal of continuous assessment and ATO, despite the availability of tools. Today the task of assessing risk must encompass an expanded attack surface in scope and complexity as assuring the secure adoption of AI technology, assessing your software supply chain for risks, and hunting for vulnerabilities in your OT infrastructure have recently emerged as additional challenging tasks – but ones that are critical to securing your business or government agency. Having automated risk analysis tools that are seamlessly integrated using standard methods and interfaces, along with a specialized capability focus on complex systems, can improve the comprehensiveness of risk assessments, reduce time to complete assessments, while providing a continuous review capability. One example of such a tool is KDM Analytics’ Blade RiskManager. I am intrigued by how it combines risk analytics and model-based systems engineering to provide enterprise level risk assessments.  Learn more about KDM Analytics in this interview with Ms. Djenana Campara – President and CEO.  You can also learn more about how Blade RiskManager helps organizations to ensure compliance to Cybersecurity Maturity Model Certification (CMMC) requirements of DoD by listening to this Active Cyber™ podcast. Or just click the ad and be taken to the KDM Analytics web site to learn more.

Spotlight on Ms. Djenana Campara

» Title: President & Chief Executive Officer, KDM Analytics

» Website: https://www.kdmanalytics.com

» LinkedIn: https://www.linkedin.com/in/djenana-campara-4610252

Read her bio below.

Chris Daly, Active Cyber™: Companies struggle to identify and understand what their vulnerabilities are, where to invest their cyber security budget and how likely they are to be hacked. How does modeling the cyber risk help to address these problems? What types of models are most useful for these problems?

Djenana Campara, President & Chief Executive Officer, KDM Analytics: Yes, companies do struggle with this; it is very complex. To do proactive/preventive security, organizations need to have intimate knowledge of their system and understand how that system can be attacked under various conditions. They need to know who may want to attack it and whether those parties have adequate means and opportunities to attack. And, if an attack happens, they need to know how successful the attackers could be.

Unfortunately, most organizations are reactive when comes to cyber security: Either they act after the attack happens and try to plug the vulnerability that enabled attack; or, they act when a known vulnerability is announced by starting a patching process that can take some time to complete. Both measures are vulnerability driven and security measures are considered after the fact. Proactive, preventive measures should be an organization’s top priority. This includes proactively assessing a system’s operational risk; identifying the top risks that need to be mitigated; understanding what is causing those risks (such as identifying attack paths and the vulnerability conditions necessary to make attacks successful); and, proactively resolving the issues by deploying mitigations to prevent any success of a future attack, thereby reducing risks.

With so much information to consider and analyze, modeling is essential. However, the problem is too complex and the threat events too numerous for manual threat modeling and assessment to be feasible. Digital transformation in this area enables us to maximize time and resources: to do less and get more from it.

Active Cyber™: Secure-by-design is a key tenet in CISA’s march to better security for the critical infrastructure. How does KDM Analytics support secure-by-design and how is your risk assessment approach integrated with the software life cycle?

Ms. Campara: Secure-by-design starts risk assessments from the time of the conceptual architecture of a system and continues risk assessments as the design progresses. This is the best way to determine security requirements and bake them into design. As the system design progresses, continuous risk assessment will provide us with updated and more in-depth security requirements. In other words, continuous risk assessment must be part of DevSecOps in the system life cycle.

KDM Analytics’ Blade RiskManager (BRM) extends digital engineering frameworks for system designs to provide continuous risk assessments – checking the pulse of the current security posture of a system and recommending mitigations if needed. This way, both system design and risk assessment work from the single source of truth without “a loss in translation, which can reduce a system’s security. Because BRM provides automated/digital risk assessment, many organizations have integrated BRM into their digital thread to provide continuous risk assessment.

Active Cyber™: Companies are looking to AI and big data analytics to better understand and evaluate the complexities of their cyber risk posture. How does KDM Analytics address these needs?

Ms. Campara: Understanding the cyber risk posture – the set of risk claims that a risk analyst is making about the cyber system under assessment – is like looking at the tip of an iceberg. The larger part that is underwater is the big data analytics. For many organizations, risk assessments are more like a few ice cubes in a glass – they lack the data analytics and instead make subjective risk claims. Other organizations do collect large volumes of cybersecurity data but ignore this data when making risk claims. Their analytics remain dormant and cannot be used to derive objective risk claims from the data – they cannot even use the data as evidence to substantiate subjective claims.

Which process is more effective? To derive risk claims from the mounds of data, or to first frame the risk claims, and then use the cyber data as evidence? Here at KDM Analytics, we believe in the “framing the risk first.” Incidentally, so does NIST, and so do risk management experts.

Our BRM product deploys AI to help risk analysts frame the risk claims and then evaluate the supporting evidence. The inputs to the AI engine are the facts: a description of the system objectives, the structure, and the behavior of the system under assessment. Based on these inputs, our AI engine creates a clear picture of who the possible attackers might be, how they could attack the systems, what attack surfaces the system provides, what tactics and techniques the attackers might use, and how the system will fail when attacked.

We call this first part of the analysis “top-down” – it is based on Systems Engineering big data, not on known vulnerabilities or network scans. For a thorough engineering approach, our AI engine also does a “bottom-up” analysis – it understands which failures caused by potential attackers will impact the objectives of the system. This is a second walkthrough of the same systems. Only when the two walkthroughs are correlated (the penetration walkthrough and the impact walkthrough) are the risk claims made.

Once the risk claims are generated, our AI engine helps risk analysts understand the cyber risk portion as they never saw it before. The AI engine routinely evaluates hundreds of thousands of possible attack paths, builds the big data, and finds the weak spots. The analyst needs to go through just ten, maybe a hundred attack paths, all at the level of the stakeholder concerns. There are no “So what?” questions. This can be done very early in the lifecycle, early enough to contribute to good cyber requirements.

In the traditional approach, cyber security data – the vulnerabilities in the Bill Of Materials, the network scans – is collected very late in the lifecycle, long after all the technology commitments have been. At that point, it’s too late to start a risk assessment. But that same data provides good evidence to the BRM risk assessment. BRM helps see if the system “as delivered” has any gaps in a good security posture that the stakeholders care about, and that is already reflected in the security requirements. Of course, the implementers do not have to wait until the last moment in the acquisition process, they can use the RBM capabilities as they start making technology commitments and do some preliminary testing and evaluations. They can see if they are on track to support the risk posture that the stakeholders already saw.

Correlating the cybersecurity evidence, known vulnerabilities and network scans with the risk claims – this is another deep lake of big data. BRM does both the top-down risk assessment and then the bottom-up cyber data evidence assessment.

Active Cyber™: There are many cyber risk and compliance frameworks available to decision makers and which may also be mandated for use depending on the context – from NIST Cybersecurity Framework 2.0 to GDPR to HIPAA to MITRE ATT&CK and many others. What frameworks do KDM Analytics support and how is cross-mapping done?

Ms. Campara: What is most important is that BRM ties fundamental risk assessment, which is related to attacks and how controls mitigate attacks, to security requirements expressed in broad terms (such as Cyber Security Framework (CSF) or Risk Management Framework (RMF) or Cyber Survivability Attributes (CSA) KPP). BRM overlays controls with the framework of users’ choice and provides full traceability of risk mitigations to a system’s security requirements. Those frameworks are supported by BRM out of the box, and support for additional frameworks can be easily added.

Active Cyber™: Significant cyber threats may emerge within very short timeframes that necessitate immediate response. Identifying threats that are dynamic and fast evolving depend on applying a wide variety of factors such as vulnerability discovery, exploit availability, and cyber threat intelligence. It seems that automating the threat assessment and risk assessment processes are necessary to keep up with the threat. In your experience, what level of process maturity must an organization have to be successful in automating theses processes? Can you characterize this level of maturity?

Ms. Campara: You are quite right, cyber security risks are caused by malicious attackers who are very motivated, and who develop sophisticated attack capabilities. Some attackers are politically motivated – the largest ones backed by nation states – but many attackers are also motivated by the idea of profit. Which group is more successful – the state-controlled and centralized ones, or those based on private enterprise? We observe an alarming level of weaponization of attack technologies and sharing of these capabilities. Selling and renting attack tools is lucrative business.

This is why the area of cyber security is so different from its cousins, safety and reliability. Both analyze risk. Safety and reliability risks are caused by Mother Nature. An aircraft design, once certified, is used for several decades; the same aircraft will safely fly anywhere in the world. Not so with cyber. Attackers’ capabilities evolve. From the cyber perspective, different theaters will present very different threats. So, the risk assessment must be continuously repeated. That is, of course, if the stakeholders wish to be proactive.

This transition to continuous risk assessment is not unlike the one from waterfall development to DevOps and Agile. From the cyber-risk perspective, people have started to call this DevSecOps. Currently, DevSecOps is very much bottom-up, doing vulnerability scans as part of DevOps. True DevSecOps must involve a continuous top-down risk assessment. It should be proactive and therefore risk centric.

However, to have continuous risk assessment and DevSecOps, you need to eliminate the human analyst from the loop – they are too slow, too subjective, and too busy. Training them takes too long. Their performance may change from day to day and person to person. Instead, the whole risk assessment and evidence evaluation must be automatic. BRM was designed with this as the main objective. Its AI engine is fed from a knowledge base (KB) that is unique to a certain threat environment. BRM can evaluate the same system for different environments, or a whole portfolio of systems within the same environment. When you get intelligence of a new attack technique, you can tweak one of the KBs and re-analyze all your systems to see if you need to be concerned.

Active Cyber™: Can you describe what a fully automated risk assessment process looks like?

Ms. Campara: It is quite complicated; however, I can say that automated model-based risk assessment is a game-changing approach. It focuses on the use of discernible concepts and a sequence of steps that maximizes assurance, otherwise it would be garbage-in/garbage-out. Automated risk assessment is based on solid engineering practices where the elements of the architecture and the corresponding elements of risk are managed together – what we call the single source of truth.

An important part of the approach is separating concerns between two inputs provided to the automated solution through its technological framework: (1) system and operational architecture information in the form of the model, vs. (2) cybersecurity knowledge containing information related to threats, undesired events, attacks and vulnerability patterns, security controls, etc. This separation of concerns enables experts in each field to focus and contribute to the respective knowledge areas. The central part of automated risk assessment is where the two inputs are integrated to enable risk analytics.

The key step in the assessment is the automated construction of an attack tree. The system’s operational architecture is analyzed to determine the relative operational importance of hosts and applications. The system data is combined with rules describing attack techniques to compute all possible attack paths (given the rules and data such as attack targets, vectors, entry points, and entry point sources) in the system. The potential attacks are chained together with an aim to compute possible attack paths and are stored in a tree data structure giving the pre-conditions and post-conditions for each attack step. Attack steps in the path depend on attack goals and attack methods chosen by the attacker. Even very small systems can have thousands of attack paths, with each path evaluated for the possibility of a successful attack and the impact if the attack is successful – which will produce a list of prioritized risks.

You can see the complexity involved in the whole process – a manual approach just does not cut it.

Active Cyber™: In your experience, what types of problems may arise due to a lack of automation for risk management?

Ms. Campara: One of our clients gave us some measurements they took related to the time/resources savings of using BRM:

• Manual effort: 10s to 100s of attack paths evaluated in weeks/months (2-3 hours per attack path)
• With BRM: 1,000s -10,000s of attack paths evaluated in seconds/minutes (0.2-0.3 seconds per attack path)

While the time savings are significant, a bigger question is: How do they know that those 100 attack paths that they manually modeled and evaluate are right ones?

The manual approach cannot generate a systematic and comprehensive attack tree with all possible attack paths – it is just not humanly possible. Therefore, coming up with a subset does not guarantee you will hit the right spots and may even make the system less secure. With this ad hoc approach, we’ll never catch up with bad actors.

Active Cyber™: Cyber-physical systems (CPS) have distinctive risk issues since CPSs can impact the physical as well as the digital world. Risk assessments need to ensure that these systems operate in a secure and safe manner. How does KDM Analytics take on goal-directed risk assessments where safety and resilience may trump security? How can automation or models help in these types of assessments?

Ms. Campara: You have used the dichotomies “cyber vs physical” and “safety and resilience vs security” and I would add one more: “cause vs failure”. BRM considers all these types of risks – let’s look at how.

In the traditional safety world, failures occur to physical assets and they impact human life. Here, we deal with random failures and operator errors – this is a rather static world. In the cyber security world, causes are attacks by capable and motivated human adversaries. This is a very dynamic world.

Not long ago, cybersecurity was called “information security”. In this world, failures were to information assets. The biggest concern for decades was a “disclosure” – a failure to keep information confidential. Initially, this related to big-government secrets.

BRM’s approach to “cyber-physical systems” (CPS) is to consider diverse causes, cyber attacks, as well as random failures in software, equipment, mechanical structures, and so on. We come from the sophisticated, diverse world of cyber so adding static causes is conceptually not difficult.

We also consider diverse failures. A typical cyber asset is information. Old “information assurance” processes stop at Confidentiality, Integrity and Availability (known as CIA) – all important, but what about assets such as Capability and Mission? You may even want to keep some mission stealth (mission confidentiality), you may want to keep your critical capabilities available (capability availability), and so on. BRM considers all these risks.

But what about structural assets? Mechanical assets? What if an attacker drills a hole in your physical server? Is this a cyber attack or a physical attack? From BRM’s perspective, the attack vector is physical, the attack is intentional/malicious, the impacts are both physical (you’ve just lost a server, that costs few thousand dollars), and cyber (some capabilities may not be available, some data may be lost, or corrupted, etc.).

Humans are another layer in what BRM considers. A human can be an attacker (a malicious insider, a careless operator, a clueless inspector, a compromised operator – social engineering), or an asset from the safety perspective (impact to the health of the operator).

BRM allows you to filter the risk model by all these aspects: cyber, physical, assets, attacks, attackers, impacts, hazards. We call it a multi-dimensional data space You can focus on only some concerns (e,g. traditional cyber), or you can see the entire risk distribution over all concerns and see what prevails.

Active Cyber™: In your experience, does it make sense / cost-effective to integrate or extend the automated risk assessment process with testbeds or cyber ranges? How would this work and what are the benefits or drawbacks of doing so?

Ms. Campara: Absolutely. This is all about transformation to digital Systems Engineering. Risk assessment is one of the key tenets of Systems Engineering, as is testing and evaluation. I already talked about BRM’s approach and how we believe automated risk assessment should be done using the Systems Engineering big data as input. BRM’s risk assessment is top-down. This fits extremely well into the agenda of the digital engineering transformation.

Testing and Evaluation (T&E), testbeds, and cyber ranges produce valuable evidence for the risk claims. BRM offers a bottom-up integration of such evidence with the risk claims. We talked about this in the context of vulnerability scans and network scans. This is the cybersecurity big data.

Scans are correlated to the top-down risk model though the thing called the Bill-Of-Material (BOM); for its counterpart, the Software BOM (SBOM), there is an upcoming joint standard by the OMG and The Internet Consortium called SPDX.

But what about more general testbeds? How can they be integrated with the risk model? Here, the Model-Based-Systems-Engineering (MBSE) approach is the key. The standard language from Systems Engineering, SysML, includes digital requirements. One of the benefits of the MBSE model is that traceability to the requirements can be described and maintained throughout the lifecycle of the system. Testbeds are also traceable to the requirements.

As the digital transformation of Systems Engineering is adopted, this traceability will be digital, part of the MBSE model. And this very MBSE model, in SysML, is directly ingested by BRM. This is how BRM can easily and cost-effectively correlate testbeds with the automatically generated risk model. With or without MBSE, requirements are the key to correlating test to the risk model. One could input a spreadsheet with the test results and requirement IDs. Digital transformation eliminates manual steps one by one. This is what makes the process streamlined and highly cost-effective.

Active Cyber™: What are the elements of a good risk assessment dashboard for analysts? for CISOs? for senior management? What timeframes should information on the dashboard be updated?

Ms. Campara: Dashboards should be always about metrics and what I call traffic lights – flashing red lights showing where stakeholders’ focus should be. How often it should be updated depends on where and how in a product life cycle it is used. As part of an of an authorization process, less frequent updates are sufficient than in the context of R&D process (DevSecOps, digital thread) or monitoring changes in a threat or configuration environment, where updates should be continuous.

Beyond that, one should also be able to drill down into any of those red lights to quickly navigate toward understanding what, where, and how and much more. Here are some examples of what a good risk assessment dashboard should help an analyst do:

• Look at two systems and see which one has greater risk in a given operational environment. Or, see which system from within a portfolio is most at risk in the environment.
• See how a system’s risk compares to the average risk of other similar systems.
• Look at one system and see which operational environment poses the most risk to it.
• Look at a system in a particular environment and identify its top risk, the top risk component, and what organizational controls can be implemented to lower a given risk. And then, recommend a set of the most effective controls within a given budget.
• Periodically update intelligence about an operational environment and monitor changes in risk across a portfolio of systems.

Thank you Djenana for providing this deep dive into how KDM Analytics provides risk assessments. Having the ability to do continuous and automated risk assessments is becoming the goal for all Defense and federal agencies, whether we are talking IT, OT, or IoT systems. I believe that Blade RiskManager sets the the gold standard for achieving these goals. For more on KDM Analytics, check out Active Cyber’s previous interview here. And thanks to my subscribers and visitors to my site for checking out ActiveCyber.net! Please give us your feedback because we’d love to know some topics you’d like to hear about in the area of active cyber defenses, authenticity, PQ cryptography, risk assessment and modeling, autonomous security, digital forensics, securing OT / IIoT and IoT systems, Augmented Reality, or other emerging technology topics. Also, email chrisdaly@activecyber.net if you’re interested in interviewing or advertising with us at Active Cyber™.