August 21, 2024
CISA has been working hard over the last 6 years to turn the tide on the never-ending stream of zero days and vulnerable systems that continue to plague our everyday lives. In 2018, it formed the ICT SCRM Task Force—a public-private partnership charged with identifying challenges and developing actionable solutions to enhance global ICT supply chain resilience. Composed of federal government and industry representatives from across the Information Technology and Communications Sectors, the Task Force serves as the Agency’s center of gravity for supply chain risk management partnership activity. Recognizing the importance of securing ICT supply chains, on May 15, 2019, the Executive Order (E.O.) 13873 on Securing the Information and Communications Technology and Services Supply Chain was signed into law. E.O. 13873 directs the federal government to strengthen efforts to prevent foreign adversaries from exploiting vulnerabilities in the ICT supply chain and protect the vast amount of sensitive information being stored in and communicated through ICT products and services. At the BlackHat conference in 2024, CISA Director Easterly called out the need to hold software producers accountable for making less buggy code. She emphasized the need to start with a supply chain that has been cleansed on the producer side, rather than putting the onus on the consumer side. She also announced the publication of a Software Acqusition Guide to provide a “demand signal” to the producers so that the interests of the consumers was unambiguous and consistent, while also providing a place where dialogue between parties could take place. 
Following on this CISA announcement, I was able to secure the opportunity to dive into this guide and what this “demand signal” is all about with two of the task force members. One thing that struck me from our exchange was how expansive an audience this Guide will affect. So you can read our exchange below or you can listen to the podcast at this link. Or do both.
Spotlight on CISA Software Acquisition Guide
» Title: Software Acquisition Guide for Government Enterprise Consumers: Software Assurance in the Cyber-Supply Chain Risk Management (C-SCRM) Lifecycle
» Panelists: Joe Jarzombek, Tim Mackey – CISA ICT Supply Chain Risk Management (SCRM) Task Force Members
» Linkedin: https://www.linkedin.com/in/joejarzombek
» Linkedin: https://www.linkedin.com/in/mackeytim
Chris Daly, Active Cyber™ – Today I’m joined by two members of the CISA ICT Supply Chain Risk Management (SCRM) Task Force. The Task Force recently came out with a software acquisition guide. It has a much longer name – Software Acquisition Guide for Government Enterprise Consumers: Software Assurance in the Cyber-Supply Chain Risk Management (C-SCRM) Lifecycle and you can find it at the CISA website as shown above.
I’d like to welcome my two guests, Joe Jarzombek and Tim Mackey to the Active Cyber Zone. Joe. Why don’t you start off by giving a little bit of background on yourself and why you decided to participate in the development of this guide?
Joe Jarzombek, CISA ICT Supply Chain Risk Management Task Force member – So I’ve actually been working the software space since my early career days when I was in the Air Force, and then when I was asked to come into the office of Secretary of Defense, I was specifically asked to start the DOD Software Assurance initiative. We started understanding that the way people are buying things actually has an influence over the products that ultimately get delivered and move forward. Next I’m in the Department of Homeland Security as the Director for Software and Supply Chain Assurance, and again, same messaging because as you know, there are many standards related to software development, even software security. The challenge is these standards are all voluntary. And so having briefed the NSTAC, for the need for a demand signal, the President’s Commission basically recommended that the ICT Supply Chain Risk Management task force take this guide on because we have a public-private collaboration with government agencies and the different industry sectors. And so we did – we launched the Software Assurance working group with the intent of developing a buyer’s guide. In other words, to send the demand signal because if buyers aren’t requesting that products be secured in a certain manner then why should suppliers up their game if the customers don’t care about it? So it’s about the demand signal, and I know Tim’s had that perspective as well.
Active Cyber™ – So Tim, give us a little bit of background on yourself too and why you became a member of this task force.
Tim Mackey, Head of Software Supply Chain Risk Strategy for Synopsys, Inc. and Co-Chair of CISA ICT Supply Chain Risk Management Task Force – So I’ve written software for years and years and years. Some of the early software that I wrote was actually in truly critical infrastructure where if something failed, you might have, for example, the EPA at your door saying, well, how do we go about cleaning this up and what is the risk associated with it? I’ve been in a variety of architecture roles, I’ve been in startups, been in large companies.
So going in to this software assurance guide effort, no one ever actually goes and says, I’m going to go and create poor quality code. But being able to elevate what is a security issue, what is a defect issue, is like a day-to-day battle within development teams. So as my career progressed to having dotted lines to chief security officers and so forth, the impact of poor decision-making within a software supply chain really started to make me focus on the problem quite nicely.
Today I’m with Synopsys and Synopsys provides a whole bunch of tools that go and help people write better software and test it better and have better outcomes. But all that tooling doesn’t matter if there’s no demand signal, if there’s no actual, impetus on the part of the software operator to care about exactly how the software was secured and tested in the first place. And right now there’s quite honestly a whole lot of opaqueness in that process. So that’s one of the big reasons why this acquisition guide is so powerful – it makes the act of getting a little bit of information in a transparent fashion from a software supplier, a software producer, a software vendor, really, really, really easy. It’s not like we have to go and have a six month research project to figure out whether or not a software vendor is doing the right thing. It’s like Mr. Vendor, provide answers that you already know the answers to, yes or no, make it nice and simple. And so that’s really what it is. And then of course, that aligns with my goofball title of Head of Software Supply Chain Risk Strategy, where it’s like, how do we help people do better? How do we have better outcomes?
Active Cyber™ – And so I guess the timing of the guide couldn’t be more coincidental given the CrowdStrike debacle, for example, where that turned out to be a quality issue of their testing from reports about their root cause analysis. So tell me more about this guide now, Tim, why we need this guide now, and what is its goal, and who was involved in its development?
Mr. Mackey – So this was very much a public-private partnership that did everything that you would want it to do work on. This project started about two and a half years ago. We have people like Joe and myself who are leading the conversation. We have people from NetApp and Microsoft and a variety of other vendors that you would expect. And then we have government partners. So contributions from folks from NASA and GSA and NSA all to give this view on what does it mean to actually go and buy secure software or demand that software be secured at the outset, what are the challenges? So as part of our research phase, we were bringing in teams from NIST, from MITRE, from other government agencies to explain how they went about recognizing that this software is or isn’t secure. So since we could actually have all that corpus of information we were not setting out to go and blaze new trails or a new standard or what have you.
Mr. Jarzombek – And in particular, a while back, NIST published the Secure Software Development framework, and that’s very key because it basically codified all these things they were bringing together from different standards. Then DHS, working with other agencies, came out with a secure software self-attestation form, and that took a subset of the NIST Secure Software Development framework – just a subset of practices that was viewed to be very key. The attestation form was evolving over time as we were developing our guide, we knew that was in play, so we were able to leverage that effort along with what NIST had done. We were also able to leverage content from the supply chain risk management practices that are in NIST 800-161. We were able to even pick up the controls that are in NIST 800-53.
The idea is that these guides and forms all have alignment and that they come together. We went beyond what is in the self-attestation form, which is very necessary, but it’s not sufficient by itself. You can’t make a truly risk informed decision about a supplier or their product from the self-attestation form. So we expanded beyond that. The guide actually picked up things out of FedRAMP, as an example, with the deployment activities that we have. We are also able to introduce some things that have become very much a part of development today – they’re bringing in a lot of open source software. They’re actually using artificial intelligence in their tools for developing software. So we want to be able to address those nuances as part of our guide.
Mr. Mackey – And one of the things to emphasize is that this isn’t just about software development. There’s a lot of emphasis in industry and in the media today saying, “if we only develop software better,…” The reality is everyone has a role to play. So if you’re not doing the right things from a development perspective within your organization, that’s one thing. But today, every organization’s ecosystem is very, very complex, and it effectively comes down to “do you have a supply chain that you depend upon that might be commercial, that might be open source.” These developers for these other sources of code are probably not following the same standards that you do and have the same security and quality targets that you do, but that’s going into the software you produce. Then that software is being shipped off and a customer’s being asked to actually operate it and deploy it. Then there’s going to be the inevitable patch side of the equation. So we took a clear look at the overall ownership life cycle of software as opposed to just saying, well, we’re going to focus on development.
Active Cyber™ – Nice. Okay. So Tim, I understand you were able to take the opportunity to visit the BlackHat conference where Director Jen Easterly announced the publication of this guide. Can you tell me a little bit about what she said about it?
Mr. Mackey – So this announcement of the guide was part of a day two keynote that she delivered quite literally yesterday at BlackHat. The hacker summer camp was well in force this week, and she was talking about the types of issues that people have with software. She discussed the distinction between what is a quality issue, what is a security issue, what are the types of scenarios that we see that result in exploited vulnerabilities, and just weave all that together in a “secure by design” narrative that CISA has been having for the better part of the last year. She introduced new concepts of “secure by demand,” but also mentioning that because this is, to use my term, fundamentally a team sport, that the acquisition teams have a role to play. Of course they’re not necessarily the most technical folks out there. So if somebody says, “yeah, sure, I have a security tool that does something and here’s my piece of paper,” they don’t necessarily have a way of assessing that. So having a level of consistent representation of what “it” is was one of the things that the acquisition guide really focused on.
Active Cyber™ – So you mentioned Secure by Design, and I think one of the other key buzzwords we’re hearing from CISA is Secure by Default, but you also brought up this other term, Secure by Demand. So these are all different principles that CISA has been pushing from a security perspective. Can you talk to me a little bit about what these principles really mean? How do they underpin the acquisition guide and how is Secure by Demand different than Secure by Design or Secure by Default? Joe, why don’t you start?
Mr. Jarzombek – A very short way of describing it: Secure by Design is about how you design your software, your products in the first place, and some very good guidance on that comes in the guide. Secure by Default says, how did you deliver it securely configured, because there’s many configuration options that people have, and to deliver it open-ended and say, “oh, but you can configure it securely yourself, Mr. Customer” is different than saying we’ve delivered it secure by default. But now Secure by Demand addresses the notion that says, “if the customer never asked for it, why should we deliver it?” And so that’s why, as Tim says, it’s all of us. It’s a team sport. If customers don’t demand it. Then a supplier might say, “…that costs me a little extra. So if you’re not asking for it, why should I deliver that?” Tim, you probably have some perspectives on that.
Mr. Mackey – Oh yes, the concepts of “I’m going to bolt security on,” or, “this is really, really expensive,” are behind a variety of poor decision making that’s resulted in large scale vulnerabilities or data breaches or ransomware attacks or pick your favorite poison. So if for example, you have a piece of software that has this hardening guide associated with it, well then that implies that by default it is not hardened to something that is recognizable. And that presumes that the person who’s actually going to set this thing up and deploy the software: a. knows that there is a hardening guide; b. can interpret what the hell it says in the hardening guide; and c. can get approval from the appropriate powers within the organization to make those sets of changes and that they align with whatever the corporate responsibility is.
So that’s one scenario of Secure by Default. Another scenario is where you have something that’s more consumer-oriented – the average consumer of software-powered things today has no clue, for example, how to patch openSSL. It just magically has to happen without bricking their device. There’s just a whole sequence of activities and expectations that you just don’t get if you’re not Secure by Default. And so that ends up being a key element of a demand signal, which is: “why should I have to invest as the procurement team or as the ultimate operator of this software in going and having this extra hardening exercise? Why didn’t you just put those as the defaults in the first place?”
Mr. Jarzombek – Right now, every vendor will say, well, to do those things cost me more upfront. That may be true, but you know what we’ve already learned – there’s many case studies that shows if you actually secured it upfront, it costs not just the end user less money, it costs the supplier less money. You don’t have to go back and repatch, you don’t have to worry about your brand reputation because you’ve been deploying faulty products.
So one of the challenges we found, especially within the government, and that’s why this guide was focused on the government, is their approach to contracting for goods and services. You’ve heard the term lowest price / technically acceptable. A lot of procurement guys focus on the lowest price. And so the guide helps you get away from initial focusing on lowest price. Instead you focus first on technically acceptable. And so we’re trying to raise the bar on what is technically acceptable in terms of what does “secure” mean in terms of software and the cyber assets that software is run on. And so if you compete on technically acceptable first, then try the lowest price, then it’s a level playing field for everyone. Otherwise, Joe’s Garage Shop will come in and say, ah, just use my software. It doesn’t cost you much.
Active Cyber™ – I remember a guy I knew from IBM research who was talking about bugs in code and he was saying, you got millions, billions lines of code and there’s a percentage of bugs that are going to be part of those. And so I kind of feel, I think we all do, that companies put out code that have bugs in it because they could go on forever testing and trying to find another bug. And so at a certain point, you have to get the product out or you lose money. I think Director Easterly and other people’s thoughts were, “Hey, we’ve got to reverse this a little bit because there’s too much of an onus on the consumer to find the bug and be the tester, so to speak.
Mr. Jarzombek – Every consumer was beta testing.
Active Cyber™ – Right? Versus the other way around. Now, I think there’s got to be a happy medium. I don’t know if we can expect that the vendor’s going to find all the bugs because there’s a never-ending supply these days. So I guess my question is how does this guide meet that balance of the vendor over-spending and the consumer over-expecting in terms of what’s going on here?
Mr. Mackey – Yes, so that’s actually pretty important question. I’m going to come at it from a couple different angles. So if you look at the guide itself, it’s organized effectively into two major parts. There’s a governance section, and then there’s all the detailed control questions and so forth. The first task for the acquisition team is actually go out and determine whether or not the supplier is doing the basics. Joe mentioned earlier about the CISA self-attestation form which are the basics.
The next reasonable question would be, “do you know, Mr. Vendor, where all your code came from? Yes or no?” Now, there’s no judgement call around yes or no being a good or a bad thing at that level. But the procurement person goes and asks the question, do you know where your code came from? Yes or no? And a vendor goes and says, “well, I’m not going to answer that. I’m not going to tell you who all my suppliers are. That’s trade secrets.”
One of the things that we did was we put in the appendix of the guide a set of rationales for each of these governance questions so that now the procurement person, who is a non-technical person typically, can go and just take that text, put that back in the response, say, “no, we care about it because these things that were written by people who are actually subject matter experts in the risk assessment of this.” Now the acquisition official can go now and tease out more information. They can say, “oh, okay. You know what? I’m just asking a yes or no question. I’m not saying I need 17 volumes of evidence. I may need 17 volumes of evidence, but I don’t need it now.”
So the guide helps the acquisition official to be able to actually recognize those types of security practices, recognize that there should be policies and procedures for specific scenarios. Asking “do you hold your software suppliers to the same level of account that you hold your internal development teams?” Really, really basic kinds of questions that get into the heart of a vendor’s security philosophy, part of the overall mindset within that supplier, or is security something that the vendor expects to be really just bolted on later?
Active Cyber™ – Okay. So both of you have mentioned so far that there’s other guides, other standards, other things to point to. How does this guide related to those things? For example, I know CMMC, we’ve talked about CMMC in the past in a previous podcast with you, Joe, so how does that relate? I can see a vendor saying, okay, I got a checklist for this thing. I got a checklist for that thing. I got to do this self-assessment. I mean, how does it all pull together here from a vendor perspective and from the government perspective?
Mr. Jarzombek – And our intent was, if you’ve gone through any type of assessment associated with relevant standards, you should get credit for it. Now the question is, is it relevant? For instance, if you’ve gone through a FedRAMP certification for Medium, then you get to skip all of our deployment questions because we’ve known you’ve already gone through that. But you know what? If you’ve gone through a CMMC assessment, if you’ve got that certification then that’s nice, but it doesn’t relate to “was the software secure in the first place,” that issue is not even part of the CMMC questions. CMMC is all about protecting data. CUI [Controlled Unclassified Information] in particular.
Now, I will tell you that if you’re using our guide to look at how the software was developed, you can actually better meet your qualifications for CMMC because it’s about developing software that doesn’t leak data. I mean, fundamentally, if you’ve got weaknesses in your software that allows somebody externally to come in and read or modify data, how can you say you’re compliant with CMMC? So it’s the converse, but just because you’ve done CMMC doesn’t mean that you’ve satisfied any of these secure software development activities.
Active Cyber™ – Well, what about things like Common Criteria and FIPS 140? How do they play into this?
Mr. Jarzombek – We didn’t do an exact mapping there, but you understand that those are different in the sense that Common Criteria, you don’t even have to look at the software. I mean, there’s a whole lot of certification that goes in there.
Mr. Mackey – You don’t even need to have a running system for a Common Criteria evaluation. You can make the security target so narrow that it’s not even a usable system.
Active Cyber™ – So in a way, this Guide is kind of a replacement for Common Criteria because Common Criteria doesn’t really go as far as you need it to go.
Mr. Jarzombek – It’s just that we’re not in that regulatory space right now. As you know, certain high assurance environments require the Common Criteria. That’s a legacy way of looking at it, but it’s still there. We’re not going to say, oh, you can throw that out. No, we’re not going that far.
Active Cyber™ – Makes people mad over at NIST, I guess. All right. Okay. So the guide sounds like it’s focused on the process of making software or the end result. I mean, that’s kind of the question, which do you say is its main focus – process?
Mr. Jarzombek – You are correct, but I will give the caveat at the very end, our very last governance question that says, do you have a license that allows for an independent third party to scan your software for security and provenance? And that’s very important because at the end of the day, while we expected suppliers to test their products, and more particularly, to understand their own supplier’s products that they’ve incorporated into theirs, and we do ask those questions, do you test for those? But the government still reserves the right that says we need to be able to scan your products. And if you have a license that says you are prohibited from looking at my product, which by the way, there are licenses that are restrictive like that, you should know that going in because maybe you don’t want to purchase that if you’re prohibited from even looking at It.
Active Cyber™ – When you say “looking at it,” are you talking about looking at the source code or are you talking about just testing it through DAST or SAST techniques?
Mr. Mackey – Yes, all of the above. So if you look at a commercial software EULA, it will have: “we disclaim all of these things and it’s all the onus is on you to go and take your due diligence around this. But oh, by the way, you cannot disassemble this. You cannot reverse engineer this. You cannot do all of these things, which will somehow cause you to know more about how this software was developed.” Essentially many software vendors don’t accept any responsibility, and generally prevent you from actually going and assessing whether or not this is an appropriately secure set of code.
Mr. Jarzombek – And so we also understand that not everybody’s just going to be willing to give their code away and have the government scan their software. And so part of our question says, “do you allow an independent third party who you have an NDA with to scan it and they hold that information so that it’s available upon requests. Information that says you scanned it for these things?” And so it doesn’t have to be the government who does it, but it does have to be somebody who’s independent from the supplier.
Mr. Mackey – Yes. So the basic scenario is for the software provider to the government to provide an SBOM [Software Bill Of Materials]. The provider could be given another SBOM from their supplier along with their application. The question is does the provider have the ability under the supplier’s EULA to scan that application using your favorite software composition analysis tool, generate an SBOM and compare the two SBOMs together the way that EULAs are written? Typically, the answer to that is, well no. Well, this is an effort to say in this government context, that we, the government, expect to be able to do an IV&V [Independent Verification and Validation] on the software artifacts that you’re supplying.
Active Cyber™ – So does the guide presuppose the maturity of SBOMs in a way?
Mr. Mackey – No, it’s not necessarily saying SBOMs in general. This could be running DAST against the system as deployed. It might be having an acceptance phase that has an independent agent involved, or maybe there’s actually other techniques to assess the security of the software. SBOM is just an example of a validation technique.
Mr. Jarzombek – Right. And Chris, I get by the nuance of your question, you understand not all SBOMs are created equal. That’s exactly right. But I will tell you, lack of an SBOM is a very telling story. So at least get started in that space. There are so many ways that you can generate an SBOM today. Oh, by the way, if you’re unwilling to do that, somebody can independently come in and generate an SBOM using your software.
Active Cyber™ – Okay. So let’s see where this is all heading. From a guide perspective, are we getting to a point system where as an acquisition official, I use the guide to review product features and I come up with a set of points to determine your product security score versus another product’s security score? Is that where we’re heading a little?
Mr. Jarzombek – No. And the guide doesn’t focus on product features. I won’t use the word scoring, but you go through the guide as a supplier and you address what you can. If you can’t affirmatively answer all the governance questions, and you say, “this is all I’m submitting right now,” then acquisition community takes a look at that and now addresses your answers with their CISO or their CIO. They may ask “can we afford the risk exposure associated with this product going into our environment?” That’s what it really comes down to – so that you’re making a more risk informed decision about potential products or the suppliers behind them. That’s what this guide’s about.
Mr. Mackey – Yes. So here’s an example. If you look at the normal security questionnaires and RFPs / RFIs that go out to suppliers every day, not just from government, but in any industry environment, they tend to be questions that are written in a way that you can’t even necessarily figure out what they’re asking. They tend to be compound questions that you don’t know what the end objective should be, but they actually always say, just answer yes or no to this thing that is a non-sequitur.
What we’re trying to do is to actually baseline everything so that we make it very, very easy for someone who has already done a CISA attestation form to say, yes, we’ve already done this. We’ve already done FedRAMP. Yes, yes, yes. There’s no requirement to provide evidence the way that things are structured. We’ve got our 19 governance questions, we’ve got control questions. There are 60 something control questions in four control groups, and underneath each of those are 316 task questions. No one in their right mind is ever going to want to answer 316 task questions “did you do all of these things?”
Mr. Jarzombek – Help inform what do you mean by the control question.
Mr. Mackey – Exactly. So if you start from the top, which is really the best way to look at it, here’s a governance question. If you answer yes to this, you get to skip other questions that are in the guide – “Yes, I am doing those things.” We created a companion piece to this guide as an Excel spreadsheet where if you go through it and you actually answer the questions, it becomes very obvious what you possibly skipped. So I can’t remember off the top of my head, Joe, if you answer yes to the CISA attestation question, I think it’s 25 control questions can be skipped,
Mr. Jarzombek – Right? It’s at least 25, 26, 27 questions. We firmly believe that virtually everyone’s now going to be requested to do the CISA self-attestation form because now individual agencies are coming out with a companion or a similar one. GSA published an attestation form, which mirrors the content of the CISA attestation form. Other agencies are doing that also, and their starting point is using the CISA self-attestation form. So we know that’s going to be asked for.
Active Cyber™ – So is reciprocity going to occur if I’m doing a form for NASA versus GSA versus DOD?
Mr. Jarzombek – In our guide, we actually said that, have you done this for others? If so, provide that. And if there’s a gap, let’s say somebody was asking for something, but it wasn’t everything that you wanted to ask for. Okay, well that’s a delta. I’m now only going to ask you for these extra questions. That’s the intent behind the document anyway.
Active Cyber™– Alright, so if I read this guide, are these questions asking about things that a layman acquisition official should be able to understand what is being asked for, and understand the responses they’re getting back? Or are we still looking at a cyber specialist getting involved in trying to help interpret both aspects?
Mr. Mackey – So there is an addendum to help clarify the governance questions, why these questions are important. So you go to that and an acquisition person can say: “okay, I understand why I’m asking this.” Now it could be that when they receive the vendor response, they get all this complex answer back. They may, themselves not being a cybersecurity expert, say: “well, I’m not really sure this is good enough.” That’s why it would be a conversation with their CISO or their CIO.
And so if you look at the guide, the questions are yes and no – there’s no specified request for evidence that should be supplied because the evidence is contextual and being able to have the simple yes or no type of answer facilitates the baselining. It also reduces the complexity of response which is intended by the guide authors, because among other things, the first time a supplier receives this guide’s questionnaire, they’ve got to do a little bit of work. The next time they see it, it’s like, “oh, wait a minute, this thing’s pretty straightforward. This is not going to take me hours and hours and hours to go figure out what the answers are.” In fact, upfront we say, if you’ve already submitted this, gee whiz, just keep using what you got and submit it. The intent is to make this as easy to use as possible. And that’s one of the reasons why we focus a little bit more on “do you have policy and process around developing X,” as opposed to being very prescriptive and saying “thou shalt perform this type of test with this tool in this context.”
Active Cyber™ – Is there a refresh requirement for these things as well? Like, you sent me this thing a year ago, the answer should have changed maybe a little bit. Maybe you’ve got a new version of your product or whatever. How does that work?
Mr. Jarzombek – So understand that right now nobody’s described a shelf life, even for the self- attestation forms. There is an intent behind this lack of specified shelf life. Remember SBOMs, for instance, the self-attestation form does not require an SBOM. The software bill of materials we view is very important, and that’s why it’s right up front as one of the governance questions. Do you provide that? An SBOM is expected to be updated every time you update your software, you update the SBOM that goes with that. So that part is to be refreshed. If you submitted a self-attestation form for a product you sold to the government a year ago, are they going to be asking you for a new attestation form? That’s to be determined.
Active Cyber™ – Right? But this does cover the SDLC, right? So it’s not just a point in time when it’s developed, but it’s also when it’s in use somewhere at an agency?
Mr. Jarzombek – To your point, and we understand that some organizations are going to realize by going through this – “Hey, we could really be better here.” They could choose to update their response. Maybe they have developed a POA&M [Plan Of Action and Milestones] to update their SDLC process because “okay, we’re not doing everything. We’re going to get better.” And so the guidance that’s in the guide – the control questions for example, can actually be used in your POA&M. Now we can affirmatively answer these things we weren’t originally doing. So you’ve built up your response due to your plan of action.
Mr. Mackey – Yes. One of the things to remember is that this is more about the actual development of the software as opposed to any individual feature. So if you have a software development pipeline that is structured a specific way and is performing all the appropriate tests, well, the individual testing software itself might get updated over time, but the pipelines themselves, how they’re configured, what are the access controls around them, all that kind of stuff, it tends to be relatively stable across multiple releases. So to Joe’s point about POA&Ms, if you ended up in a situation where you needed a POA&M in order to get the contract, well now you can actually use the guide responses to say, well, this is what should be in the POA&M, and then measure against whatever the progress is towards addressing that POA&M’s requirements.
Active Cyber™ – I see your point, but I also think that these processes by which code is created, they tend to go bad after a while. You’ve got people that weren’t trained properly that are added into the SDLC process, people come and go. You also just have breakdowns in the quality of the testing, CrowdStrike, for example. So it seems to me that you would want a refresh period or at least something like that to reflect change. I mean, even Microsoft got accused of bad culture, security culture there for a while, right? So it would seem to me that this is not just a point in time, but it would be a continual thing. I don’t know how you would implement that with this guide.
Mr. Jarzombek – You’re mostly correct on that aspect. If it’s truly a security culture, you’re not going to let it diminish as much. The challenge is people introduce a new practice. If it’s not actually ingrained as part of the corporate culture about how you bring in and field products, yes, you’re going to miss that. But so long as it retains leadership attention that “we’re going to do this,” you’re more likely to get good results. I’m not saying our guide’s going to correct a bad culture, but it certainly should influence what people incorporate saying, “are we doing these things? What are we doing to make sure that we do these things?”
Active Cyber™ – I did an interview a while back with a person that was involved with bug bounties. In some ways, a bug bounty company can be considered part of the SDLC since the bounty hunter gets paid for finding defects in the code. How does your guide incorporate that practice as far as is that okay to be done as a method for your quality test? Is it okay saying, “Hey, I’ll pay for bounties as a way of independently testing my software?”
Mr. Jarzombek – It’s a form. But basically, remember, you can’t test in security. You’re only basically saying, well, you obviously didn’t do everything here. We caught it here at test. Sometimes people complain that testing takes too long, but if you actually look at what people are doing when they’re doing testing, they’re actually fixing things that should have been done in the development or even the design trying to fix it then. And so testing can take very long in order to pass a test. They’re often fixing things, so don’t blame it all on the testers. And going back to bug bounties, just because I found it doesn’t mean I’ve corrected how you actually developed it, but it is a form. I mean, you can say, we’ve had bug money. It’s a form of saying, how did you independently test it? It’s a form of doing that.
Active Cyber™ – Okay. So I’m looking at SBOMs, for example, and I’m thinking, okay, they’re kind of new. Do you feel like this guide is going to help drive greater acceptance of SBOMs throughout the industry?
Mr. Jarzombek – Again, the demand signal, yes.
Mr. Mackey – Yes. And this is really truly focused on the demand signal. So being able to actually have SBOMs is one thing, being able to have SBOMs that are accurate and reflective of what’s in the software is another thing. Well, fundamentally, SBOMs are part of phase two and phase two requires people to start asking for SBOMs and for SBOMs to start having some consequence.
In general, everybody’s created SBOMs. There are multiple reports that have come out that the quality of SBOMs is quite suspect, and we need to get over that particular hurdle because, to the best of my recollection, the only entity that’s actually requesting SBOMs as part of any truly regulatory workflow is FDA. And this guide wouldn’t be the FDA approach to things, but they can be the leader in that space.
Active Cyber™ – I know the IoT folks over at NIST recently came out with a labeling program saying this IoT product has been certified to a certain level of security. When I think about IoT, I also think about how much of those products are coming from overseas places, and it may not be likely that you would get an SBOM or any particular type of certification that could be realistically approved or apprised from those sources per se. So how are you going to deal with those nuances in the supply chain for things like IoT?
Mr. Mackey – So that becomes a case of what is the demand signal? So if the demand signal is I need to have a self-attestation form, I need to have a specific subset of the questions in the acquisition guide answered in the affirmative. If that offshore supplier isn’t willing to play well, maybe a reseller or a subprime or a prime might be willing to proxy that deficiency and take ownership and responsibility for their particular supply chain partner. But this really is that demand signal that you can’t just go and pick the least cost thing off of Amazon and call it a day.
Mr. Jarzombek – And to the point, as Tim said, a lot of times it’s a system integrator, it’s a reseller, somebody who wants to make the sale to the government. That entity says, “oh, but the customer demands this extra thing.” It may be an SBOM. Well, they can take that OEM’s or vendor’s product and say, if we’re going to resell this, we’re going to have to generate an SBOM. So they can bring in the tools and generate an SBOM of the product that was offered by somebody else. So the reseller can now pass that on to the government entity or anyone else who was saying, “we actually need an SBOM for us to buy this.”
Active Cyber™ – Okay, that sounds like a good way of handling it. So we’re coming up to the end here. Any final thoughts or comments about the guide, Joe?
Mr. Jarzombek – Well, from my perspective, this guide has definitely raised the bar for assuring products bought by the government. It’s a common way of bringing true transparency to what suppliers are doing. And remember, the supply chain is not just the one the government is buying from. It’s the downstream side of it. And so the questions are actually asking, how do you as a supplier deal with your third party components? If you brought in open source, if you brought in something from overseas, whatever, we acknowledge that this is going to be happening, but we want to know how you’ve managed your risk before you pass on this product. To us, that’s really what it is so that we can have that discussion to make a risk informed decision.
Mr. Mackey – And for me, it really becomes a case of being able to enhance trust across the entire software supply chain, independent of whether it’s an open source component, a commercial component, a contracted component, a what have you. Being able to have a greater level of transparency that the suppliers are doing the same kinds of things that I would expect my team to do in this context, or at least that we’re aligned in terms of our objectives and our measure of outcome success is what this guide is after.
We’ve actually, as Joe said, raised the bar. The ability to actually go back and say, we’re demanding these simple things and we’re demanding a level of consistency for the quality and security of what’s being delivered. That’s a win.
Active Cyber™ – This is great. I think I had my doubts when we first started on this discussion about the benefits of this guide, but I think you’ve convinced me that this thing definitely has some good possibilities for doing things good in the security world. So I appreciate your work on this, and I appreciate the time today to go over the guide and what’s in it, and I call out my visitors to check it out for themselves.
Mr. Jarzombek – Actually, the way that they can check that out, go to your favorite search engine, just put in “software acquisition guide,” and the CISA website will pop up and you can go there. Not only do you get the guide, you will also get the spreadsheet, the tool that actually allows you to go through it and quickly answer how are you doing relative to those control questions.
Mr. Mackey – And in the spirit of transparency, on that page is a little survey that if you like the guide, if you found it useful, if you think that it needs an improvement here or there, fill out the survey, let us know because this is version one.
Thanks Joe and Tim for this in depth review of the new Software Acquisition Guide. Thanks also for your hard work in making this guide a reality. Providing vendors a clear and consistent demand signal for building security into the supply chain, SDLC, and products will hopefully lead to more ‘secure by design” and “secure by default” products. And thanks to my subscribers and visitors to my site for checking out ActiveCyber.net! Please give us your feedback because we’d love to know some topics you’d like to hear about in the area of active cyber defenses, authenticity, PQ cryptography, risk assessment and modeling, autonomous security, digital forensics, securing OT / IIoT and IoT systems, Augmented Reality, or other emerging technology topics. Also, email chrisdaly@activecyber.net if you’re interested in interviewing or advertising with us at Active Cyber™.
