with Sammy Migues
Is information security training a complete waste of money as some security pundits suggest? Not when it comes to software security. Actual data from the Building Security In Maturity Model (BSIMM) study of 51 firms sheds some light on the latest tempest in a teapot. The fourth version of the BSIMM, will be released on Sept. 18 and includes a completely revised set of data about the role training plays in 51 real software security initiatives.
Dave Aitel's Blustery Exaggeration: Training is Worthless
Dave Aitel apparently doesn't think much of training, especially basic security awareness training. His recent article in CSO magazine, “Why you shouldn't train employees for security awareness,†certainly sparked a (well calculated) firestorm. Of course Dave lives off of saying outrageous things. Fortunately, he's really smart and can usually either back them up with something tangible or just smile when you catch him spouting nonsense.
Sadly though, Dave's anti-training hypothesis is increasingly common among security people with little or no operational experience. On one hand, if you are a CSO responsible for a cast of thousands, you know just how much security ignorance needs stamping out (hint: even the most obvious lessons can help). On the other hand, if you are an uber-elite penetration tester, security ignorance is your best friend.
Ultimately, disproving Dave's hypothesis properly would require proving a negative. That's because you would need to both train some people and not train the same people, then observe the differences. Short of advances in time travel, this is not something we're going to be able to do.Â
That means figuring out what to do about training boils down to thinking about risk management and proper allocation of resources. Should we completely abandon awareness training? No. Should we put all of our eggs in the training basket and stop paying for other technical controls? No. Well then, just how many eggs should we put where? Incidentally, that's the kind of data we're trying to get to in the BSIMM project (see below).
An analogy with commercial antivirus products may help. Is AV useless (as some experts claim) because some viruses get through? Um, no. Epidemiology studies show that prophylaxis in even just 90%-95% in some populations prevents epidemic spread. So it's the dingbats who think you must have 100% AV coverage for it to be effective who really need educating!  (Plus AV is cheap these days and we can save the “monoculture†scenario for another time.)
Lets pick apart Dave's main argument against awareness training-the phishing and social engineering argument-using the same logic we just used to salvage AV. Dave argues that an average click-through rate (employees clicking on malware) of 5-10% “even with these [training] programs†is an awful result and should lead us directly to abandoning our awareness training programs. But 5-10% is a manageable number. Look at it this way: training reduced an intractable problem of every employee doing something stupid to just a handful per hundred. This is a good thing from the perspectives of risk management, due diligence, and every other grown-up way of slicing it. Because of this, training helps with insurance claims, E&O insurance, FTC consent decrees, and piles of other stuff that anyone with operational experience understands.
While we're on this issue, we just have to point out that every social engineering attack will work on someone. Training simply raises the bar; it's not an impermeable shield. Besides, when phishing stops working, we'll see more kidnappings, blackmail, hostage situations, and other forms of coercion.Â
Ultimately, when Dave says, “Instead of spending time, money and human resources on trying to teach employees to be secure, companies should…,†he is correct about everything but the “instead of†part. When Dave says “It's the job of the CSO, CISO, or IT security manager to make sure that threats are stopped before reaching an employee-and if these measures fail, that the network is properly segmented to limit the infection's spread,†he's right. However, reducing secondary measures (such as training) to zero remains a spectacularly bad business decision.
Awareness Training for Developers
This whole debate becomes even more extreme (and the answer even more obvious) when we talk about awareness training for developers and architects. In our view, awareness training for developers is an essential practice. That's because developers are busy building the things we're supposed to use, and when they build insecure things we all suffer. Even a little bit of awareness can go a long way.
So how much effort do organizations put into training their development staff as opposed to the other possible things they could spend on? Let's find out.
BSIMM4 and Training for Developers
The intriguing thing about the BSIMM Project is that it collects, organizes, and publishes data describing the actual software security activity of 51 firms. Those companies among the 51 who graciously agreed to be identified include: Adobe, Aon, Bank of America, Box, Capital One, The Depository Trust & Clearing Corporation (DTCC), EMC, F-Secure, Fannie Mae, Fidelity, Google, Intel, Intuit, JPMorgan Chase & Co., Mashery, McKesson, Microsoft, Nokia, Nokia Siemens Networks, QUALCOMM, Rackspace, Salesforce, Sallie Mae, SAP, Scripps Networks, Sony Mobile, Standard Life, SWIFT, Symantec, Telecom Italia, Thomson Reuters, Visa, VMware, Wells Fargo, and Zynga.
Training is one of the 12 core practices described in the BSIMM, and the BSIMM has plenty to say about how training is being used when it comes to software security. We provide a pre-release look here at the completely overhauled Training practice as covered in BSIMM4.
For what it's worth, the Training practice had become increasingly problematic statistically from BSIMM2 to BSIMM3. For BSIMM4, we took the bull by the horns and completely revisited all of the activities and their levels. To accomplish the overhaul, we made the six changes to the Training practice that all involved moving activities between levels. You see, BSIMM is a data driven model, and training has undergone some big changes in the last four years (our data set has grown ten times as large as well)! This is the first time in the BSIMM project that we have completely overhauled one of the twelve practices.
Here is a skeleton-level view of the 12 activities taken directly from the BSIMM document. These 12 activities encompass all of the software security training activity that we observed among the 51 firms. These are statements of fact, not opinions about what you should do. In the BSIMM document itself, you can find detailed descriptions of the activities which all include actual examples. We have included one such description here as an example.
T1.1 Provide awareness training. The Software Security Group (SSG) provides awareness training in order to promote a culture of security throughout the organization. Training might be delivered by members of the SSG, by an outside firm, by the internal training organization, or through a computer-based training system. Course content is not necessarily tailored for a specific audience. For example, all programmers, quality assurance engineers, and project managers could attend the same Introduction to Software Security course. This common activity can be enhanced with a tailored approach to an introductory course that addresses a firm's culture explicitly. Generic introductory courses covering basic IT security and high level software security concepts do not generate satisfactory results. Likewise, providing awareness training only to developers and not to other roles is also insufficient.
Note that the breakdown of BSIMM activities into levels is meant only as a guide. The levels provide a natural progression through the activities associated with the practice. However, it is not at all necessary to carry out all activities in a given level before moving on to activities at a higher level in the same practice. That said, the levels that we have identified hold water under statistical scrutiny. Level 1 activities (straightforward and simple) are commonly observed, Level 2 (more difficult and requiring more coordination) slightly less so, and Level 3 (rocket science) are much more rarely observed.
Here are actual data reporting how many times each of the twelve activities in the Training practice were observed among our population of 51 firms. This gives you some idea of how widely adopted various activities are.
Of particular note to our discussion here (and highlighted), awareness training (of the software security variety) is provided to the development staffs of 38 out of 51 firms (74.5%).
The training practice as a whole encompasses only 12 of the 111 activities described in the BSIMM. Remember, these activities are not opinions about what you should do in your software security initiative, they are facts about what 51 firms are doing today in theirs.Â
As we said above, training is only one of 12 core practices. Here is a look at how level of effort stacks up in each of the 12 practices including training.
The practices each have three bars representing three distinct verticals in our study of 51 firms. The bars all equate level of effort in a given practice to raw activity count per possible activity count (a percentage) for the practice. The blue bars represent Financial Services firms (19 firms), the red bars represent Independent Software Vendors (19 firms), and the green bars Technology firms (13 firms).
If you're looking for answers regarding how much effort to allocate to training there are some interesting data here. Training shows the least amount of activity among the twelve practices when averaged across all three verticals (28.31%)! But it isn't zero.
Is there Trouble in Training-land?
There are several training challenges we have observed over the years as we have collected data for the BSIMM. Many of them relate to the “how†of training, not the “what†and “why.â€
Number one is that many firms started with training alone years ago, then “checked the box†and promptly stopped thinking about it. Meanwhile their developer churn has been huge and software security has been galloping ahead as a discipline. So the “we did that†green box on the CISO's dashboard now applies to developers who work elsewhere. The time has come for firms who find themselves in this situation to take another look at who was trained when and what they may or may not remember.
Number two is pushback in some firms from the internal developer community who claim (vociferously) that they “get†security (since many have now grown up with the theme through school and internships) and that much of secure development is “simply knowing what procedure or class or function or whatever to call when.†This has depressed demand for software security awareness training that establishes foundational knowledge in favor of “just tell me how to write this line of code right now to do this specific thing.†Sadly, developers seem to know less than they need to know about software security writ large. Getting them to understand this knowledge gap is the key. We hate to say it, but sometimes a penetration test can break the “we already know all that†logjam.
Number three is simply the quality of training available in the market. Developers are rightfully wary of yet another time-suck in their already busy schedule. Training that isn't tailored to their needs, necessary for a problem they have now, and accessible on their schedule likely won't be successfully deployed.
Ultimately, we believe awareness training is something that all smart CSOs will continue to invest in, whether it is for the entire staff to understand the hostile environment around them or for developers to understand more than “security problem X = use function call Y.â€
About the author:Â
Gary McGraw, Ph.D., is CTO of Cigital Inc. a software security consulting firm. He is a globally recognized authority on software security and the author of eight best selling books on this topic. Send feedback on this column to editor@searchsecurity.com
In the case of sabotage, a business gets attacked from within its own safe walls, eventually putting it in a deep slumber it might never recover from. This kind of thing happens often in even the largest enterprises; it's not just something rare that occurs once to some random company out in the outer edges of the Internet. Just because airplanes are more likely to crash than you are likely to be sabotaged doesn't mean you shouldn't prepare for it, either.
Just how prevalent, and important, are mobile applications for commerce today?