Stories from the Front Lines of Security Leadership: Accommodating the gray areas of information security

For Jonathan Kamens, CISO at the Boston financial technology startup Quantopian, balancing security and innovation means realizing that you can’t immediately fix every security issue. While initially uncomfortable with this idea, Kamens, over the course of his IT and security career, realized that information security is not black and white.

In part one of his interview, Kamens, who became Quantopian’s first CISO in January after working as the vice president of operations for more than four years, talks about how he accommodated the gray areas of security.

Does Quantopian face any unique information security challenges as a startup?

One of the most important roles for a successful startup is to move fast. And maintaining both the quality and the security of our application when we're shipping new code multiple times per week is definitely a challenge. We use a lot of strategies for mitigating the risk. The two that are most worth mentioning are seniority and code reviews.

By seniority, I mean that we're not the kind of startup whose entire engineering staff is green engineers who are right out of college. We aren't that old, but we do have quite a few extremely experienced, talented engineers. And those senior engineers are always paying attention to what's going on across the organization. That happens organically; it's not something we make them do. They're curious, and they want to stay informed. They frequently will spot potential security issues in our development processes early because they're paying attention.

Every line of code, pretty much, that goes into our platform, goes through a code review process before it gets shipped. That means at least two people have looked at the code seriously and thought about any potential security implications before we ship it. When we are shipping something that's explicitly security-relevant to our platform, we try to have another person do a code review, so that the original author and two code reviewers have looked at the code, line-by-line, before it gets shipped.

How can you balance innovation and security when you have to move quickly?

As a day-to-day reality we have to acknowledge that not all security issues are alike. There are minor security issues and there are major ones. We always fix the major issues very quickly. Someone reports a security issue to us, and we look at that, and you go, “Holy crap! That's significant.” That we'll fix within a day.

Sometimes the minor security issues need to be prioritized and fixed later because we don't have the resources to fix them right away. We'll take steps to mitigate them if it's necessary to do so. I have a security queue of the issues that I would like an engineer to knock off when we have resources to allocate to the security backlog. That's just the reality when you innovate quickly.

Starting earlier in my career when I perceived security as more black and white, I had a lot more difficulty with this idea of recognizing that you can't fix everything right away. You have to balance risk against resources against what opportunity costs. What are you not implementing, in terms of innovation, if you're taking time to go back and fix the security hole?

Quantopian CISO Jonathan Kamens

As I've gotten older and more experienced, I've gotten better at prioritization. Yes, it's my job to advocate for security. But if I were to advocate to management that every issue I put a security tag on in our bug database has to be fixed within a week or something ridiculous like that, I would lose my standing to demand resources when really significant issues that need to be fixed right away came along. The trick is not to waste your capital on the stuff that really can wait.

As a result of that, when some security issue comes across my desk and I decide this can't wait, we get it fixed right away because my coworkers trust me not to cry wolf when it isn't really necessary. I can interrupt engineers and get them to focus on what I need them to focus on.

How do you accommodate the gray areas of security?

The ideal would be that I have a team of engineers whose job it is to fix everything that I decide needs to be fixed, the day I want them to fix it.

I believe our platform is secure enough. I do not believe our platform is secure as it could be. Therefore, there are things that I would like to do on our platform to make it more secure. The question is is it secure enough to protect it against the things that might compromise it without doing those things? I believe the answer to that is yes, and that's what I think my job is. My job is not to make our information security and our platform as secure as humanly possible and throw in every possible security bell and whistle that we can, damn the cost. That's not my job. My job is to weigh the cost against the benefit, figure out what the minimum viable security product is and get that into the application.

That is challenging for me because I'm a security wonk, and I love that half percent increase in security that we might get from doing one particular thing. I'm going well, “It's a half percent increase in security, so we should do it because it's an increase.”

There are so many other things that we need to do to be a successful company. We could do something that will give us a 20 percent increase in some really significant metric about the success of our application. It’s recognizing that those business requirements and business needs need to come first. It doesn't do any good for the application to be secure if no one's using it and the company goes bankrupt and closes. Those tradeoffs have to be made, and they have to be made in a realistic way.

You absolutely need to make a decision as CISO in collaboration with the rest of the people on the management team and with individual engineers. I don't make decisions in isolation. I'm talking to everybody here all the time. What can we do to have the level of security we need while impacting people as minimally as possible? It's an ongoing tradeoff. If it’s not the biggest, it’s one of the biggest responsibilities of the CISO.

Why did you transition to the role of CISO?

When I was the vice president of operations, I was responsible for every aspect of keeping our platform up and running, including the security. I've worked in information security in one way or another for most of my career. I've always been passionate about both operations and security. It's not like this was a new thing for me. As we grew larger and identified the path we were going to take to profitability, it became clear that we needed someone to be entirely focused on information security.

My deep knowledge of our platform and our current security posture combined with my security knowledge and passion made me the obvious choice for that. Therefore, I was promoted to CISO. Then we promoted someone else from within to take over as VP of operations. He focuses more on the operations side of it, and I focus on the security side of it, which, by the way, is not to say that our new VP of operations doesn't think about security. His primary focus is on site reliability and operations. It's not like security can live in a silo with only your CISO and the people who work in security.

At what point does a company decide that a CISO is needed?

I won't pretend to have enough experience with that tipping point to be able to answer the general question. This is the first time I've crossed that boundary. I will, however, tell you how it went down here. It was a combination of two different things: the fact that there was a regulatory requirement for having a CISO and the fact that we just needed more energy to be paid attention to security combined to make the choice pretty obvious.

Many startups decide the way to make money is with ads. You end up with ads on Facebook, ads on Twitter, ads on Snapchat. Ads weren't going to do it for Quantopian. Eventually, we settled on being a fund where we could charge people a fee to use our platform.

When it became clear that this was the path to profitability, this idea of being a fund, it became clear that there needed to be a CISO because, in fact, that's a regulatory requirement for funds. Someone needed to have that title, and it would preferably be someone who was actually focused on that. That need was part of the impetus for us to convert me over to CISO.

The other impetus was as we got bigger, we're a bigger target. There's more to be gained by someone compromising our platform now than there was four and a half, five years ago. I think yes, there's an organic aspect of it where eventually you realize we are not devoting enough resources to keeping the platform secure. We need to have somebody whose job it is to be paying attention to this more than anybody else and whose job it is to evangelize and advocate throughout the company for security so that other people can't just forget to think about security.

The Cybereason series Stories from the Front Lines of Security Leadership presents insights from CISOs, security leaders and IT executives on topics including what’s required to succeed as a security executive, how to convey the importance of security to an organization and how security leaders can advance their careers. Do you know a security executive who has great insights and would like to talk with us for this series? Email us at ciso.series@cybereason.com.