Independence versus conflict of interest in security reviews

I was giving a lecture to some soon to be graduating folks today, and at the end of the class, a student came up and said that he wasn’t allowed to work with auditors because “it was a conflict of interest”.

No, it’s not. And here’s why.

Conflict of interest

It’s only conflict of interest if a developer who wrote the code then reviews the code and declares it free of bugs (or indeed, is honest and declares it full of bugs). In either case, it’s self review, which is a conflict of interest.

The only way the auditor is in a conflict of interest is if the auditor reviews code they wrote. This is self-review.

An interesting corner case that requires further thought is a rare case indeed: I wrote or participated in a bunch of OWASP standards, some of which became bits of other standards and certifications, such as PCI DSS section 6.5. Am I self-reviewing if I reviewing code or applications against these standards?

I think not, because I am not changing the standard to suit the review (which is an independence issue),  and I’m not reviewing the standard (which would be self-review, which is a conflict of interest).

Despite this, for independance reasons where I think an independence in appearance issue might arise, I always put in that I created or had a hand in these various standards. Especially, if I recommend a client to use a standard that I had a hand in.

Independence

Independence is not just an empty word, it’s a promise to our clients that we are not tied to any particular vendor or methodology. We can’t be a trusted advisor if we’re a shill for a third party or are willing to compromise the integrity of review at a customer’s request.

The usual definition of independence is two pronged:

• Independence in appearance.  If someone thinks that you might be compromised because of affiliations or financial interest, such as if you’ve previously held a job. For example, if you’ve always worked in incident response tool vendors, and then move to a security consultancy, you might feel you are independent, but others might perceive you as having a pro-IR toolset point of view.
• Independance in actuality. If you own shares or get a financial reward for selling a product, any recommendation you make about recommending that product is suspect. This is the reason I want OWASP to remain vendor neutral.

If either of these prongs are violated, you are not independent. But just as humans are complex, there are many aspects to independence, and I’ve not learnt them all. I know most of the issues, been there got the t-shirt.

If you are independent, there are a few areas of independence that I refuse to relinquish (and I hope you do too!):

• Scoping questions. I don’t mind customers setting a scope, but I will often argue for the correct scope before we start. Too narrow a scope can railroad a review into giving an answer the client wants, rather than a proper independent review.
• Review performance independence. If the client tries to make the review so short that I can’t complete my normal review program in an effective manner or they stop me from getting the information I need, or tries to frame negative observations in a lesser or “meh” context, I will resist. I want to ensure that the review is accurate, but not at the expense of performing my methodology or being up to my usual standards.
• Risk ratings and findings. In the last few years, I’ve had to resist folks trying to force real findings to become unrated opportunities for improvement (by definition all of my findings are such), or trying to reduce risk ratings by arguing with me, or getting words changed to suit a desired outcome. Again, I want the context to be accurate and I will listen to your input / arguments, but only I will write the report and set risk ratings. Otherwise, why bother hiring an external reviewer? You could write your own report set the ratings to suit. It doesn’t work that way.

Does independence always need to be achieved?

My personal view on this has changed over the years. I used to toe the strict party line on independence.

However, sometimes as a reviewer, you can be part of the solution. I personally believe that a good working relationship between the reviewer and the folks who produced the application or code is a good thing. Both parties can learn from working closely together, work out the best approach to resolving issues, and test it rapidly.

As long as the self-review aspect is properly managed, I believe this to be a good path forward. I don’t see this as being any different as a traditional review that recommends fix X, and then reviewing that X has been put into place. However, if the auditor is editing code, that has crossed the line.

Work with the business to document agreed rules of engagement early prior to work commencing. Both parties will get a lot more mileage from a closely cooperative review than a “pay someone to sit in the corner for two weeks” that is the standard fare of our industry.

Conclusion

Working together has obvious independance issues, particularly from an appearance point of view. So to the excellent question from a student today, working tightly with the auditors is not a conflict of interest, but it can be an independence issue if not properly managed by both the business and the reviewer.