The 60 Second SaaS Vendor Assessment

You can outsource the function but not the risk, and that’s why we (should) all do SaaS vendor risk assessments.

Vendor assessments can of course be time consuming affairs, both for the assessed and the assessor (you don’t wanna know how many times I spell checked both of those words).

Before I go any further, I should clarify — there is no substitute for doing an actual, proper, risk assessment where you look at policies, audits and all of those things, once you’re 99% sure that you’re going to sign up with a given vendor. The method described here, is more of a quick and dirty screening process that can help you pick up on any potential red flags to ask about before you get anywhere close to that point — saving everyone time. I’ve been using it for a while, and yes, it really only does take about 60 seconds.

It’s a mixture of technical indicators and assumptions made on them based on organizational psychology, so not perfect by any means, but usually pretty accurate — in my experience. So, with disclaimers out of the way, let’s go:

Go to the vendors marketing website, do they have a dedicated section about security practices, or mention their security practices in line?

Pretty easy to understand why this is important — organizations with security programs should be proud of them and willing to discuss them alongside any other elements of their product or offering.

The presence of a dedicated security section on a marketing site typically shows that there is some degree of mutual trust between security/engineering, and marketing teams — which has positive incident response implications as well.

Is there a security.txt file on the marketing site?

One thing you don’t want to be doing for longer than needed during an incident, or when trying to report a discovered vulnerability, is spending time trying to find a way to contact the vendor’s security team. Security.txt is a standard designed to address that, by placing contact information in a defined location, in a plain text file on the domain.

Security.txt files take around 5 seconds to deploy, and can save hours of frustration. So really, they are a bit of a no-brainer for any size security team. Lack of one could indicate a company that doesn’t want their security team to be reachable by outsiders, which is somewhat concerning.

Do some DNS lookups — DNS is a source of a great deal of information.

DNS TXT records are really powerful open source intelligence sources for finding out about the make up of an organization.

I like to use the MX Toolbox SuperTool on TXT lookup mode against the main domain for a company, this’ll tell you the following:

1. Does the vendor have an SPF record set up, the bare minimum DNS-based email security standard? If so, is it a giant SPF record with many entries that suggest if you do business with them, your data might end up being shared with a lot of different sub-processors?
2. Does that SPF record end in “-all” (aka hard fail) or “~all” (aka soft fail)? If soft fail, it could suggest that the security team doesn’t have enough influence to convince engineering, IT or marketing to pull the trigger and set a hard limit on where email for the domain is coming from. Or worse, they simply might not know.
3. How many service/domain verification TXT records are there? Are they all for reputable services/vendors that you’d have no problem doing business with yourself? You can find out a lot about a vendor’s vendors using TXT records.
4. Compare the list of vendors from the TXT records with the sub-processors section in the vendor’s public facing privacy policy, if they have one. Are there organizations in here that would definitely be considered sub-processors that aren’t in the policy? If so, what does that suggest? At the very least, some sort of disconnect between policy and reality.
5. Are there multiple verification TXT records for the same vendor? If so, that suggests there may be some disconnected purchasing, or shadow IT going on, if the same org has verified the same domain with the same vendor multiple times.

Check for a DMARC policy, bonus points if that policy is fully fleshed out

While you’re using the MX Toolbox SuperTool, flip the tool to ‘DMARC Lookup’ and run it against the domain. DMARC tells you a lot about an organizations security posture, including:

1. The presence of a DMARC record of any kind suggests a general interest in going above and beyond for DNS-based email security measures. That’s good.
2. If there is a dedicated DMARC reporting email listed in the record (the ‘rua=’ parameter) , it shows that the vendor is thinking enough about user and employee phishing using their domain to actually have the reports sent there.
3. If the DMARC record is in quarantine mode, it suggests someone has spent time actually reviewing the DMARC reports and is confident/has enough influence to fully enable the DMARC policy.

Search for the company name and the word ‘security’ on LinkedIn

See how many people work in dedicated security roles at the company. Of course, there is no wrong or right answer here, and everything is very dependent on the provider, but you’d hope that by now, for the majority of SaaS companies over a certain size there would be at least one person dedicated to security.

LinkedIn can also tell you a lot about how long folks stay in security roles at the company. If there are lots of short tenured security people, especially in leadership positions, that could suggest a lack of influence, heightened frustration, or burnout. All very worrying things to drill into.

Does the vendor support SSO at all pricing tiers, or just the higher end?

SSO.tax famously explains why this is important, better than I can, but the presence of SSO support on all tiers doesn’t just suggest a company that isn’t about to put a price on a basic security control.

Anyone who has ever been on the other side of the equation knows how important SSO is — and should really be advocating for it from their own company’s offering. It doesn’t have to be free, but it should be in reach. If this isn’t the case, then again — it can indicate a disconnect between the product teams and the security teams, and those are two groups you really want talking to each other.

Check out the certificate transparency logs for the domain

Head on over to crt.sh and have a look at the cert transparency logs for a given domain. Cert transparency logs can shed more light on vendors in use by your prospective vendor, and also provide some clues about infrastructure hosting the SaaS product that you might want to remember for later.

At this point your 60 seconds is up — but you’ve probably learnt quite a bit already. Now you can use you’re time assessing the vendor more effectively than if you’d just shown up on a Zoom call to hear some polished pitch that may be lacking in some of the critical details!