How to Accurately Classify a Log in your SaaS application?
Following my previous blog post on Audit Logs for SaaS Enterprise Customers, I’ve received a bunch of questions on what’s the difference between Audit Logs and other types of Logs you would typically encounter while developing a SaaS application.
These questions brought back memories from my days as R&D manager at a cyber solution product, and of the product management discussions we used to have. Back then, while planning a certain feature for a sprint cycle, I remember that we needed to decide where a certain log would go. The question was super simple: ”Where does this log belong?” But with the VP R&D, VP Product and Solution Architect in the room, we had at least as many opinions.
In this blog I’ll use examples to explain the different logging options. But before we jump in, let’s briefly list the different types of logs we will encounter in almost any SaaS application.
3 Types of Logs in a SaaS Application
Type 1: Developer Logs
This is probably the easiest one to grasp. TL;DR: log everything that happens on the backend of your SaaS application to a developer logs storage. Categorize by the level of importance (or severity), make sure you log the context — and that’s it. You should be all set.
- Scale — High. Since you log everything here, you can expect large amounts of logs. Consider using one of the standard market solutions for this one: ELK, Logz.io, Coralogix, etc.
- Retention — Medium. You should keep the logs for as long as you would want quick access to them for matters of traceability. Developer Logs are usually kept for 3 months and then either moved to cold storage or deleted.
- Personas & Use Cases — Developers, for traceability. They would be the main users, either directly by querying the logs storage or by using the logs in predefined monitoring tools and alert systems.
- Privacy and Sensitivity — High. Business-wise, all your flows are described on these logs, so leakage could be devastating both to you and to your customer base.
- Compliance —
1. From SOC2 aspects (and similar) you cannot store PII in these logs.
2. From GDPR aspects (especially “the right to be forgotten” clause), you cannot store PII here since it will be very hard to allow the deletion of this data if the customer asks for it.
Type 2: Business Events
In this type of logs we usually refer to any kind of business-related output your product produces for your customer’s administrators or end-users. Usually, the characteristics of this log type depend on the type of product you’re managing.
A Cloud Security Permission and Identity Management product might have events like:
- Excessive permission detected for user John Doe
- John Doe has tried to act outside of his permission scope
- 10 redundant roles have been detected in your environment
An HR management product might have events like:
- Employee Jane Doe has 5 tremendous annual reviews in a row
- Jane Doe is showing signs of churn
All of these are business-value events that your customers would love to know and be alerted about, since this is usually the core value they’re paying for.
- Scale — Depending on the product. You only log insights which are interesting for your users from a business perspective
- Retention — High. Usually, you will keep these logs for a long time so you can access them if your customers need them. You might offer to extract those logs as files (Excel, CSV, PDFs, etc) or even to SIEM solutions such as Splunk, LogRhythm, QRadar, etc.
- Personas & Use Cases — Administrators and End-Users for business purposes. As mentioned in the examples above, this is the reason they pay for your product.
- Privacy and Sensitivity — High (depending on the product). Business-wise, this data is usually multi-tenant based, so leakage would expose the organization to the type of data enrichment you handle. In cases of cybersecurity or cloud solutions, for example, this could be very sensitive.
- Compliance —
1. From SOC2 aspects (and similar) PII should be handled sensitively and your customers should have control over who gets to see what. RBAC and ABAC concepts as well as granular permissions and policies should be used to make sure that only authorized personas can see the different types of data.
2. From GDPR aspects, almost all seven core principles should apply, including “Data Minimization”, “Accuracy” and others.
Type 3: Audit Logs
As described in my previous blog post, you would usually audit log everything that might be interesting to your customer’s administrators in terms of traceability of system actions and administrative manners in your product.
User John has deleted a permission for user Jane
- Developer Log—Yep. Since you should log (almost) everything as a developer log, this would definitely go as a developer log. Severity would usually be “Info”.
- Business Event— Nope. Unless your core product centers around permissions and roles then usually this would not be classified as a business event.
- Audit Log — Yep. This would usually be an administrative action, so you should log it as an audit log.
Anomaly engine has detected an anomaly on John Doe’s email usage
- Developer Log — Yep. Engine events should be developer logged so that if an error occurs in one of the engine’s decisions, the developers can trace it back.
- Business Event — Yep. This is exactly the type of information your customers pay your SaaS product for.
- Audit Log — Not exactly. This one starts to be tricky. Usually, you wouldn’t log the specific engine’s finding as an Audit Log since this is not a specific administrative event in your system. But, it’s important to note that Beginning and Ends of your product’s engines total scan cycles should be logged in an audit log. Usually, you will have two audit logs for these types of events. The first will be “Anomaly engine scan started”, and the second: “Anomaly engine scan successfully ended”.
More Interesting Classifications
|5 consecutive login attempt failures|
in any SaaS product
(Not as a
is your core value)
|Phishing email detected|
in Email Security product
|Server Error in password change|
in any SaaS product
|User has accessed the main dashboard|
in any SaaS product
|Bitcoin transfer was securely made|
in Blockchain security product
Every SaaS application requires 3 types of logs: Developer Logs, Business Events, and Audit Logs. Accurately classifying your logs is important since each one of the log types has different characteristics for retention, scale, who uses them and for what reason, and privacy & compliance aspects.
Frontegg provides a solution for Audit Logs and Business event logging (coming soon), so your product can benefit from the highest level of scale, accurate retention, exporting to SIEM solutions, privacy, and compliance — all with a push of a button.