Poor secrets management has long been a challenge in the IT industry, especially with compromised credentials and secrets regularly ranking among the top factors involved in security incidents in reports from leaders, such as IBM.
However, with the advent of Cloud-native ecosystems and DevSecOps practices, the sprawl of poor secrets management has only been exacerbated. Organizations that produce studies on the topic, such as GitGuardian, have found that there are over 6 million exposed secrets in public GitHub repositories alone in 2021, twice as many as the year prior.
Most of the issues with poor secrets management and credential exposure can be traced back to poor practices of inventorying secrets, centralizing organizational secrets management approaches, and a lack of awareness among staff. With the introduction of containers and Infrastructure-as-Code (IaC), coupled with application code, the problem is only accelerating.
The Phases of Your Game Plan
With this problem being so pervasive, organizations need to be prepared for not if, but when a credential leak or secret exposure occurs. They need to have established processes in place and even exercise them in advance so that they are able to respond and mitigate the impact to not only themselves but also potentially their customers and business partners. These practices tie back to having a broader organizational Incident Response Plan (IRP).
Much like other IR activities, it will involve a plan that occurs in phases, such as Preparation, Identification, Containment, Eradication, and Recovery. It will also involve capturing lessons learned and bolstering existing secrets management practices to avoid the situation from happening again and also improving secret management IR activities that may have not gone smoothly during the previous incident.
Preparation
During preparation, you will want to identify the team members involved in the activities and the escalation paths associated with responding. You’ll also want to ensure that sufficient logging mechanisms are in place to support your IR activities and identify whether the compromised credentials are or have been used. This will help give insight into the malicious actors’ activities.
Identification
In the identification phase, you need to determine the initial method by which the secret or credentials was compromised. This may involve humans and social engineering methods or technical activities to find exposed secrets.
When you’ve identified the method used to lead to the compromise, you are now armed with information to look for other victims as well. This could involve information from other staff or technical telemetries, such as system and access logs. By using these logs, you’re able to see what the credentials were used to do and what other systems, data, or users may be impacted.
Containment
Through containment, you’ll begin to mitigate the impact and isolate its reach from your broader organization and infrastructure. You’re able to revoke the compromised credentials and secrets so that they can no longer be abused and either temporarily or permanently revoke the credentials, tokens, or secrets associated with the activities. Systems involved in the activities can be identified and isolated to avoid any further impact across your organization.
Recovery
From these steps, your processes involve preserving evidence associated with the compromise. They also involve potentially rebuilding the systems and infrastructure which may have been involved and may not be compromised. Organizations are able to restore—from existing system images, backups, snapshots, and other methods—back to when the systems hadn’t been compromised quite yet and are in a “known good” state.
Organizations need to be prepared to notify customers and partners if the compromise puts their data or systems at risk. This is increasingly common with either sensitive data or customers if you’re operating as a Managed Service Provider (MSP) or Cloud Service Provider (CSP).
Improving Your Secrets Management
Organizations can also capture lessons learned from the incident to improve both their secrets management and their ability to respond to future incidents.
This is, of course, a gross simplification and overview of a complex process, especially depending on the nature of your infrastructure and the size of the organization and compromise. That said, organizations should definitely turn to guidance, such as NIST’s 800-61 Computer Security Incident Handling Guide, as well as guidance from industry leaders, such as SANS, to really formalize and bolster their IR capabilities.
Want more cybersecurity insights? Visit the Cybersecurity channel:
