Intro
Developing software inevitably involves dealing with secrets: passwords, API keys, access tokens, connection strings, and certificates. Keeping those secrets out of your application is critical, especially when that software is deployed anywhere outside the developer's local machine.
Dealing with secrets requires special care. The number one rule is to never commit secrets to source control, especially when working on a project in a public repository on GitHub or another DevOps platform. Still, developers do it all the time, and hackers need only minutes to discover leaked secrets.
What can you do to prevent secret leaks or at least minimize their impact? Use automated tools that can detect secret leaks after, or preferably even before they occur. This step-by-step guide will walk you through implementing secrets detection as early as in the IDE using SonarQube for IDE with SonarQube Cloud and SonarQube Server, two widely used static code analysis solutions for keeping your code base Clean and free of issues.
What Is Secrets Detection?
Secrets detection is the process of identifying sensitive information that has been or is about to be exposed in your codebase, both in source code files written in various programming languages and in language-agnostic config files or even Infrastructure as Code (IaC). This sensitive information, called secrets, includes API keys, passwords, cryptographic keys, database connection strings, and more. Secrets detection tools scan the codebase for patterns or formats commonly associated with sensitive data and flag them for human review. Secrets detection is a subset of static code analysis, which is used to scan for other issues in code that lead to security vulnerabilities, bugs, and accumulation of technical debt.
Detecting and flagging secrets that are not adequately secured helps prevent their exposure to malicious actors, who may use the leaked secret to gain unauthorized access to critical parts of your product's infrastructure and secure data. Detecting exposure in an application in production and even when the code containing the secret has been committed to a repository is often too late requiring costly work to "rotate" or change the key or password so that it can no longer be exploited. Leaving it up to developers to manually prevent secrets from entering your code leads to leaks for several reasons. Developers might not be clear on what constitutes a secret, or protecting the secrets in a safe way might be too complicated or unknown by the developer. Sonar's solutions aid developers as they work within their normal workflow by identifying secrets and guiding them through issue resolution before becoming a problem.
Implementing Secrets Detection with Sonar
In two stages of the Software Development Life Cycle (SDLC), Sonar's static analysis solutions detect many common secret formats in many different languages at the start of development, in the IDE, and as a second layer of protection in your source code repository:
- SonarQube for IDE detects secrets in your code editor in the IDE and warns you before you accidentally commit them to source control. This kind of prevention is ideal because if a secret doesn't make it into a common code repository, you don't need to take any remediation steps. Your problem has been solved before it has leaked into the CI/CD pipeline.
- If a secret has managed to make its way into source control, SonarQube Cloud and SonarQube Server can detect it by running an analysis on every commit. Thus, the leak will be revealed early, giving your team the time required to remediate by removing, purging, or invalidating the secret.
SonarQube for IDE, SonarQube Cloud, and SonarQube Server each provide more than ninety code analysis rules aimed at detecting leaked secrets out of the box. These rules cover over 80 cloud services including AWS credentials, Azure account keys, Google Cloud service accounts, Facebook app keys, GitHub and GitLab tokens, SSH private keys, PostgreSQL and MySQL database passwords, Slack and Zapier webhook URLs, and many more commonly used secrets.
Secrets Detection in the IDE
First, let's go through how to detect and resolve discovered secrets with SonarQube for IDE in the IDE before they're leaked. SonarQube for IDE is an IDE plugin that extends the capability of SonarQube Cloud and SonarQube Server to the IDE by helping developers maintain Clean Code while writing code before it is committed to a shared repository.
SonarQube for IDE integrates into popular code editors: Visual Studio Code (a.k.a. VS Code), JetBrains IDEs, Visual Studio, and Eclipse. For example, if you're using VS Code, you can install SonarQube for IDE: Visual Studio Code. Open VS Code's Extensions pane, search for SonarQube for IDE, and install the SonarQube for IDE extension when it shows up in the search results.
SonarQube for IDE immediately starts analyzing the source code in the project you have open in VS Code. When it finds a problem, it adds a squiggle underline highlighting the problematic code in the code editor. It also adds a problem entry to the Problems view and problem-related commands to VS Code's Quick Fix menu with helpful instructions to guide you through resolution.
SonarQube for IDE can locate common secrets across a variety of programming languages. For example, if you paste a GitHub personal access token as a string into JavaScript code, SonarQube for IDE will immediately let you know that it's something you may want to reconsider:
When you work with Google APIs, you often generate and download service account credentials as JSON key files. What if, on a bad day, you make a dubious decision to add a service account credential object from that file directly into your code? Since the JSON object contains a private key, SonarQube for IDE will promptly flag the embedded private key with an underline squiggle identifying where in the code the issue was found:
Note that if you have the Problems view open in VS Code, SonarQube for IDE will add any problems that it detects, including information about detected secrets.
SonarQube for IDE's secrets detection also works with standalone files. For example, when you open a PEM file with a private key in a VS Code editor tab, SonarQube for IDE displays a squiggle over its opening line. If you press Ctrl+., you can see SonarQube for IDE's quick actions, similarly to how you'd expect to see them in a code file:
Additionally, whenever SonarQube for IDE detects that a new secret file is opened in a VS Code editor tab, it shows a pop-up notification:
When you have SonarQube for IDE installed in your IDE, be sure to pay attention to any new warnings about secrets. Follow its instructions to exclude secrets from your code before they leak into your repository.
Secrets Detection in the CI/CD Pipeline
What if a secret has made it into your code repository? You want to make sure your team gets notified about it as soon as possible. Early awareness enables a prompt response and mitigation of the threat.
The second layer Sonar offers to handle early detection of secrets in code is to use SonarQube Cloud or SonarQube Server integrated into your DevOps platform. By running static analysis checks as part of your CI/CD pipeline, you make sure that you'll be aware of a secret leak as soon as the code is committed to your repository which includes an automated analysis step.
Let's explore the timely detection of secrets entering the CI/CD pipeline using SonarQube Cloud. Go to SonarQube Cloud and choose the DevOps platform account. For this walkthrough, choose to sign up with GitHub.
In the next screen, click Authorize SonarQube Cloud. The Welcome screen will suggest that you find or import your GitHub organization:
Select Import an organization. When you're asked where to install SonarQube Cloud's GitHub app, choose your personal GitHub account instead of any GitHub organizations you may be a member of. When SonarQube Cloud requests permissions, keep the defaults and click Install.
On the Create an organization screen, follow these steps:
- Keep the suggested defaults under Import organization details.
- Under Choose a plan, select Free plan, then click **Create Organization**.
You've now set up your SonarQube Cloud account:
Next, set up analysis for one of your projects. For this demo, fork this sample GitHub repository containing example secrets.
Back in your SonarQube Cloud account, click Analyze a new project. Search for the forked project, select it, and click Set Up.
In the Set up project for Clean as You Code screen, select Previous version, then select Create project.
SonarQube Cloud will quickly scan the selected GitHub repository. Once it's done, it will display the initial analysis summary:
Under Security, choose 2 H to see the list of high-security issues that SonarQube Cloud has detected:
If you click one of the two security issues detected, you'll see a detailed view of the issue along with its description and specific lines of code that are affected:
From now on, SonarQube Cloud will run its code-quality scan on every push to this repository and report the results back to its UI. This kind of CI workflow promptly notifies you of leaked secrets so that you're better armed to address these leaks when they occur.
Detecting Company Specific Secrets
SonarQube Server, SonarQube Cloud, and SonarQube for IDE detect many publicly known types of secrets. However, your company is likely using dozens of internal private services, each with its own format of secrets. Exposing secrets from any of these private and proprietary services also poses a high-security threat. With SonarQube Server Enterprise Edition, you can cover these company-specific secrets by creating custom secret definitions and enabling them as custom rules for your SonarQube Server static code analysis.
When you have custom secrets configured in SonarQube Server Enterprise, make sure to enable Connected Mode to extend SonarQube Server's custom pattern detection into your IDE. Connected Mode pairs SonarQube for IDE with your SonarQube Server instance, extending your company customizations into the IDE and ensuring that both your local edits and the commits that you push to the repository are inspected using the same code analysis settings that are set in SonarQube Server. This means SonarQube for IDE can also use your custom secret rules to prevent custom secrets from entering the CI/CD pipeline and reduce the risk of leaking.
Summary
You have learned what secrets detection is, why it's important, and how Sonar's code-quality analysis tools help implement this practice.
With leaked secrets, time is money. The earlier you know about a secret leak, the more time you have to take remediation steps, and the less likely a malicious actor will take advantage of it. SonarQube Server and its SaaS twin, SonarQube Cloud both help you reveal secrets early in the CI/CD pipeline as they enter your repository, minimizing the impact of these incidents. But of course, preventing secrets from ever entering the repository is better than letting them happen. So if you want to step up your prevention game, don't forget to integrate SonarQube for IDE into your favorite IDE.
This guide was written by Jura Gorohovsky.
- August 7, 2024