Introduction

Sentinel is an advanced secret scanner meticulously designed to prevent the leakage of sensitive information such as API keys, credentials, passwords, and other confidential data into public repositories, production environments, or any unintended locations. Built with security in mind, Sentinel empowers developers, security engineers, and DevOps teams to maintain the integrity of their projects by identifying potential vulnerabilities before they become exploits.

In today's fast-paced development landscape, where code is shared across teams and integrated into CI/CD pipelines, the risk of accidental secret exposure is ever-present. Sentinel addresses this by combining powerful detection mechanisms including regular expressions (regex), entropy analysis, and heuristic pattern recognition. This multi-layered approach ensures comprehensive coverage, catching even the most subtle leaks that might evade simpler tools.

Whether you're scanning a local project directory, integrating into your automated workflows, or customizing rules for specific providers, Sentinel offers flexibility without compromising on performance. It's 100% JavaScript-based, making it lightweight and easy to integrate into Node.js environments. With support for various output formats like terminal logs, JSON, and HTML reports, Sentinel provides actionable insights tailored to your needs.

This documentation serves as a complete guide to understanding, installing, and utilizing Sentinel effectively. We delve into each aspect with detailed explanations, code examples, and best practices to ensure you can leverage its full potential in securing your codebase.

Features

Sentinel stands out with its robust set of features, each engineered to provide thorough secret detection while maintaining efficiency and ease of use. Below is a detailed breakdown of its core capabilities:

• Regex Scanner: Utilizes a comprehensive set of predefined regular expressions to detect common secret patterns from providers like AWS, GitHub, Stripe, Google Cloud, Slack, and more. This scanner is highly accurate for known formats, ensuring quick identification of standard credentials.
• Entropy Scanner: Analyzes strings for high entropy levels, which are indicative of randomly generated secrets such as passwords or API tokens. By calculating Shannon entropy, it flags potential leaks that don't match fixed patterns but exhibit randomness.
• Heuristic Scanner: Employs intelligent keyword detection and contextual analysis to identify suspicious terms like "password", "secret_key", or "token" in code, configurations, or logs. This adds a layer of protection against non-standard or obfuscated secrets.
• Multi-Format Reporting: Generates reports in terminal output for immediate feedback, JSON for programmatic parsing, and HTML for visually appealing, shareable summaries. Reports include details like file paths, line numbers, secret types, and matched content.
• Fast and Deep Scan Modes: Choose between --fast for regex-only scans in time-sensitive scenarios or --deep for full-spectrum analysis including entropy and heuristics, ideal for thorough audits.
• Extensibility with Custom Rules: Easily add or modify detection rules via JSON files, allowing adaptation to organization-specific secrets or emerging threats without altering the core codebase.
• CI/CD Pipeline Integration: Seamlessly integrates with tools like GitHub Actions, GitLab CI, and Jenkins. Run scans as part of your build process to enforce security gates and prevent merges with exposed secrets.
• Deduplication of Findings: Intelligently removes duplicate detections based on file, line, and content, ensuring clean, non-redundant reports that focus on unique issues.
• Directory Ignoring: Exclude non-relevant paths like node_modules, dist, or .git to optimize scan times and reduce false positives from vendor code or binaries.

These features are built on a foundation of performance optimization, making Sentinel suitable for large repositories without significant overhead. In practice, this means faster scans in CI/CD without sacrificing detection accuracy, ultimately fostering a culture of proactive security.

Installation

Installing Sentinel is straightforward, whether you prefer a global CLI tool for quick access or a local setup for development and customization. We recommend Node.js version 18 or higher for compatibility. Below are detailed steps for each method:

Global Installation (Recommended for CLI Use)

This method installs Sentinel as a global npm package, allowing you to run it from anywhere via the command line.

npm install -g @0xlayout/sentinel-security

After installation, verify by running sentinel --version. This should display the current version, confirming successful setup.

Local Installation (For Development or Customization)

Clone the repository and install dependencies locally if you plan to contribute, modify rules, or integrate deeply into your projects.

git clone https://github.com/0xlayout/sentinel.git
cd sentinel
npm install
npm link

The npm link command creates a symlink, enabling global CLI access while pointing to your local copy. This is ideal for testing changes.

Best Practices for Installation

• Always exclude directories like .git, node_modules, dist, and build in your scans to avoid unnecessary processing.
• If encountering permission issues during global install, use sudo or configure npm to install globally without root privileges.
• For containerized environments (e.g., Docker), add Sentinel to your Dockerfile for reproducible security checks.

Usage

Sentinel's CLI is intuitive and powerful, with options to tailor scans to your specific needs. All commands require a target directory as the first argument. Here's an in-depth guide to using Sentinel effectively:

Basic Commands

Start with a simple scan to get familiar:

sentinel /path/to/your/project

This performs a default scan (regex-based) and outputs results to the terminal.

Scan Modes

• Fast Scan: Quick regex-only detection for rapid checks.

sentinel /path/to/project --fast

Copy
• Deep Scan: Comprehensive analysis including entropy and heuristics for maximum coverage.

sentinel /path/to/project --deep

Copy

Advanced Options

Customize your scans with these flags:

Example: Deep Scan with Reports and Ignores

For a complete audit:

sentinel /path/to/project --deep --json --html --ignore node_modules dist test

This command runs a deep scan, generates both JSON and HTML reports, and skips the specified directories to focus on source code.

Architecture

Sentinel's architecture is modular and extensible, designed for scalability and maintainability. It follows a pipeline-based approach where data flows through distinct stages, each handling a specific aspect of the scanning process. This design allows for easy addition of new scanners or reporters without disrupting existing functionality.

Core Components

1. File Loader: Recursively traverses the target directory, loading files while respecting ignore patterns. It filters out binary files, large assets, or excluded paths to ensure efficiency. This component uses Node.js's fs module for file I/O, reading content line-by-line to handle large files without memory overload.
2. Scanners: The heart of Sentinel, comprising three independent modules:
   • RegexScanner: Loads rules from JSON files in src/scanners/providerRules/ and applies them to each line of code. Rules are compiled into RegExp objects for fast matching, supporting global and case-insensitive flags where appropriate.
   • EntropyScanner: Implements Shannon entropy calculation on strings exceeding a configurable length threshold. The formula used is H = -sum(p_i * log2(p_i)), where p_i is the probability of each character. Strings with entropy above 3.5 (adjustable) are flagged as potential secrets.
   • HeuristicScanner: Scans for keywords in context, using a scoring system based on proximity to assignment operators (e.g., =) or string literals. It avoids false positives by ignoring comments or quoted documentation strings.
3. Deduplication Engine: Processes raw findings, using a hash-based approach (e.g., MD5 of file+line+match) to eliminate duplicates. This ensures reports are concise, especially in repos with repeated code snippets.
4. Reporting Module: Aggregates results and formats them accordingly. For JSON, it uses JSON.stringify with pretty-printing; for HTML, it generates a static page with tables, CSS for severity (e.g., red for high-risk), and summary statistics like total findings and scan duration.

The entire process is asynchronous where possible, leveraging Promises to handle I/O-bound operations efficiently. In deep mode, scanners run in parallel using Promise.all to minimize latency. This architecture not only ensures high performance but also facilitates testing, as each component can be unit-tested independently.

Repository Structure Overview

• bin/: Contains the CLI entry point (sentinel), which parses arguments using a library like Commander.js.
• docs/: Houses usage examples, advanced configurations, and API references for developers extending Sentinel.
• src/: Core source code, including scanner utilities, entropy calculators, regex helpers, and rule loaders.
• package.json: Defines dependencies, scripts for building/testing, and CLI bin configuration.

Custom Rules

One of Sentinel's strengths is its extensibility. You can define custom detection rules to tailor the scanner to your unique environment, such as proprietary API keys or internal credential formats. Rules are stored as JSON files in the src/scanners/providerRules/ directory and are automatically loaded during scans.

Creating a Custom Rule File

Each rule file is a JSON object with an array of rules. Here's an extensive example demonstrating various rule types:

{
  "rules": [
    {
      "name": "Custom API Key",
      "pattern": "custom_[a-zA-Z0-9]{20,40}",
      "description": "Detects custom API keys starting with 'custom_' followed by 20-40 alphanumeric characters.",
      "severity": "high",
      "examples": ["custom_abc123def456ghi789"]
    },
    {
      "name": "Internal Password Pattern",
      "pattern": "(?i)internal_pass:[a-zA-Z0-9!@#$%]{12,}",
      "description": "Case-insensitive detection of internal passwords prefixed with 'internal_pass:'.",
      "severity": "medium",
      "examples": ["internal_pass:Secure!123456"]
    },
    {
      "name": "Database Connection String",
      "pattern": "db:\/\/[a-zA-Z0-9]+:[a-zA-Z0-9]+@[a-zA-Z0-9]+\.[a-z]{2,3}",
      "description": "Matches simple database URIs like 'db://user:pass@host.com'.",
      "severity": "critical",
      "examples": ["db://admin:secret@localhost.com"]
    }
  ]
}

Save this as a new file, e.g., custom.json, in the rules directory. Sentinel will compile the pattern into a RegExp during initialization.

Best Practices for Custom Rules

• Use specific patterns to minimize false positives; test rules against sample code before deployment.
• Include description and severity fields for better reporting context.
• For complex patterns, leverage regex features like lookaheads ((?=...)) or groups.
• Regularly update rules to cover new secret formats from evolving services.

Custom rules are only active in --deep mode, ensuring they integrate seamlessly with built-in scanners.

Reporting

Sentinel's reporting system is designed for clarity and usability, providing multiple formats to suit different workflows. Reports include metadata like scan mode, duration, total files scanned, and findings categorized by type and severity.

Terminal Output

Default format for interactive use, color-coded for quick scanning (e.g., red for critical findings).

Example excerpt:

[HIGH] AWS_SECRET_KEY in src/config.js:12 - AKIA************

JSON Report

Ideal for automation, parsing, or integration with issue trackers. Generated with --json.

[
  {
    "file": "src/config.js",
    "line": 12,
    "type": "AWS_SECRET_KEY",
    "match": "AKIA************",
    "severity": "high",
    "scanner": "regex"
  },
  {
    "file": "env/prod.env",
    "line": 5,
    "type": "HIGH_ENTROPY",
    "match": "xYzAbC123!@#RandomString",
    "severity": "medium",
    "scanner": "entropy",
    "entropy_score": 4.2
  }
]

This array structure allows easy filtering or aggregation using tools like jq.

HTML Report

Visual report with tables, sortable columns, and color highlights. Includes a summary dashboard with stats. Generated with --html, it's static and shareable via email or web servers.

CI/CD Integration

Integrating Sentinel into your CI/CD pipelines enforces security as code, preventing merges with exposed secrets. It's lightweight, so it adds minimal build time. Below is a detailed example for GitHub Actions, with adaptations for other platforms.

GitHub Actions Workflow

Add this YAML to .github/workflows/sentinel-scan.yml:

name: Sentinel Security Scan

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  scan:
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        
      - name: Set up Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '18'
          
      - name: Install dependencies
        run: npm ci
        
      - name: Run Sentinel Deep Scan
        run: npx @0xlayout/sentinel-security . --deep --json --html --ignore node_modules dist
        
      - name: Upload Scan Report as Artifact
        uses: actions/upload-artifact@v3
        with:
          name: sentinel-report
          path: |
            sentinel_report.json
            sentinel_report.html
          
      - name: Fail if Critical Findings
        run: |
          if grep -q '"severity": "critical"' sentinel_report.json; then
            echo "Critical secrets detected! Failing build."
            exit 1
          fi

This workflow runs on pushes and PRs, installs Sentinel via npx (no global install needed), performs a deep scan, uploads reports, and optionally fails the build on critical findings using grep.

Adaptations for Other Platforms

• GitLab CI: Use image: node:18 in .gitlab-ci.yml, add stages for checkout, install, and scan.
• Jenkins: In your Jenkinsfile, use sh 'npx @0xlayout/sentinel-security . --deep' within a Node.js tool block.
• Best Practices: Set thresholds for failure (e.g., >0 high-severity), archive reports for auditing, and notify teams via Slack/email integrations.

Contributing

Sentinel is an open-source project, and contributions are highly encouraged to enhance its capabilities. Whether you're adding new rules, improving performance, fixing bugs, or updating documentation, your input helps make Sentinel better for everyone.

How to Contribute

1. Fork the Repository: Create your own copy on GitHub and clone it locally.
2. Set Up Development Environment: Run npm install to get dependencies.
3. Make Changes: Focus on areas like:
   • Adding/improving rules in src/scanners/providerRules/.
   • Enhancing scanners (e.g., better entropy thresholds or new heuristics).
   • Optimizing file loading for very large repos.
   • Adding tests using Jest or similar frameworks.
   • Improving docs with more examples or visuals.
4. Test Your Changes: Use npm test if tests exist, or manually scan sample repos.
5. Commit and Push: Use descriptive commit messages, e.g., "feat: add custom rule for Azure keys".
6. Open a Pull Request: Describe your changes, reference issues, and provide before/after examples.

We follow the Contributor Covenant Code of Conduct. For security vulnerabilities, refer to SECURITY.md for responsible disclosure.

License

Sentinel is released under the MIT License, granting you broad freedoms to use, modify, and distribute the software. This permissive license encourages adoption while requiring preservation of the copyright notice and license text in derivatives.

Full License Text

MIT License

Copyright (c) 2025 0xlayout

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
            

By using Sentinel, you agree to these terms. For questions, contact the maintainer via GitHub issues.