BlackIce, an open-source, containerized toolkit created to expedite AI security testing and Red Teaming, has been formally released by Databricks. BlackIce, which was first presented at CAMLIS Red 2025, tackles the fragmentation and configuration issues that security researchers frequently encounter when assessing Machine Learning (ML) and Large Language Models (LLMs). Databricks seeks to offer a solution similar to "Kali Linux," but specifically designed for the AI threat landscape, by combining 14 popular open-source security tools into a single, repeatable environment.

Significant practical challenges in the current AI security ecosystem are the driving force behind BlackIce. Red teamers frequently encounter “dependency hell,” where different evaluation tools require conflicting libraries or Python versions.

Moreover, managed notebooks frequently limit users to a single Python interpreter, which makes it challenging to coordinate intricate, multi-tool testing procedures. By providing a version-pinned Docker image, BlackIce lessens these problems. To guarantee stability, the architecture separates tools into two groups.

To preserve independent dependencies, static tools—which are assessed through command-line interfaces—are installed in separate Python virtual environments or Node.js projects. Dynamic tools, which allow for advanced Python-based customization and attack code development, are installed in a global Python environment with carefully managed requirement files. This structure allows researchers to bypass setup hassles and focus immediately on vulnerability assessment. Integrated Toolset and Capabilities The toolkit consolidates a diverse array of tools spanning Responsible AI, security testing, and adversarial ML.

These tools can be used from a shell or within a Databricks notebook and are accessible via a single command-line interface. Prominent tools like Microsoft's PyRIT, NVIDIA's Garak, and Meta's CyberSecEval are included in the first release.

Table 1: Tool Organization Category for BlackIce Integrated Tool Inventory GitHub Stars (Approx) LM Evaluation Harness Eleuther AI Assessment 10.3K PromptfoPromptfoo LLM Evaluation 8.6K CleverHans Lab Adversarial Machine Learning 6.4K Garak Vulnerability Scanning for NVIDIA 6.1K Adversarial Robustness of ART IBM 5.6K Giskard AI Testing 4.9K CybersecurityEvaluation of Meta Safety 3.8K PyRIT Red Teaming with Microsoft 2.9K EasyEdit Editing the ZJUNLP Model 2.6K Promptmap N/A Quick Injection 1K AI Fuzzy Fuzzing CyberArk 800 Fickling Bit Trail Pickle Safety 560 Dreadnode Rigging LLM Communication 380 Judges The quotient AI Assessment 290 To ensure the toolkit meets enterprise security standards, Databricks has mapped the capabilities of BlackIce to established risk frameworks, specifically MITRE ATLAS and the Databricks AI Security Framework (DASF).

This mapping verifies that the toolkit addresses important threat vectors, such as supply chain vulnerabilities and prompt injection. Table 2: Capability of Risk Framework Mapping Prompt Injection/Jailbreak MITRE ATLAS Reference DASF Reference AML.AML, T0051 (Prompt Injection).Jailbreak (T0054) 9.12 LLM jailbreak, 9.1 Prompt inject AML Indirect Prompt Injection.Indirect Injection (T0051) 9.9 Control of input resources AML.T0057 (Data Leakage) LLM Data Leakage 10.6 Output of sensitive data AML.T0062 (Discover Hallucinations) Hallucination Detection 9.8 LLM delusions AML.T0015 (Evade Model), AML.T0043 (Craft Data) Adversarial Evasion (CV/ML) 10.5 Attacks using black boxes AML for supply chain safety.Supply Chain Compromise (T0010) 7.3 Vulnerabilities in the ML supply chain Databricks has made the BlackIce image available publicly on Docker Hub.

To guarantee smooth interaction with Databricks Model Serving endpoints right out of the box, the toolkit comes with custom patches. Security experts can use the tag databricksruntime/blackice:17.3-LTS to pull the most recent Long Term Support (LTS) version. Users can use Databricks Container Services to set up their compute clusters to point to this image URL for integration into Databricks workspaces, allowing for instantaneous orchestration of AI security assessments.

LinkedIn and X can provide daily cybersecurity updates. To have your stories featured, get in touch with us.