-
-
Notifications
You must be signed in to change notification settings - Fork 633
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BBOT Fails in Docker-Based Integration Due to Daemonic Process Limitation #2354
Comments
@AnshSinghal thanks for the report. We haven't run into this issue before. Can you post your steps to reproduce? |
Thanks for your response! Here are the exact steps to reproduce the issue in a Docker-based environment: 1. Environment Details:
2. Steps to Reproduce:Step 1: Create a DockerfileFROM python:3.11-slim-bullseye
# Environment variables
ENV PROJECT_PATH=/opt/deploy/bbot
ENV USER=bbot-user
ENV HOME=${PROJECT_PATH}
ENV BBOT_HOME=${PROJECT_PATH}
# Create a non-root user
RUN useradd -ms /bin/bash ${USER}
# Install system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential libssl-dev libffi-dev cargo openssl \
libpq-dev curl unzip git make bash tar p7zip-full p7zip && \
apt-get clean && apt-get autoremove -y && \
rm -rf /var/lib/apt/lists/* /tmp/* /usr/share/doc/* /usr/share/man/*
# Upgrade pip and install Python packages
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir quart hypercorn bbot
# Pre-install BBOT dependencies
RUN bbot --install-all-deps -y
# Set up project directory
WORKDIR ${PROJECT_PATH}
# Copy application files
COPY --chown=${USER}:${USER} app.py entrypoint.sh ./
# Make scripts executable
RUN chmod u+x entrypoint.sh app.py
# Expose port
EXPOSE 5000
# Entrypoint
ENTRYPOINT ["./entrypoint.sh"] Step 2: Create
|
Ah I see. It seems like the main issue is that Quart's processes are daemonized. Normally, BBOT's main process isn't a daemon so this isn't an issue. But of course BBOT needs to spawn processes in order to work properly. We regularly use BBOT inside docker. We also have tests that run BBOT scans from inside a fastapi endpoint similar to what you have above. So I'm pretty confident this is specific to Quart. |
The simplest solution would be to call BBOT's CLI (e.g. with EDIT: I guess you've already tried that. Maybe try EDIT 2: It's hypercorn. It's a known bug and is about to get fixed. |
Here’s your response in a concise and structured manner: Thanks for your response! Firstly, I realized I had shared the wrong import asyncio
from bbot.scanner import Scanner
from hypercorn.asyncio import serve
from hypercorn.config import Config
from quart import Quart, jsonify, request
app = Quart(__name__)
@app.route("/run", methods=["POST"])
async def run_scan():
data = await request.get_json()
target = data.get("target")
presets = data.get("presets", ["web-basic"])
modules = data.get("modules", ["httpx"])
if not target:
return jsonify({"error": "No target provided"}), 400
# Pass a configuration override to disable problematic modules
scanner = Scanner(
target,
modules=modules,
presets=presets,
output_modules=["json"],
config={
"modules": {
"ffuf_shortnames": {"enabled": False},
"filedownload": {"enabled": False}
}
}
)
results = []
async for event in scanner.async_start():
results.append(event)
return {"results": results}
if __name__ == "__main__":
config = Config()
config.bind = ["0.0.0.0:5000"]
asyncio.run(serve(app, config)) I originally tried running BBOT inside Flask, but I encountered another issue—Flask would terminate the process without any errors. My assumption is that Flask’s synchronous request handling struggles with the async execution of BBOT’s scanner, leading to process termination before the scan completes. Regarding using subprocess or BBOT’s CLI directly, I explored that as well. However, in IntelOwl, analyzers need to return structured JSON responses with detailed scan results. Since BBOT's CLI runs as a separate process, capturing and handling its output properly inside IntelOwl would require extra layers of parsing, error handling, and state management. Would you be able to point me to specific code examples of BBOT being run inside Flask? If there’s a way to make it work, I’d be happy to test that approach. I checked the issue that you provided and it asks me to use Uvicorn. |
There shouldn't be anything special required to run BBOT inside flask. That's definitely a strange error so if you can post the steps to reproduce, I'll take a look. On another note, I saw you excluded a couple modules. Are they misbehaving? If so we should make issues for them so they can get fixed. |
Hey! Thanks for your response. above are the files to reproduce. Quart is not working which is also the same issue in #191 |
Also @TheTechromancer I think there is some problem with the module iis_shortname
|
Hi,
I’m trying to integrate BBOT as a Docker-based analyzer for IntelOwl, but I’ve been running into persistent issues due to Python’s multiprocessing restrictions. Specifically, I keep encountering the error:
This occurs because BBOT internally spawns child processes while running within a daemonized process. Since Python does not allow daemonic processes to create children, the scan fails when BBOT attempts to launch its internal modules.
What I’ve Tried:
I’ve attempted several workarounds to resolve this issue, including:
Monkey-Patching Multiprocessing:
multiprocessing.Process.__init__
to forcedaemon=False
.Switching to
billiard
(Celery’s Fork of Multiprocessing):billiard
, but the issue persists.Using Quart Instead of Flask:
Changing Python Versions:
Manually Installing BBOT Dependencies:
gcc
,openssl
, etc.) and pre-installed BBOT dependencies usingbbot --install-all-deps -y
.Forcing
multiprocessing.set_start_method("spawn", force=True)
:Request for Help
Since modifying BBOT’s internal multiprocessing behavior is not an option from my side, I wanted to check:
Any guidance or potential solutions would be greatly appreciated! If needed, I’d be happy to contribute to any changes that could help make BBOT more container-friendly.
You can check my workarounds on this PR
The text was updated successfully, but these errors were encountered: