Skip to content

Add support for environment proxies (HTTP, HTTPS, SOCKS) and improve Node.js proxy compatibility - #46

Open
amirziveh wants to merge 1 commit into
zcaceres:mainfrom
amirziveh:main
Open

Add support for environment proxies (HTTP, HTTPS, SOCKS) and improve Node.js proxy compatibility#46
amirziveh wants to merge 1 commit into
zcaceres:mainfrom
amirziveh:main

Conversation

@amirziveh

Copy link
Copy Markdown

This PR implements comprehensive proxy support for the fetch server, allowing it to respect standard environment variables and providing better compatibility across different runtimes (Bun and Node.js).

❌ The Problem
Previously, the server had limited proxy support:

Lack of Environment Support: The server ignored standard proxy environment variables (HTTP_PROXY, HTTPS_PROXY, ALL_PROXY), forcing users to manually pass the --proxy flag for every request.
Node.js Compatibility: The implementation relied on the proxy option in the fetch API. While this works in Bun, it is silently ignored by the native fetch implementation in Node.js, making proxies non-functional when running on Node.
Strict URL Requirements: The --proxy flag required a full URL including the protocol (e.g., http://...). Providing just an IP and port (e.g., 127.0.0.1:10808) resulted in "Invalid URL" errors.
Stream Incompatibility: Using node-fetch to solve the Node.js proxy issue introduced a crash in readResponseText because node-fetch returns Node.js Readable streams instead of the Web Streams expected by the existing code.
✅ The Solution
Environment Proxy Detection:
Implemented a new getProxyUrl helper that automatically detects and prioritizes proxy settings from HTTPS_PROXY, HTTP_PROXY, and ALL_PROXY (and their lowercase variants).
Multi-Protocol Support:
Integrated https-proxy-agent and socks-proxy-agent to provide robust support for HTTP, HTTPS, and SOCKS proxies.
Cross-Runtime Fetch Logic:
Bun: Continues to use the native proxy option for maximum performance.
Node.js: Now uses node-fetch combined with the appropriate proxy agent to ensure the proxy is correctly applied.
Smart URL Resolution:
Added logic to automatically prepend http:// to proxy strings that lack a protocol, allowing users to provide simple host:port strings.
Unified Stream Handling:
Updated readResponseText to detect the type of response body. It now supports both Web Streams (via getReader()) and Node.js Readable streams (via for await...of), ensuring stability across all environments.
🛠️ Changes
Modified src/Fetcher.ts to implement the new proxy resolution and fetch logic.
Added node-fetch, https-proxy-agent, and socks-proxy-agent as dependencies.
Verified that all existing tests pass and the CLI works correctly with both environment variables and the --proxy flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant