Add detailed logging for asset download process by aviraxp · Pull Request #3 · KernelSU-Modules-Repo/modules

aviraxp · 2025-12-11T12:53:42Z

Enhanced logging for asset download and extraction, including metadata, error details, and diagnostic steps.

Copilot

Pull request overview

This PR enhances the asset download and extraction process with comprehensive diagnostic logging to aid in troubleshooting failures. The changes add detailed error reporting, metadata logging, and diagnostic steps when extraction fails.

Increases buffer size for module.prop extraction to reduce truncation issues
Adds detailed logging of asset metadata before processing (name, size, contentType, downloadUrl)
Implements extensive diagnostic error handling that attempts to download failed assets using curl, inspect zip contents, and examine file signatures for post-mortem analysis

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-11T12:57:37Z

scripts/fetch-data.ts

+
    // Extract module.prop content from zip URL (internal network, stable)
+    console.log(`Running runzip to stream module.prop from remote zip (this may fail if URL needs special headers)`);
    const { stdout: modulePropContent } = await execAsync(`runzip -p "${downloadUrl}" module.prop`, {


The command construction is vulnerable to command injection. If the downloadUrl contains special shell characters (like backticks, dollar signs, or unescaped quotes), they could be interpreted by the shell. Consider sanitizing the URL or passing it as an argument in a safer manner to prevent potential shell injection attacks.

Copilot · 2025-12-11T12:57:38Z

scripts/fetch-data.ts

+      const tokenHeader = GRAPHQL_TOKEN ? `-H "Authorization: Bearer ${GRAPHQL_TOKEN}"` : '';
+      console.warn(`Diagnostic: saving remote asset to ${tmpFile} using curl (may reveal HTML or error pages)`);
+      try {
+        const curlCmd = `curl -sSL -f ${tokenHeader} "${downloadUrl}" -o "${tmpFile}"`;


The curl command construction is vulnerable to command injection. The downloadUrl is directly interpolated into the shell command without proper escaping. If the URL contains backticks, dollar signs, or other special shell characters, they could be executed as commands. Use proper argument passing or URL validation to prevent shell injection.

Copilot · 2025-12-11T12:57:38Z

scripts/fetch-data.ts

+      // If file exists, try to list zip contents (unzip -l) and dump a head of bytes
+      if (fs.existsSync(tmpFile)) {
+        try {
+          const { stdout: listOut } = await execAsync(`unzip -l "${tmpFile}"`, { maxBuffer: 200 * 1024 });


The tmpFile path is directly interpolated into the shell command without proper escaping. Although tmpFile is constructed from a controlled path, if Date.now() or other path components can be influenced, this could lead to command injection. Use proper argument passing or ensure the path is safely escaped.

Copilot · 2025-12-11T12:57:38Z

scripts/fetch-data.ts

+          console.warn(`unzip -l failed on ${tmpFile}: ${listErr?.message || listErr}`);
+          // Try zipinfo as alternative
+          try {
+            const { stdout: zipinfoOut } = await execAsync(`zipinfo -1 "${tmpFile}"`, { maxBuffer: 200 * 1024 });


The tmpFile path is directly interpolated into the shell command without proper escaping. This is vulnerable to command injection if the path can be manipulated. Use proper argument passing or ensure the path is safely escaped.

Copilot · 2025-12-11T12:57:39Z

scripts/fetch-data.ts

+          // Try zipinfo as alternative
+          try {
+            const { stdout: zipinfoOut } = await execAsync(`zipinfo -1 "${tmpFile}"`, { maxBuffer: 200 * 1024 });
+            console.log(`zipinfo -1 output (first 200 entries):\n${zipinfoOut.split('\n').slice(0,200).join('\n')}`);


Inconsistent spacing in the slice method call. Line 642 uses 'slice(0,200)' without a space after the comma, while line 636 uses 'slice(0, 200)' with a space. Consider maintaining consistent formatting throughout the code.

Suggested change

console.log(`zipinfo -1 output (first 200 entries):\n${zipinfoOut.split('\n').slice(0,200).join('\n')}`);

console.log(`zipinfo -1 output (first 200 entries):\n${zipinfoOut.split('\n').slice(0, 200).join('\n')}`);

Copilot · 2025-12-11T12:57:39Z

scripts/fetch-data.ts

+        if (curlErr?.stdout) console.warn(`curl stdout (truncated): ${String(curlErr.stdout).slice(0,1000)}`);
+        if (curlErr?.stderr) console.warn(`curl stderr (truncated): ${String(curlErr.stderr).slice(0,1000)}`);


Inconsistent spacing in the slice method call. Line 628 uses 'slice(0,1000)' without a space after the comma, while other similar calls in the code use 'slice(0, 1000)' with a space. Consider maintaining consistent formatting throughout the code.

Suggested change

if (curlErr?.stdout) console.warn(`curl stdout (truncated): ${String(curlErr.stdout).slice(0,1000)}`);

if (curlErr?.stderr) console.warn(`curl stderr (truncated): ${String(curlErr.stderr).slice(0,1000)}`);

if (curlErr?.stdout) console.warn(`curl stdout (truncated): ${String(curlErr.stdout).slice(0, 1000)}`);

if (curlErr?.stderr) console.warn(`curl stderr (truncated): ${String(curlErr.stderr).slice(0, 1000)}`);

Copilot · 2025-12-11T12:57:39Z

scripts/fetch-data.ts

+      try {
+        console.log(`Processing asset for ${repo.name}@${node.tagName}: assetName="${zipAsset.node.name}", size=${zipAsset.node.size}, contentType=${zipAsset.node.contentType}`);
+        // Show downloadUrl partially (avoid leaking extremely long urls)
+        const shortUrl = zipAsset.node.downloadUrl ? (zipAsset.node.downloadUrl.length > 200 ? `${zipAsset.node.downloadUrl.slice(0,200)}...[truncated]` : zipAsset.node.downloadUrl) : 'N/A';


Inconsistent spacing in the slice method call. Line 755 uses 'slice(0,200)' without a space after the comma, while many other similar calls use 'slice(0, 200)' with a space. Consider maintaining consistent formatting throughout the code.

Suggested change

const shortUrl = zipAsset.node.downloadUrl ? (zipAsset.node.downloadUrl.length > 200 ? `${zipAsset.node.downloadUrl.slice(0,200)}...[truncated]` : zipAsset.node.downloadUrl) : 'N/A';

const shortUrl = zipAsset.node.downloadUrl ? (zipAsset.node.downloadUrl.length > 200 ? `${zipAsset.node.downloadUrl.slice(0, 200)}...[truncated]` : zipAsset.node.downloadUrl) : 'N/A';

Copilot · 2025-12-11T12:57:40Z

scripts/fetch-data.ts

+    try {
+      const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'diag-asset-'));
+      const tmpFile = path.join(tmpDir, `asset-${Date.now()}.zip`);
+      const tokenHeader = GRAPHQL_TOKEN ? `-H "Authorization: Bearer ${GRAPHQL_TOKEN}"` : '';


The GRAPHQL_TOKEN is exposed in the shell command string, which can leak sensitive credentials through process listings and logs. Additionally, this token header construction is vulnerable to command injection if the token contains special characters. Consider using curl's config file or netrc file for authentication instead of passing the token directly in the command line.

Copilot · 2025-12-11T12:57:40Z

scripts/fetch-data.ts

+        } catch (headErr: any) {
+          console.warn(`Failed to read head bytes of saved file: ${headErr?.message || headErr}`);
+        }
+


Temporary directories and files are created but never cleaned up. The diagnostic message at line 661 states they are kept for post-mortem analysis, but this can lead to disk space accumulation over time. Consider adding cleanup logic or documenting the cleanup procedure in the code comments.

Suggested change

// NOTE: The diagnostic asset and directory are intentionally kept for post-mortem analysis.

// To avoid disk space accumulation, periodically remove old diagnostic files and directories:

// rm -rf $(find "${os.tmpdir()}" -maxdepth 1 -type d -name 'diag-asset-*')

// Adjust the command as needed for your environment.

aviraxp · 2025-12-12T01:44:33Z

@copilot 2-12T00:25:10.6063542Z Failed to extract props from https://github.com/KernelSU-Modules-Repo/HyperUnlocked/releases/download/v2.1.2/HyperUnlocked-212-v2.1.2.zip: req.on is not a function. (In 'req.on("response", function(d) { 2025-12-12T00:25:10.6064928Z req.abort(); 2025-12-12T00:25:10.6065306Z if (!d.headers["content-length"]) 2025-12-12T00:25:10.6065778Z reject(Error("Missing content length header")); 2025-12-12T00:25:10.6066232Z else 2025-12-12T00:25:10.6066593Z resolve(d.headers["content-length"]); 2025-12-12T00:25:10.6067069Z })', 'req.on' is undefined)

Copilot · 2025-12-12T01:44:41Z

@aviraxp I've opened a new pull request, #6, to work on those changes. Once the pull request is ready, I'll request review from you.

Add detailed logging for asset download process

1c1f448

Enhanced logging for asset download and extraction, including metadata, error details, and diagnostic steps.

Copilot AI review requested due to automatic review settings December 11, 2025 12:53

Copilot started reviewing on behalf of aviraxp December 11, 2025 12:54 View session

aviraxp had a problem deploying to github-pages December 11, 2025 12:54 — with GitHub Actions Failure

Copilot AI reviewed Dec 11, 2025

View reviewed changes

Update fetch-data.ts

3800cb7

aviraxp had a problem deploying to github-pages December 11, 2025 13:04 — with GitHub Actions Failure

Copilot AI mentioned this pull request Dec 12, 2025

[WIP] Add detailed logging for asset download process #6

Closed

11 tasks

aviraxp closed this Dec 12, 2025

aviraxp deleted the aviraxp-patch-1 branch December 21, 2025 13:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add detailed logging for asset download process#3

Add detailed logging for asset download process#3
aviraxp wants to merge 2 commits intomainfrom
aviraxp-patch-1

aviraxp commented Dec 11, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 11, 2025

Uh oh!

Copilot AI Dec 11, 2025

Uh oh!

Copilot AI Dec 11, 2025

Uh oh!

Copilot AI Dec 11, 2025

Uh oh!

Copilot AI Dec 11, 2025

Uh oh!

Copilot AI Dec 11, 2025

Uh oh!

Copilot AI Dec 11, 2025

Uh oh!

Copilot AI Dec 11, 2025

Uh oh!

Copilot AI Dec 11, 2025

Uh oh!

aviraxp commented Dec 12, 2025

Uh oh!

Copilot AI commented Dec 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	console.log(`zipinfo -1 output (first 200 entries):\n${zipinfoOut.split('\n').slice(0,200).join('\n')}`);
	console.log(`zipinfo -1 output (first 200 entries):\n${zipinfoOut.split('\n').slice(0, 200).join('\n')}`);

		if (curlErr?.stdout) console.warn(`curl stdout (truncated): ${String(curlErr.stdout).slice(0,1000)}`);
		if (curlErr?.stderr) console.warn(`curl stderr (truncated): ${String(curlErr.stderr).slice(0,1000)}`);

	const shortUrl = zipAsset.node.downloadUrl ? (zipAsset.node.downloadUrl.length > 200 ? `${zipAsset.node.downloadUrl.slice(0,200)}...[truncated]` : zipAsset.node.downloadUrl) : 'N/A';
	const shortUrl = zipAsset.node.downloadUrl ? (zipAsset.node.downloadUrl.length > 200 ? `${zipAsset.node.downloadUrl.slice(0, 200)}...[truncated]` : zipAsset.node.downloadUrl) : 'N/A';

+        // NOTE: The diagnostic asset and directory are intentionally kept for post-mortem analysis.
+        // To avoid disk space accumulation, periodically remove old diagnostic files and directories:
+        //   rm -rf $(find "${os.tmpdir()}" -maxdepth 1 -type d -name 'diag-asset-*')
+        // Adjust the command as needed for your environment.

Conversation

aviraxp commented Dec 11, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

aviraxp commented Dec 12, 2025

Uh oh!

Copilot AI commented Dec 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants