-
Notifications
You must be signed in to change notification settings - Fork 917
[FEAT] Add Gaussian Blur to RawImage #1103
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
BritishWerewolf
wants to merge
5
commits into
huggingface:main
Choose a base branch
from
BritishWerewolf:add-gaussian-blur
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
11396c0
WIP. Create first draft of Gaussian BLur algorithm.
BritishWerewolf becfb05
Move small channel loop into third level instead.
BritishWerewolf d239023
Reduce the complexity of the from O(kernelSize^2) to O(kernelSize * 2).
BritishWerewolf aa50275
Replace each loop with a chunk of Promises.
BritishWerewolf dcd33cf
Replace a fixed chunk value with the hardware concurrency value.
BritishWerewolf File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -651,6 +651,126 @@ export class RawImage { | |
} | ||
} | ||
|
||
/** | ||
* Generates a 1D Gaussian kernel. | ||
* @param {number} kernelSize - Kernel size (must be odd). | ||
* @param {number} sigma - Standard deviation of the Gaussian. | ||
* @returns {Array<number>} - The 1D Gaussian kernel. | ||
*/ | ||
generateGaussianKernel(kernelSize, sigma) { | ||
// Kernel must be odd because each pixel must sit evenly in the middle. | ||
if (kernelSize % 2 === 0) { | ||
throw new Error('Kernel size must be odd.'); | ||
} | ||
|
||
const kernel = []; | ||
const center = Math.floor(kernelSize / 2); | ||
const sigma2 = sigma * sigma; | ||
let sum = 0; | ||
|
||
for (let i = 0; i < kernelSize; i++) { | ||
const x = i - center; | ||
const value = Math.exp(-(x * x) / (2 * sigma2)); | ||
kernel[i] = value; | ||
sum += value; | ||
} | ||
|
||
// Normalize the kernel | ||
for (let i = 0; i < kernelSize; i++) { | ||
kernel[i] /= sum; | ||
} | ||
|
||
return kernel; | ||
} | ||
|
||
/** | ||
* Performs a Gaussian blur on the image. | ||
* @param {number} kernelSize - Kernel size (must be odd). | ||
* @param {number} sigma - Standard deviation of the Gaussian. | ||
* @returns {Promise<RawImage>} - The blurred image. | ||
*/ | ||
async gaussianBlur(kernelSize = 3, sigma = 1) { | ||
const kernel = this.generateGaussianKernel(kernelSize, sigma); | ||
const halfSize = Math.floor(kernelSize / 2); | ||
|
||
const width = this.width; | ||
const height = this.height; | ||
const channels = this.channels; | ||
|
||
// Rather than checking an entire grid of elements, we can instead do | ||
// two separate passes with a 1d array (rather than 2d). | ||
// Consider a 3x3 kernel, instead of calculating each pixel 9 times, we | ||
// can instead calculate two sets of 3 values and then combine them. | ||
const horizontalPass = new Float32Array(this.data.length); | ||
const verticalPass = new Uint8ClampedArray(this.data.length); | ||
|
||
const numChunks = navigator.hardwareConcurrency || 4; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This was originally a hard coded value, passed into the function, however this approach makes more sense. |
||
const chunkHeight = Math.ceil(height / numChunks); | ||
|
||
// Process horizontal pass in chunks. | ||
const horizontalPassPromises = Array.from({ length: numChunks }, (_, chunkIndex) => { | ||
return new Promise(resolve => { | ||
const startY = chunkIndex * chunkHeight; | ||
const endY = Math.min(startY + chunkHeight, height); | ||
|
||
for (let y = startY; y < endY; y++) { | ||
for (let x = 0; x < width; x++) { | ||
for (let c = 0; c < channels; c++) { | ||
let sum = 0; | ||
|
||
for (let kx = -halfSize; kx <= halfSize; kx++) { | ||
const pixelX = Math.min(Math.max(x + kx, 0), width - 1); | ||
const dataIndex = (((y * width) + pixelX) * channels) + c; | ||
const kernelValue = kernel[kx + halfSize]; | ||
sum += this.data[dataIndex] * kernelValue; | ||
} | ||
|
||
const outputIndex = (((y * width) + x) * channels) + c; | ||
horizontalPass[outputIndex] = sum; | ||
} | ||
} | ||
} | ||
resolve(); | ||
}); | ||
}); | ||
|
||
// Wait for all horizontal chunks to complete. | ||
await Promise.all(horizontalPassPromises); | ||
|
||
// Process vertical pass in chunks. | ||
const verticalPassPromises = Array.from({ length: numChunks }, (_, chunkIndex) => { | ||
return new Promise(resolve => { | ||
const startY = chunkIndex * chunkHeight; | ||
const endY = Math.min(startY + chunkHeight, height); | ||
|
||
for (let y = startY; y < endY; y++) { | ||
for (let x = 0; x < width; x++) { | ||
for (let c = 0; c < channels; c++) { | ||
let sum = 0; | ||
|
||
for (let ky = -halfSize; ky <= halfSize; ky++) { | ||
const pixelY = Math.min(Math.max(y + ky, 0), height - 1); | ||
const dataIndex = (((pixelY * width) + x) * channels) + c; | ||
const kernelValue = kernel[ky + halfSize]; | ||
sum += horizontalPass[dataIndex] * kernelValue; | ||
} | ||
|
||
const outputIndex = (((y * width) + x) * channels) + c; | ||
verticalPass[outputIndex] = sum; | ||
} | ||
} | ||
} | ||
resolve(); | ||
}); | ||
}); | ||
|
||
// Wait for all vertical chunks to complete. | ||
await Promise.all(verticalPassPromises); | ||
|
||
this.data = verticalPass; | ||
return this; | ||
} | ||
|
||
async toBlob(type = 'image/png', quality = 1) { | ||
if (!IS_BROWSER_OR_WEBWORKER) { | ||
throw new Error('toBlob() is only supported in browser environments.') | ||
|
@@ -699,7 +819,7 @@ export class RawImage { | |
/** | ||
* Split this image into individual bands. This method returns an array of individual image bands from an image. | ||
* For example, splitting an "RGB" image creates three new images each containing a copy of one of the original bands (red, green, blue). | ||
* | ||
* | ||
* Inspired by PIL's `Image.split()` [function](https://pillow.readthedocs.io/en/latest/reference/Image.html#PIL.Image.Image.split). | ||
* @returns {RawImage[]} An array containing bands. | ||
*/ | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason for this, is because a kernel is essentially a square that is applied to every pixel in an image.
Consider a grid of 5x5, it would look like this, with our pixel (
P
) in the middle.Here, it is clear to see that there is an even spacing around the pixel.
Now, consider what would happen if we used an even kernel size of
4
.The pixel does not sit neatly in the middle of the kernel and this is a problem.