GSoC Idea Page

##More image plugins

OIIO uses a plugin system to implement input/output of individual image file formats. It would be great for somebody to write additional plugins to expand the range of formats we allow. High-priority formats include GIF, XBM/XPM, DICOM, various camera RAW formats. Other format suggestions would be welcome, too -- tell us what we're missing that would be helpful.

Volunteers need not implement all of these. Doing just one important/hard one, or 2 medium difficulty ones would be a great summer project. Also, we may accept more than one student working on format plugins, if they are working on different ones.

Hint: don't submit a proposal that cuts and pastes from the previous paragraphs. If you want to write plugins, decide exactly which ones and justify your choice -- WHY that format is strategically important to OIIO, who uses it, what is involved in supporting it well, what is your strategy for implementation. Be prepared to discuss the tricky parts (example: if you want to write a plugin for a palette-only format, how are you going to decimate the color space when outputting a "full-color" buffer?).

Requirements: C++, knowledge of (or willingness to learn) the picayune details of one or more image file formats.

##Additional image algorithms and oiiotool functionality

We have a command-line utility called "oiiotool" (and, internally, a set of image operations under the ImageBufAlgo namespace, that operate on ImageBuf objects) that can do a bunch of image processing operations in a scriptable way. Currently, it's just the basics -- file and data format conversion, changing metadata, rotating or resizing, cropping, color space conversion, that kind of thing. But it only scratches the surface of what the tool could grow into.

There's no limit to what can and should be added to this tool. Just as a few examples: compositing operations (over, in, out, etc.), text captioning, various intensity adjustments (contrast stretching, color remapping, histogram equalization), image repair (median filtering, de-noising), edge detection, image enhancement, etc.

Applicants should be prepared to propose a specific set of operations, with justifications for selecting them as well as a thorough understanding of the implementation details. Be sensitive when proposing algorithms -- we'll prefer proposals that make a solid but incremental and sensible step forward for OIIO and that are likely to be widely useful (especially for OIIO's active user base of VFX studios). Proposing to implement a fancy algorithm from your thesis, that nobody has heard of and has no obvious VFX use, is in many ways less interesting to us than something simple but undeniably useful, such as a blazingly fast super high quality resize operation.

I like to explain that OIIO (like any software) is like a jigsaw puzzle in the process of being put together. You want to add a piece or two. It's better to find and add a piece that attaches directly to the clump we have already assembled than to start working on a part far away from where the rest of us are working.

Requirements: C++, knowledge of image processing and sampling.

##iv viewer thumbnail view

Enhancement to the iv viewer to add a thumbnail view window for managing multiple images simultaneously. We want a separate window that shows thumbnails of the loaded images, selection of a subset of loaded images from thumbnails, and extension of various existing operations to apply to multiple selected images.

Requirements: C++, Qt.

They key to the thumbnail view project is going to be a few details, I think:

1. The graphical design -- Is it a separate window? a sidebar? a togglable view in the main window? What resolution should the thumbnails appear? Should they be scalable? Should there be a visual cue to indicate that the image has multiple subimages or MIPmap levels? Or other important things about the image?

2. Functionality -- Exactly what operations are allowed on the thumbnail icons themselves? Can you reorder the image stack by manipulating the thumbnails? Can you select more than one thumbnail at a time, and if so, what operations that we do in iv make sense to operate on multiple images at once?

3. The data and I/O management problem -- some image formats have embedded thumbnails, but what if they are not the resolution of our thumbnail display? What do you do with a 4096 x 4096 image that has no embedded thumbnail - does it need to read the whole thing in? Is there a strategy for generating the thumbnail view that doesn't cause iv to pause for while reading all the images? Should there be a "placeholder" for thumbnails not easily available? Should there be a separate thread to generate unavailable thumbnails in the background without holding everything else up? Should thumbnails be cached on disk somewhere so big thumbnail-less files don't need to regenerate them every time, and if so, how will we know when it's out of date or needs to be reclaimed?

A good proposal would lay out a comprehensive design for how it all should work, with justifications for the design decisions and a concrete plan of attack.

##iv speed overhaul

Our iv viewer is not the peppiest thing around. It could use some extra love to find out in what ways it is less responsive speed-wise than we would prefer, and then address its optimization. We want it to be super fast and responsive.

Before making this proposal, you should use iv a bit and have some intelligent things to say about where it excels, where it lags, and what you intend to do about it.

Requirements: C++, Qt, OpenGL/GLSL.

##iv playback

Enhancement to the iv viewer to allow an image sequence to be played back at a reliable speed. It doesn't (and shouldn't) be a full-featured movie player -- that's not really iv's design goal -- but often it is used to view a sequence of stills and so should have some basic abilities to play them at speed when they represent all the frames of a shot.

This differs from iv's existing "slide mode" (which simply cycles through displaying loaded images with n seconds delay between subsequent images), which will surely be made obsolete by this project. A better playback mode will need to be a correctly-timed, capable of frame accurate playback at 24 or 30 fps for 2048-res images, and will have "movie player-like" controls (play, stop, forward, back, advance frame, etc.). It differs from most other movie players (such as VLC) by having all of iv's other properties and features, such as HDR display, OpenColorIO color management, pan, zoom, stop adjustments, channel selection, pixel view, etc.

This will be a challenging project, don't underestimate the challenges of making a large stack of big images play smoothly at a defined frame rate without hiccups.

Requirements: C++, Qt, maybe some OpenGL/GLSL.

##iv color profiles/LUTs and more color management Implement professional quality color reproduction for the iv viewer. Sub-projects may include:

• Integration of OpenColorIO into iv (we currently rely on it for color conversion in oiiotool, but haven't yet made it part of the viewer).
• More smarts with image formats that store their color primaries.
• Use of ICC profiles to tune to the specific display.
• Tone mapping (one or more algorithms) for decent representation of high-dynamic-range (HDR) images on standard displays.

Requirements: C++, some Qt, some OpenGL/GLSL, knowledge of the principles of color science.

##Caching for writable ImageBuf's

Our ImageBuf class currently is very good a using the ImageCache underneath for ImageBuf's that are only being read. In that way, an application can open thousands of images totaling hundreds of GB to multi-TB data sets while using a very small in-memory footprint and without the application itself needing to know anything about how the caching works, which files are actually currently open or which tiles reside in memory, etc.

But any time you write into an ImageBuf, the whole thing has to reside in memory. It would be great to restructure ImageBuf/ImageCache so that it was possible to also write huge images without their needing to stay resident in RAM.

They key to this is a detailed proposal that foresees all the tricky design decisions: where do the tiles reside when paged out? how will they be cleaned up after exit? after crash? how to avoid collisions among multiple OIIO application instances running simultaneously on the same machine? how to tune performance? can it be done without a significant change to the ImageBuf API? can it be done without severely degrading the performance of images small enough to fit in main memory?

Requirement: C++, strong algorithmic skills

##Ptex images and texturing

We have a preliminary Ptex reader, but it's very rough. It could use a lot more work, tests, a working Ptex writer, extensions to handle correct texturing with filtering across the face boundaries, etc.

If you don't know what Ptex is, this project may not be for you. But if you do, and would like OIIO to better support it, this would be a great project.

Requrements: C++, knowledge of texture mapping, familiarity with Ptex

##Shadow mapping

The TextureSystem component of OpenImageIO does great 2D texture sampling and directional environment mapping. There are API calls that imply that it should be able to do shadow map lookups, but this is currently unimplemented. It would be great for somebody to make that work so that renderers using OIIO could count on it for shadow mapping.

Requirement: C++, knowledge of computer graphics and shadow mapping techniques

It probably helps if you have a candidate renderer to test it out with, to show that it can make the right pictures with the OIIO shadow lookups.

##Procedural and REST-ful images

An ImageInput doesn't need to read a file -- it can come up with any method it likes to generate the pixels. There are all sorts of handy procedural images that could be useful: ramps, Perlin noise, etc. One wrinkle on this is that we may need to pass information to the ImageInput about such things as resolution requested. We've discussed support for "REST-ful" names like looking up texture from an image named "perlin?res=4096" and the perlin plugin would know how to decode this into a request for a particular maximum resolution.

Requirement: C++, good imagination

##maketx memory efficiency for huge images

"maketx" is our tool to turn image files into tiled, multiresolution texture maps.

It would be nice to have maketx able to convert truly huge ("gigapixel") images while keeping a reasonable memory footprint. Currently, it needs to hold at least the top level of the mipmap entirely in memory. Any speed enhancements of maketx would also be welcome.

Requirements: C++, some knowledge/interest of texture mapping issues.

##maketx conversion of different environment map formats

Conversion of various environment map formats: it would be nice to have modes where a variety of environment map formats (spherical, light probes, etc.) could be converted into either latlong or cube map.

Requirements: C++, some knowledge/interest of texture mapping issues.

##OpenCV integration

OpenCV seems very popular these days. OpenCV and OIIO have very different goals, neither is a substitute for the other. But can/should they work better together? Is it helpful/possible to make an OpenCV plugin that uses OIIO to support all the formats that OIIO is capable of handling? Or is there some "pipe fitting" code that would make it easier for applications using both OpenCV and OIIO to shuttle the data back and forth?

OK, I'm just making stuff up at this point. I don't know if there's a useful project here or not. But perhaps a student who is familiar with both OpenCV and OIIO will see an opportunity for some kind of collaboration or integration and pitch a project that creates a helpful bridge between the two packages.

Requirements: C++, lots of experience with OpenCV.

##Android or iOS ports

Sure, why not? I don't know if this is even remotely feasible. But it might be a fun thing to have working. It may be nothing more than an OIIO-API wrapper around the native libraries for those platforms, but even that might be very helpful as a way to make apps that are more portable among the various mobile and desktop systems.

##Your idea

Go ahead and suggest something else. Also, there's a much more extensive list of smaller and lower-priority tasks that can be found in our ticket system.