Given the request to build a standalone Python-based color clustering algorithm for Rookery, I put together a project proposal.
Project Overview: Based on our discussion, the focus would be on developing a standalone Python module for clustering colors in images of artwork. The larger goal is to identify color patterns across a collection while ensuring the code is well-structured and thoroughly documented.
Scope of Work:
Develop a Python-based module for color clustering in artwork images.
Implement efficient algorithms to detect and analyze color patterns.
Prioritize clear, well-structured, and readable documentation.
Deliver a functional module within an estimated 4-6 week timeframe.
We agreed on deliverables and compensation, then signed a contract and NDA.
At Bluebird's request, I started with the k-means algorithm for clustering. I experimented with using the mean, median, and mode of different methods like Silhouette, Davies-Bouldin, Elbow, and Calinski-Harabasz.
During testing I found that Rookery's sample user, Robin Murphy, had a largely black & white collection. I had Bluebird pull from a larger cache of American Cyborg images to create a more robust dataset.
I started with bar charts, then tested different methods of data visualization.
Some initial results:
A wireframe for the front end:
I began testing Hue-Saturation-Value as an analytic framework, in addition to Red-Green-Blue.
I got the code running on a Render server, and published my first module: Color Engine Phase 1. This completed our initial contract.
Having completed Color Engine Phase 1, I scoped out Color Engine Phase 2:
Week 1-2: Research & Experimentation
- Explore techniques for identifying standout colors within an image.
- Test alternative color spaces (e.g., LAB, HSV) to see if they improve clustering
results.
- Try different dimensionality reduction techniques beyond PCA to compare
effectiveness.
Week 2-4: Algorithm Development & Testing
- Refine the clustering approach based on initial experiments.
- Develop an algorithm that enhances color separation and dominance detection.
- Run tests on various images to evaluate how well the method captures key colors.
Week 4-6: Integration & Website Update
- Finalize the method and integrate it into the color engine.
- Optimize performance and document findings.
- Update the website with results and any interactive features.
My findings in the Hue-Saturation-Value space are very promising:
Minutes from our meeting to survey development:
Image Processing with Masks Phoenix explained their updated image processing approach using separate masks based on both saturation and value, addressing limitations of a single mask. They described creating two masks to capture high value/low saturation and low value/high saturation elements.
Dynamic Mask Adjustment Phoenix detailed how the saturation mask is applied based on the percentage of the original image's saturation, categorizing it into low, optimal, and high saturation ratios. Based on these categories, they automatically adjust the minimum value of the saturation mask.
Palette Generation Phoenix described generating color palettes using the original image and the two masks (saturation and value). They utilized a Python library called 'palette' which employs a faster version of k-means clustering.
Palette Refinement For the saturation and value masks, Phoenix initially set a large palette size of 250 colors but the original palette yielded 15. They then removed black colors originating from the masks and subsequently paired down the remaining colors based on their Euclidean distance.
Color Thresholding and Palette Combination Phoenix used trial and error to determine thresholds for how close colors could be before being paired down. They combined the palettes from the saturation and value masks first, using thresholds that were slightly adjusted for low and high saturation images. To cover more bases, the original image's palette was then added to the combined palette.
Enforcing Color Diversity Phoenix implemented helper functions to ensure more color diversity in the final palettes, even metaphorically "hammering" in a specific pink color that wasn't consistently appearing. Bluebird was impressed with the algorithm's ability to extract colors even from challenging images.
Explanation of Masks Bluebird requested a further explanation of what the masks do. Phoenix clarified that the value mask isolates bright areas, while the saturation mask identifies highly saturated colors. Their reasoning was that bright or very saturated colors typically draw the eye.
Benefits of Distance Filtering Phoenix explained that distance filtering was implemented because there can be overlap in colors identified by the value and saturation masks. This filtering step helps to avoid duplicate colors or colors that are too similar after the initial combination.
Masks for Non-Dominant Colors Phoenix noted early in the process that masks would be crucial for extracting colors that are not necessarily dominant in the image. There were instances where the value mask yielded no results, requiring a fallback to just the saturation mask. High saturation also presented similar difficulties to low saturation in terms of filtering.
Future Steps: API Development Phoenix stated that their next step is to build the API to display the two palettes side-by-side as requested and make small front-end adjustments.
Future Directions: "Node-Network" Idea Bluebird introduced the concept of finding connections between artworks based on their component colors and vocabulary, moving beyond physical room groupings. Phoenix suggested that either the paintings or the colors themselves could serve as nodes in such a network. Bluebird mentioned that this is a new, more experimental direction compared to the existing pattern language.
Exploring Artist-Specific Palettes Bluebird proposed analyzing a collection of artworks by a single artist to identify their characteristic color palette. They mentioned local artists (Dana Sherwood, Michael McGrath) whose colorful work might yield interesting results. Phoenix expressed curiosity about how their algorithm would handle these paintings.
Developing an Appreciation for Art Phoenix shared that working on the coding project involving art and colors has led them to a newfound appreciation for patterns and art itself, despite previously considering themself solely logic-based.
API Development Plans Phoenix outlined their plans for the API, including starting from scratch and passing a JSON script from the backend to the front end to render colors client-side. This approach aims to improve speed and allow for local caching of images. They also suggested adding a download button for example images.
Interest in Network Concepts Phoenix expressed excitement about working with network concepts, an area of coding they haven't professionally explored. Bluebird mentioned Principal Component Analysis as a related area of potential interest. Phoenix stated that working with Rookery has taught them they can tackle unfamiliar tasks and succeed.
Work on color network map
Work on Login Screen Phases 1 and 2
