What is Going On with our Color Standards?

So, today I was in the middle of what will hopefully be the final pass of “Warrior Poet’s Guide to Writing Plug-ins for GIMP 2.10”. It’s been an interesting journey; I had to resort to demonstrating techniques in C instead of Python, as an example. I’ve also reviewed and completely rewritten about half of the chapters. Punctuality may be important, but quality and information-congruency, and while-we’re-on-the-subject longevity, are a bit more valuable to me.

I got to the point where colors were described in GEGL, in the format of an interpreted string. The means of doing so is relatively simple—you can define any color in an RGB or RGBA color space, using a range in zero to two hundred and fifty five, a percentage, or a floating point between zero and one. Random experimentation showed me that you could even mix and match them—you could have your R component as a percentage, G as an integer value, and B as a fraction of one, and GEGL would figure it out. I honestly have no idea why anyone would do this, but it was a fun discovery.

What’s more, you can use the full CSS color constant list, right in your program. Instead of this:

GeglColor *color = gegl_color_new("rgb(0.0, 1.0, 0.0)");

You can do this:

GeglColor *color = gegl_color_new("lime");

To a lot of people, that may seem like a good thing; but for me, the hair on the back of my neck was already standing on end. The more I thought about it, the more random it seemed, “standardized” or not. To begin, why is “Green” defined as only roughly 50% of the green channel; but “Lime” designated as high-intensity green? Moreover, have any of us actually seen a lime that it objectively this color? Because I definitely haven’t, of any species, and I make black-lime teas and key-lime pies that will just melt you.

Moreover, on what hardware were these colors determined? One monitor to another, even modern models, will render things slightly differently. In the old days (early-1990s) we had it relatively simple—everybody had agreed to a resolution of 640×480 for television screens, and there were only three phosphors commonly in use for them—we had yttrium oxide-sulfide with a europium activator for red, zinc sulfide and copper for green (also known as P31), and zinc sulfide with just a hint of silver for blue (P22B). These were easy to get our hands on, they consistently emitted the same color when they were sprayed with electrons from the CRT, and people could plan on them.

It’s almost 2020 now, and we now have IPS displays, LCDs, a scattered handful of CRTs still lingering around (probably), plasma, LED displays—heck, OLED and QLED complicate it even further. These all render color in decidedly different ways; in fact, Sharp even introduced its Quattron television series that had an additional channel for yellow built into the display. (And no, there was no technical reason to do this; a study at Queen Mary University in London showed any difference to be negligible. There was no data being broadcast to drive the yellow sub-pixel!)

What’s more, the human eye is only roughly capable of differentiating between about a hundred different tones per type of cone cell (color receptor), which is 100 × 100 × 100 = 1,000,000 colors. Which tones they happen to be is subject to both variation and interpretation, and it is apparently possible to train your eye to have a “refined palette” for tonal differences.

For this reason, radiologists, as an example, will buy 8K displays with as many as twelve bits of channel sensitivity—that’s 4,096 tonal variations per color element—to prevent relevant medical data, like-I-don’t-know-cancer, from being lost in the overflow. Such displays aren’t cheap. If you’re curious, a typical monitor only has eight bits of color depth, or 256 different tones, if that, and some error on which tone is reproduced; but don’t worry, unless you’ve trained your eyes for years, you will not likely notice the difference on a 10-bit monitor.

The variation in color between displays is not a new thing; I remember as a child finding a “hue” dial on an old CRT TV set, which seemed to alter the color between red and blue. I remember it being decidedly skewed toward red, which may have been an artifact of the set. What it ultimately did was alter the strength of the aligning electromagnet inside the set, spraying more electrons toward one edge of the RGB-phosphor-chain than the other. (I spent some time playing with it and learned a great deal about how video worked even then, down to the RGB sub-pixels, using water magnification through drops from a squirt gun, much to the anger of my grandfather.)

So how did we come to this strange set of CSS colors? If they don’t even represent the same thing on every monitor, why choose arbitrary names? Where did the standard come from? Pleasantly, it’s at least an old standard. It came from the X Windowing System, initially release X10R3. If you happen to be using a Unix-based operating system on the Linux or BSD kernel, you can still find a relic of the original file, at /usr/share/X11/rgb.txt.

That was February 1986, more-or-less the dawn of color computer monitors. VGA monitors didn’t even exist until 1987, so we’re really down in the mud here. The best we had was CGA/EGA, and only 69 colors were listed at the time. The file was originally written by Jim Gettys at MIT; specifically calibrated for the VT240 monitor. It had an 80 × 24 character display, and cost just short of US$1,400. (Doing the math, that’s maybe 73¢ per character.)

We didn’t get the full traditional range of colors until around 1989, when Paul Raveling, John C. Thomas, and Jim Fulton (on file maintenance) added the standard named set. Fulton made a number of stupid-but-critical changes to the file, including removing the color “transparent” (that would have been a serious problem down the road…) and a few blank lines. He calibrated everything to his specific HP monitor.

In a sense, that answers most of my questions right there. I think we can rest assured that (0, 255, 0) looks exactly like lime on Jim Fulton’s exact monitor that he was using in 1989. On that specific monitor, it is a damned fine lime green. And no, we don’t actually have any idea what model number of HP monitor that actually was.

Most of his color names purportedly came from Sinclair paint swatches. This brings me to a new concern, though; many years ago, I worked at a paint desk at a local Home Depot. We had a fancy little marketing gadget that would match the color of the paint to any sample put under it, which sounds like it works better than it actually does. Most of the time, after formulating a sample, we would get to the real concern—was this not-quite-objective photographic match precisely what the customer had in mind?

After a while, employees developed something of an artist’s touch for adjusting the color to meet customer expectations. I was well aware even then of how arbitrary the device was, but it was importantly a place to start. Right next to it we had a display allowing people to view paint samples under fluorescent, “natural”, and “incandescent” light, which of course were all actually fluorescent bulbs which may, or may not, have a light filter wrapped around them. And no, I didn’t trust it completely, but like I said, it was a start.

Interestingly, John C. Thomas eventually took inspiration from a box of Crayola crayons, while refining the color set. Almost comically, more than half of the colors used were represented. The last real update to the X11 set was in 1994; but in 2001, the W3C decided to merge the X11 set with the CSS3 standard.

There were immediate problems here, mostly involving color-nomenclature overlap. The grays, as an example, were entirely off. I honestly have no idea how it was standardized, but evidently the W3C just decided to codify what everyone was already doing.

I suppose this post has gone on for long enough; but I’ll end it with a (slightly-desperate) plea to stop using these arbitrary color names; and also note that a lot of them came from the defunct Sinclair paints, and Crayola; not that they had any idea that they were being used as a reference. I’ll leave a half-hour-long talk and CSS Conf 2014 from Alex Sexton here, which covers a lot of the same material. His email citations are quite entertaining!

Alex Sexton, CSSConf2014

If anyone ever finds what remains of that old HP monitor, it’s probably worth millions now.

The Warrior Poet’s Guide to Writing Plugins for GIMP 2.10

After publishing The Warrior Poet’s Guide to Python and Blender 2.80, I was at a deciding point. It’s clearly not meant to be the only element in the series, but I had a lot of options then. I could throw money into marketing (well, more of it), I could break for a while and get back to one of my in-house projects, or I could start on the next book.

I’m not a person to waste time, not even when I arguably should relax; so, I decided to get started on the next book. The theme of the series is professional, proven, and most importantly available tools, which are flexible enough to use to write one’s own plug-ins and add-ons. On top of that, these books are directed at people who are ambitious and driven enough to use the information, but may not always know where to begin.

The choice was obviously the GNU Image-Manipulation Program, or GIMP. GIMP has had a resounding impact on the VFX industry. Its only realistic competitor for use is Photoshop, which is saying something—GIMP was produced entirely by volunteer engineers for projects of their own. It has objectively been a spearhead for the open-source software movement. It’s spawned various other projects out of both its own original code, and its ideals, such as the GTK+ windowing toolkit.

Moreover, unlike Blender, you can easily extend GIMP with an assortment of different languages. It has native support for C, Perl, Scheme, and Python. There’s an awful lot going on under the hood in that software. However, with GIMP being software built by volunteering professionals, many of the tutorials are frequently out-of-date. (Documentation takes time too, after all!) I will frequently come across often-cited walkthroughs on the internet, which recommend techniques which have been deprecated for nearly a decade.

I’m actually glad that the walkthroughs are still there; people who aren’t used to actual source-code diving need a place to start. However, version 2.10.14 was released not long ago, the world has changed in GIMP, and no one talks about the new features. The existing material isn’t enough. Where are the saga-long YouTube videos on GeGL tricks? Where are the WordPress pages of colorspace filters and distortion stunts?

I’m now four chapters into writing The Warrior Poet’s Guide to Building Plug-ins for GIMP 2.10, focusing on Python and C. There are still worlds of material, and associated artistry-magic, to cover here. Moreover, it’s pivotal that the material is accessible to the broadest part of my target audience, so I may have a few appendices on subjects like Python, C, and GeGL. I may even append something on basic Python to Python and Blender 2.80.

If you’re interested in writing your own plug-ins for modern GIMP, then check back to this website over the next few weeks. I’m in full science-journalism mode and hope to have the initial draft done by the end of November. After that, there are a host of different products I could take on next, from LADSPA in Audacity, to OSL.

We’ll see what comes next.

Exporting Ultra-High-BPP Gray-Scale with FreeImage

Are you trying to encode data that corresponds to a specific point? Perhaps a height map, or an MRI or CAT scan? Even a density map?

If so, you may be considering storing it in an image or video format. Of course, if you’ve tried that, you may have bumped into the sharp limit of what our monitors can represent, and for that matter, what our eyes can realistically see.

Most images have a maximum of 8-bit-channel color depth, that is, eight bits per pixel color, or 256 possible shades each, of red, green, or blue. The reason for this is that the eye does have its limit on perceivable and noise-free differences in shade. According to the Young-Helmholtz trichromatic theory¹, Our eyes have three types of cone cells responsible for detecting color, distributed across low, medium, and high wavelengths. (Overlap is why we perceive a gradual blend between colors, with intensity-detecting rod cell sensitivity somewhere in the blue range, partially explaining relative “brightness” of different shades. Hence, the rainbow!)

Each cone has a resolution of roughly 100 shades. That means that our eyes, in theory, are capable of distinguishing between 100³ or 1,000,000 different colors. For the rare tetrachrome with a fourth cone cell type, it’s possibly closer to 100,000,000. In any case, 256 shades for red, green, and blue (corresponding to low, medium, and high wavelength color) gives us around 256³ or roughly 16,700,000 possible colors. All the same, for the trained eye, monitors do exist that can render high-bit-depth color, extending into the 12-bit-per-channel range; but beware that they are not cheap.

The drive for higher bit depth usually stops with scientists; however, when we’re talking about storing data (rather than appearance) in an image, eight bits only get us two hundred and fifty six possible values. For more extreme precision, 16-bit (or even higher) can look awfully appealing.

But enough introduction. How does one export a high-bit-per-pixel single-channel image? This is fairly easy to do, and generally cross-platform, with a toolkit called FreeImage. However, like everything else, it recognizes that most of its usage comes down to displays, and it isn’t immediately obvious how to do so.

There’s a specific (and very efficient) function in that toolkit which can set a pixel color according to a provided parameter. It’s usually easy to use. The specification is thus:

DLL_API BOOL DLL_CALLCONV FreeImage_SetPixelColor(FIBITMAP *dib, unsigned x, unsigned y, RGBQUAD *value);

Where FIBITMAP is a FreeImage bit mapping, x and y are unsigned values specifying coordinate from the bottom-left, and RGBQUAD is… RGBQUAD is, ah… uh oh.

RGBQUAD is 32-bits. It contains one byte for each primary color, and one byte reserved (where you could in theory store the alpha channel or any other special value). However, that limits us to, one, color; and two, no more than eight bits per channel. So, if we’re dealing with 16-bit gray-scale, we can’t use it; and if you try to pass in a pointer to a ushort you’re asking for trouble on compilation.

To modify monochromatic data, broadly speaking, we have to edit the scan lines directly. RGBQUAD has to be purged from the code for anything high-bit-def. There is precious little data on doing this available; but it comes down to another wonderful method included in FreeImage, specifically for cases like this one.

DLL_API BYTE *DLL_CALLCONV FreeImage_GetScanLine(FIBITMAP *dib, int scanline);

A scan line is the line drawn, classically by the CRT beam in older monitors, across the width of the screen. It carried over to image and texture terminology, where alignment matches a monitor’s in most cases. So, a scan line is the range of data corresponding to one “sweep” of the screen.

This method returns a BYTE *, which many will recognize as a pointer to a specific byte in memory. If you need a pointer to, say, a ushort (16 bits), you can pretty much just cast it in most languages. So, with a little pointer arithmetic, we can access the location of every quantized color value, regardless of what bit count it’s made of, in memory!

The code that ultimately ended up working for me, in D, was this.

for(int y = 0; y < height; y++) {
	ubyte *scanline = FreeImage_GetScanLine(bitmap, y);
	for(int x = 0; x < width; x++) {
		ushort v = cast(ushort)(data[cast(ulong)((y * width + x)] * 0xFFFF);
		ubyte[2] bytes = nativeToLittleEndian(v);
		scanline[x * ushort.sizeof + 0] = bytes[0];
		scanline[x * ushort.sizeof + 1] = bytes[1];
	}
}

For each scan line in the image, we iterate over the number of pixels in its width. (You must know this ahead of time.) We calculate our value and store it in v. (data was sent to the function in a parameter, as a double[]. It’s the data to be rendered in the image, as a list of values, of dimension height × width of the image.)

v is a ushort, so it’s already two bytes and interpreted as unsigned. Of course, a ushort can’t be dropped into a byte slot, it needs twice the space, but the equivalent byte array, available through std.batmanip.nativeToLittleEndian, will do just fine. My machine is working with little-endian format, so this is all that’s needed.

I suppose you could also do it manually with a little pointer math, if you enjoy that kind of thing or don’t have an appropriate package in your environment.

We can then marshal those bytes into their equivalent positions on the scanline array. We’re done! The image can be exported to a file.

Unfortunately, you may bump into an issue where the output is upside down. This isn’t as noticeable for noise arrays, but stands out on anything with actual dimensions. To change this, you need to fill from the bottom-up instead of the top down. This is fixable with a single line of change.

ushort v = cast(ushort)(data[cast(ulong)(((height - 1) - y) * width + x)] * 0xFFFF);

By decrementing from the maximum value of y, you can reverse the order of iteration and flip your image.

PNG and TIFF are two good formats for 16-bit-depth gray-scale images. It’s hypothetically also possible with higher bit depths, but the procedure should rough be the same, and there are too many possibilities to tangle with right now. I recently even heard something about 96-bit-depth monochrome, but that’s 2⁹⁶ or ~8 × 10²⁸ values and I’m not sure we can even get that kind of precision on most measurements. (Even Mössbauer spectroscopy only goes to change detection of a few parts per 10¹¹, and last I checked, that’s the finest measurement any scientist has yet made.) I suppose it’s a comfort to know that, if we ever are that certain about anything, modeling it as an image is a possibility.

Also note that higher-than-16-bit channels are very difficult to read with most image software. I start bumping into trouble around 32-bit, and others can expect the same. So, roll your own, or come up with a better method of parsing the data, like a hex editor.

Final Code

void toFreeImagePNG(string fileName, const double width, const double height, double[] data) {
     FIBITMAP *bitmap = FreeImage_AllocateT(FIT_UINT16, cast(int)width, cast(int)height);
     for(int y = 0; y < height; y++) {
         ubyte *scanline = FreeImage_GetScanLine(bitmap, y);
         for(int x = 0; x < width; x++) {
             ushort v = cast(ushort)(data[cast(ulong)(((height - 1) - y) * width + x)] * 0xFFFF);
             ubyte[2] bytes = nativeToLittleEndian(v);
             scanline[x * ushort.sizeof + 0] = bytes[0];
             scanline[x * ushort.sizeof + 1] = bytes[1];
         }
     }
     FreeImage_Save(FIF_PNG, bitmap, fileName.toStringz);
     FreeImage_Unload(bitmap);
 }

Fun fact: The image used for this page is 16-bit, though the data doesn’t really necessitate it. (One of my generated topographies; I’ve noticed plateauing on my data, and am upping the output to 16-bit, with this, to remove it. The resolution still needs to be scaled up above 256×256, though.)

You can check this, on most Linux distros, with:

$file multarraya_fi.png
 multArrayA_FI.png: PNG image data, 256 x 256, 16-bit grayscale, non-interlaced

Happy sciencing!

¹ Kalat, James W. (2001). Biological Psychology, Seventh Edition, Bellmont, CA: Wadsworth/Thompson Learning

Webcam Sampling

Did you know that this is doable from within Blender? I added a new chapter to The Warrior-Poet’s Guide on it tonight. It shows how to access OpenCV, the binding standard of computer vision, from Python; it also shows how to use ffmpeg to stream webcam data directly into Blender.

Either contains a world of possibilities for image manipulation. OpenCV and ffmpeg both support a variety of concurrent devices, in different manners. In fact, I made a note at the end about someday dedicating a Warrior-Poet’s Guide to each of them. This opens up a range of possibilities for 3D image processing.

My web camera, projected onto the surface of a beveled cube, in quasi-Predator-Vision.

Purchase the book on Amazon to gain access to this wonderful trick. I can’t wait to see what people start doing with it!

Geomorphology with Blender

Geomorphology is the science of the shape of mountains, rivers, islands, and other natural landmasses. Paleogeomorphology is the study of faults, erosion, and even tree rings which provide data on earthquakes which may have happened thousands of years ago.

The most fascinating aspect of geomorphology is the time-bracketing of events. Earthquakes can leave faulting which, relative to known processes like C₁₄ uptake and shoreline erosion, can result in windows of several centuries over which they may have occurred.

For a computational designer, it’s more traditional that mountains and terrains are generated purely from Perlin noise and Voronoi diagrams. However, while the process is remarkably efficient, they never seem to look quite right. Without the involvement of human hands, there is rarely any erosion or character on the terrain. This provides limited value in the context of a game, but what if we want to see a mountain with a history behind it? What if we want the land to tell a story, as real terrain often does?

Sample terrain fragment after applying wind erosion, basic particulation, tree groves and fundamental water erosion.

An in-house project, currently going on here in the studio, uses a LAN-scoped web interface built in D, and a local server, to evolve land masses over time, from a baseline. Techniques like Söbel softening, very basic Voronoi diagrams, and Perlin distribution of density allow for erosive processes to iterate year by year. Since this is effectively a bake, time hasn’t been as much of an object; however, for 512×512 images it has been moving surprisingly fast. Depending on detail, we can currently run it in roughly two seconds; combined with rendering (including volumetrics) Eevee can have it ready in about fifteen seconds.

Currently, output is saved as a sequence of 8-bit gray scale PNG height maps and density maps, unfortunately that limits us to just 2⁸ or 256 values! An effort is being made to expand it to 16-bit gray scale PNG, bringing us to 2¹⁶ or 65,536 grades, or even 24-bit TIFF at 16,777,216 possible grades. However, most displays are only capable of 8-bit color, so not as many options remain. Most of these files are for scientific output, purely; as far as I know, no monitor is capable of accurately representing anything higher than 16-bit color, and most only 8-bit.

The FreeImage toolkit allows for, in theory, 32-bit gray scale or even double-precision floating point gray scale (64-bit), but that may do more harm to the project than good. After all, portability is important, and nothing meant for the human eye ever seems to go above 16-bits, so there is little support for this in most design software.

In theory, Blender (or any number of other 3D design programs) would truncate it at 16-bits anyway, and after a number is above the atomic unit of operation for a processor (typically 64 bits these days), there’s a noticeable slowdown. Time may be more valuable than precision here.

The remaining standing question is, what was our planet like for its baseline, before any tectonic or fluvial geomorphing? If we wanted to start from a clean-slate, with basic density voxels, would that give us a realistic result? It’s very hard to say at this point in history. We have a rough idea of how the planet formed, but between the stresses of heat and age, could any reliable evidence of that still exist? This period is generally known as the Hadean period, taking its root from “Hades”. The world was indeed very hell-like; it had an abundance of highly radioactive and short-lived elements, which have since decayed, and given that it was a mass of recently-pounded together space rocks, it was ridiculously hot. We’re talking about peak environmental erosion here, so assuming a perfect sphere with reasonable thermal radiance may still apply, whether the world ever looked like that or not.

One of the only remaining elemental minerals from that period is known as Hadean Zircon. Zircon, consisting of ZrSiO₄. It has a fusibility (melting-point) of 2550° C or 4620° F, which is ridiculously high. Its Mohs scale hardness value is around 7.6, a little short of topaz. It is also largely insoluble. Given the gemstone’s relative indestructibility, it seems reasonable that it would be one of the last surviving elements from the Hadean period; even still, only roughly 1% of zircons found globally are confirmed to be Hadean.

Hadean Zircon Fragment
Valley, John W., et al. “Hadean Age for a Post-magma-ocean Zircon Confirmed by Atom-probe Tomography.” Nature Geoscience 7.3 (2014): 219-223

The plan is to move from 2D-mapping to 3D-voxel data, and work from there.