In image analysis, I collect pixels of different objects with different colors. Then I want to compare those colors with the color of a pixel in another image to find out which (if any) of the previously sampled objects it looks like.

So, each sampled object forms a 'cloud' of points in a three dimensional RGB (red, green, blue) 24 bit space. How should I proceed in order to find out what 'cloud' (=object color sample) a new pixel is closest to?

The 'RGB clouds' are bent and have varying thickness. So they are irregular, but rather well connected. Typically, the cloud is streched out between 'in the shadow' to 'reflecting light strongly'. Since the 24 bit space is pretty huge, only a fraction of the space in a cloud is occupied by sampled points.

Should I fit a function for each cloud? But conventional curve fitting can't do the trick here, no? The thickness of the point cloud is NOT the same thing as a higher variance. There is no reason to accept a point further outside of the cloud where the cloud is thick, than where it is thin.

I think I need to fit surfaces around the clouds, rather than curves through the middle of them. Then I could measure the distance from an observed point, to the closest cloud surfaces. I've never heard of anything like it, and I'm confused. But this kind of problem is hardly unusual. Does anyone have a reference to some similar exampel of this being done? Or anyone who can name drop some relevant google search term or somehting?