For starters, to be precise, note that we are talking about the coefficient of static friction.
Let's look down the axes of the cylinders for a moment, specifically I want to look at a cross-section of the three cylinders. Note that the axes form an equilateral triangle.
What we wish to do is simple in principle: Use Newton's second law on each of the cylinders. For notational convenience I will call the bottom left cylinder 1, the bottom right cylinder 2, and the top cylinder 3. In all Free-Body diagrams I will use a coordinate system with +x to the right and +y straight up.
Two last comments before we start:
1) We may assume we are talking about a maximum static friction force for a given coefficient since we wish to know the minimum possible coefficient. So we can take f = (mu)N.
2) We need to find directions for the friction and normal forces. Note that static friction and the normal force will always be at right angles, and the friction force will be tangent to the surface (it always is, by definition). Then the normal force MUST be along the direction of a line segement starting at the center of the cylinder and ending at the point of contact between the cylinders, since this line will be perpendicular to the tangent. (If this statement is confusing I suggest you look up some geometry on circles and tangents.)
FBD for 1:
There is a weight (w1) acting straight down, a normal force between the horizontal plane and the cylinder (N1p), a friction force between the cylinder and the plane (f1p) pointing to the right, a normal force between cylinders 1 and 2 (N12) (more on this force later), a normal force between cylinders 1 and 3 (N13) acting along the line connecting the axes of cylinders 1 and 3, and a friction force between cylinders 1 and 3 (f13) acting perpendicular to N13 and pointing "upwardish." (I'll calculate the angles in a moment.)
First a note about N13. In finding the limit of the coefficient of static friction we may neglect this force. Why? Because in the mimimum case we need to assume the two bottom cylinders are soley being held in place by friction. So we may take N13 = 0 N.
Let's find the angle N13 makes with the -x axis. We know it's going to point down and to the left along the line connecting the axes. This is one side of that equilateral triangle, so the angle it makes with the -x axis will be 60 degrees below the -x axis, the same value as the base angle of the equilateral triangle. Similarly, f13 is pointing up and to the left and is at right angles to N13. Thus f13 makes an angle of 30 degrees above the -x axis.
Now we can do Newton's 2nd:
(Sum)F1x = f1p - f13*cos(30) - N13*cos(60) = 0 (since nothing is accelerating.)
(Sum)F1y = N1p - w1 + f13*sin(30) - N13*sin(60) = 0
f1p = (mu)N1p
f13 = (mu)N13
(mu)N1p - (mu)N13*cos(30) - N13*cos(60) = 0
N1p - m*g + (mu)N13*sin(30) - N13*sin(60) = 0
We have (count them!) four unknowns in these two equations. We need more equations. Fortunately we have two more cylinders we can do FBDs on.
I'm not going to work out all of the setup for you on the other cylinders, simply give you the equations that Newton's 2nd provides. They are set up in the same manner as the first cylinder was and if you have any quesitons about them simply ask.
-(mu)N2p + (mu)N23*cos(30) + N23*cos(60) = 0
N2p - m*g + (mu)N23*sin(30) - N23*sin(60) = 0
(We have added two equations, but also added two new unknowns. We still need more equations.)
-(mu)N31*cos(30) + N31*cos(60) + (mu)N32*cos(30) - N32*cos(60) = 0
-m*g + (mu)N31*sin(30) + N31*sin(60) + (mu)N32*sin(30) + N32*sin(60) = 0
N31 = N13
N32 = N23
since they are 3rd Law pairs. So all of the unknowns in these last two equations appear in other equations. Thus we finally have enough equations to solve the system for mu.
I am not going to solve this system. Have a crack at it and see what you can do. If you have further questions, just let me know.