Threading.Interlocked methods are not full fences

Everybody is in the wrong lane

When you google Interlocked class, you will find many blogs/articles/other things claiming that methods in the Interlocked class generate full fences. Even the excellent book CLR via C# from Jeffrey Richter claims it on page 803 (Interlocked Constructs).

Now what I am saying is: no, Interlocked operations are not full fences. When everything is coming your way, you’re in the wrong lane, you could answer. But not this time, and I am going to prove it.

What do Interlocked methods actually do?

To see what is the implementation of the Interlocked methods, check the generated assembly code of the following simple program:

class Program
{
    static int source = 10;
    static int destination = 0;

    static void Main(string[] args)
    {
        Debugger.Break();

        Interlocked.Exchange(ref destination, source); 
    }
}

The generated code is:

// Debugger.Break();
00000000  call        65BE7D10 

// Interlocked.Exchange(ref destination, source);
00000005  mov         eax,dword ptr ds:[00293358h] 
0000000a  xchg        eax,dword ptr ds:[0029335Ch] 

00000010  ret 

At the core of the implementation of Interlocked.Exchange() method (which is inlined), there is an x86 XCHG operation. From the manual of the processor we can learn that XCHG does a full fence. It seems that I am the one who drives on the wrong side. But wait a minute! CPU is only one component of the processing flow when we use a high level language like C#. The C# compiler still can reorder operations, and we have the jitter too. If an Interlocked operation is a full fence, it prevents the compiler and the jitter to do reordering across the call of Interlocked.Exchange() method.

The test

It is hard to predict when will the compiler/jitter optimize, but there is a common scenario: variables in the conditional expression of a for loop are often optimized into registers. If we use an Interlocked.Exchange() in the for loop, the program flow crosses the full fence before it reaches the evaluation of the conditional expression (not the first time, but after). According to the rules of full fencing, a read operation cannot cross the fence, so before the second evaluation of the conditional expression, the variables must be read from the memory again. Let’s see:

class Program
{
    static int source = 10;
    static int destination = 0;

    static void Main(string[] args)
    {
       Debugger.Break();

       for (var i = 0; i < source; i++)
       {
            Interlocked.Exchange(ref destination, source);
       } 
    }
}

And the generated code:

// Save stack state
00000000  push        ebp 
00000001  mov         ebp,esp 
00000003  push        esi 

// Debugger.Break
00000004  call        65A27D10 

// for loop, i = 0. So "i" will be in esi
00000009  xor         esi,esi 

// first evaluation of the conditional expression. It works
// a bit different as we expected. Here the jitter knows that
// i = 0, so i < source is false if source <= 0. This is what
// checked below:
// read "source" into edx:
0000000b  mov         edx,dword ptr ds:[001A3358h]     // enregister "source"

// source <= 0? 
00000011  test        edx,edx 

// if it is, goto after the loop
00000013  jle         00000022 

// the implementation of the Interlocked.Exchange().
// at this moment variable "source" is in edx. It is ok to use it
// because this is the first execution of the loop,
// so we have not crossed the fence yet.
00000015  mov         eax,edx 
00000017  xchg        eax,dword ptr ds:[001A335Ch]     // the fence is here

// i++
0000001d  inc         esi 

// this is the second and further evaluation of the conditional
// expression. but wait! the enregistered value in edx is being used, 
// so the read operation at 0x0000000b crossed the fence!!
0000001e  cmp         esi,edx                          // using enregistered value

// execute loop again, if i < enregistered "source"
00000020  jl          00000015 

// back to caller.
00000022  pop         esi 
00000023  pop         ebp 
00000024  ret 

As we can see, the program flow crossed the fence, but the read operation of the source variable was used behind the fence. If Interlocked.Exchange() would be a full fence, the “soure” variable would not be enregistered. We can try this using a real fence, Thread.MemoryBarrier():

class Program
{
    static int source = 10;
    static int destination = 0;

    static void Main(string[] args)
    {
        Debugger.Break();

       for (var i = 0; i < source; i++)
       {
            Interlocked.Exchange(ref destination, source);
            Thread.MemoryBarrier();
       } 
    }
}

And the generated code:

// Save stack state
00000000  push        ebp 
00000001  mov         ebp,esp 

// Debugger.Break
00000003  call        65D37D10 


// for loop, i = 0
00000008  xor         edx,edx 

// This time it is a direct operation on "source". But the logic 
// remained the same, instead of i < source, it check source <= 0
0000000a  cmp         dword ptr ds:[00143358h],0 
00000011  jle         0000002C 

// Interlocked now uses direct read
00000013  mov         eax,dword ptr ds:[00143358h] 
00000018  xchg        eax,dword ptr ds:[0014335Ch] 

// implementation of MemoryBarrier(). It uses a lock prefix when does "nothing"
// with the top of the stack. The lock prefix makes the fence at CPU level. We 
// do not really need it now, because CPU fence was made by xchg just one operation
// before.
0000001e  lock or     dword ptr [esp],0 

// i++
00000023  inc         edx 

// this is the second and further evaluation of the conditional
// expression. Now it re-reads value of "source" variable
00000024  cmp         edx,dword ptr ds:[00143358h] 
0000002a  jl          00000013 

// back to caller.
0000002c  pop         ebp 
0000002d  ret 

Conclusion

We found an example where a method of Interlocked class did not act as fence. The confusion around this class probably comes from the fact, that at machine code level it uses fencing instructions.