Phase difference measurement under noise (How-to?)

Dear All,

 I try to measure the phase difference between a delayed signal and a reference one. I use the following blocks:

Multiply conjugate block (Inputs are the delayed and reference signals)

Complex to Arg

Moving Average

The flowgraph works good when there is no noise. However when I add noise the variations are pretty wild.

These blocks constitute the basic structure of a PLL's phase comparator which should work under noisy Input conditions.

What am I doing wrong?