14th May

1. Introduction

I now have two methods for performing functional cFDR:

Using interpolation (i.e. James’ method of interpolating to find L curves)
Using contours (i.e. defining L curves directly as the contours of the cfdr curves)

I compare the results when using these two methods. Note that the mehods only differ in the final step of defining the L curves (the cfdr/ccut values are the same).

I also include two test data points to the analysis representing p,q pairs which should give $v\approx 1$.

Red: $(p=1, q=max(q[sub])=0.8107107)$ gives ccut = 0.9986857. Here I set $q$ equal to the biggest $q$ value out of the SNPs I find $v$-values for, which incedently is the minimum $q$ over all 121,000 SNPs.
Green: $(p=1, q=lims[4]=1.154623)$ gives ccut = 1. Here I set $q$ equal to the upper q limit for the KDE of the joint, which is set to be 10% higher than $q[sub]$. Note that this is a lot higher than the biggest $q$ in the whole dataset.

2. Results using interpolation

We see that when I don’t do any normalisation in the final integration step, some $v>1$. I investigate normalising the integral by the full L region (green example, defined on $[0,1] \times [lims[3], lims[4]]$) or by the L region in the red example.

3. Results using contours

Why is the integral over the red region bigger than that over the green region????

The v-values now extend even higher than 1. I investigate normalising the integral by the full L region (green example, defined on $[0,1] \times [lims[3], lims[4]]$) or by the L region in the red example.

4. Tailing off

If we are to use the contour method then we need to fix the tailing off. I consider a single example where the interpolation method doesn’t give any tailing off but the contour method does. The tailing off is due to the extension of the L curve at $q=-2.764940271$ which is approximately min(q[sub]) and is present in all the indices where we see tailing off. This bulge outwards increases the area of the L region therefore increasing the integral.

What does cgrid look like for $q<-2.764940271$? Upon visual inspection of my adjusted cfdr curves, I notice that when we zoom in there is a problem for very low q. The red lines are for $q<-2.764940271$ and this is the source of the problem. [The yellow line is the next smallest q value, where the bulging doesnt happen (i.e. $q=-2.744202$)]

This occurs because of the following bending in the cfdr curves before I force them to be non-increasing using cgrid$z = apply(cgrid$z,2,function(x) cummin(x-rev(cummin(rev(x))))).

I think the problem is that the other curves drop back down (so arent forced straight to 0), whereas the red curves do not reach below this value and thus drop straight down to 0.

Now I need to figure out how to resolve this..

5. Comments

Possible problems of normalising by the red curve for the integral:

What if max(q[sub]) is small (reflecting more relevant annotation), is it really ok to set this $v$ value to 1?
It seems to me that more high P values should equal 1.

Side note on understanding the bending which occurs even for cfdr=1 (green)

The bend occurs between yval2[55] and yval2[63], i.e. $-2.749246<q< -2.716198$, which is exactly where the cfdr curves extend beyond 1. This means that when we read off the cfdr curves at ccut=1, the corresponding P value is not 1 as the cfdr curve hasnt reached 1 yet, as shown in the plot below (read up from ccut shown by the dashed line to get the x coordinate of the L curve.

Another approach could be to prevent the cfdr curves from extending above 1. I can think of 3 possible ways to do this and show the results (using the contour method):

Set any $cfdr>1$ equal to 1.
Scale any cfdr curves that extend beyond 1 to the $[0,1]$ range (independently).
Scale all cfdr curves by the maximum.

Once I have a fully working method, I will investigate further making my cfdr curves exactly in $[0,1]$.