Calculate the correlation coefficient

Question

Given a series of numbers for events X and Y, calculate Pearson's correlation coefficient. The probability of each event is equal, so expected values can be calculated by simply summing each series and dividing by the number of trials.

Input

1   6.86
2   5.92
3   6.08
4   8.34
5   8.7
6   8.16
7   8.22
8   7.68
9   12.04
10  8.6
11  10.96

Output

0.769

Shortest code wins. Input can be by stdin or arg. Output will be by stdout.

Edit: Builtin functions should not be allowed (ie calculated expected value, variance, deviation, etc) to allow more diversity in solutions. However, feel free to demonstrate a language that is well suited for the task using builtins (for exhibition).

Based on David's idea for input for Mathematica (86 char using builtin mean)

m=Mean;x=d[[All,1]];y=d[[All,2]];(m@(x*y)-m@x*m@y)/Sqrt[(m@(x^2)-m@x^2)(m@(y^2)-m@y^2)]

m = Mean;
x = d[[All,1]];
y = d[[All,2]];
(m@(x*y) - m@x*m@y)/((m@(x^2) - m@x^2)(m@(y^2) - m@y^2))^.5

Skirting by using our own mean (101 char)

m=Total[#]/Length[#]&;x=d[[All,1]];y=d[[All,2]];(m@(x*y)-m@x*m@y)/((m@(x^2)-m@x^2)(m@(y^2)-m@y^2))^.5

m = Total[#]/Length[#]&;
x = d[[All,1]];
y = d[[All,2]];
(m@(x*y)-m@x*m@y)/((m@(x^2)-m@x^2)(m@(y^2)-m@y^2))^.5

Answer 1

PHP 144 bytes

<?
for(;fscanf(STDIN,'%f%f',$$n,${-$n});$f+=${-$n++})$e+=$$n;
for(;$$i;$z+=$$i*$a=${-$i++}-=$f/$n,$y+=$a*$a)$x+=$$i*$$i-=$e/$n;
echo$z/sqrt($x*$y);

Takes the input from STDIN, in the format provided in the original post. Result:

0.76909044055492

Using the vector dot product:

where are the input vectors adjusted downwards by and respectively.

Perl 112 bytes

/ /,$e+=$`,$f+=$',@v=($',@v)for@u=<>;
$x+=($_-=$e/$.)*$_,$y+=($;=$f/$.-pop@v)*$;,$z-=$_*$;for@u;
print$z/sqrt$x*$y

0.76909044055492

Same alg, different language. In both cases, new lines have been added for 'readability', and are not required. The only notable difference in length is the first line: the parsing of input.

Answer 2

Mathematica 34 bytes

Here are a few ways to obtain the Pearson product moment correlation. They all produce the same result. From Dr. belisarius: 34 bytes

Dot@@Normalize/@(#-Mean@#&)/@{x,y}

---

Built-in Correlation function I: 15 chars

This assumes that x and y are lists corresponding to each variable.

x~Correlation~y

0.76909

---

Built-in Correlation function II: 31 chars

This assumes d is a list of ordered pairs.

d[[;;,1]]~Correlation~d[[;;,2]]

0.76909

The use of ;; for All thanks to A Simmons.

---

Relying on the Standard Deviation function: 118 115 chars

The correlation can be determined by:

s=StandardDeviation;
m=Mean;
n=Length@d;
x=d[[;;,1]];
y=d[[;;,2]];
Sum[((x[[i]]-m@x)/s@x)((y[[i]]-m@y)/s@y),{i,n}]/(n-1)

0.76909

---

Hand-rolled Correlation: 119 chars

Assuming x and y are lists...

s=Sum;n=Length@d;m@p_:=Tr@p/n;
(s[(x[[i]]-m@x)(y[[i]]-m@y),{i,n}]/Sqrt@(s[(x[[i]]-m@x)^2,{i,n}] s[(y[[i]] - m@y)^2,{i,n}]))

0.76909

Answer 3

Q

Assuming builtins are allowed and x,y data are seperate vectors (7 chars):

x cor y

If data are stored as orderded pairs, as indicated by David Carraher, we get (for 12 characters):

{(cor).(+)x}

Answer 4

MATLAB/Octave

For the purpose of demonstrating built-ins only:

octave:1> corr(X,Y)
ans =  0.76909
octave:2> 

Answer 5

J, 30 27 bytes

([:+/*%*&(+/)&.:*:)&(-+/%#)

This time as a function taking two arguments. Uses the vector formula for calculating it.

Usage

   f =: ([:+/*%*&(+/)&.:*:)&(-+/%#)
   (1 2 3 4 5 6 7 8 9 10 11) f (6.86 5.92 6.08 8.34 8.7 8.16 8.22 7.68 12.04 8.6 10.96)
0.76909

Explanation

Takes two lists a and b as separate arguments.

([:+/*%*&(+/)&.:*:)&(-+/%#)  Input: a on LHS, b on RHS
                   &(     )  For a and b
                         #     Get the count
                      +/       Reduce using addition to get the sum
                        %      Divide the sum by the count to get the average
                     -         Subtract the initial value from the average
                             Now a and b have both been shifted by their average
                             For both a and b
                *:             Square each value
         (+/)&.:               Reduce the values using addition to get the sum
                               Apply in the inverse of squaring to take the square root
                               of the sum to get the norm
       *&                    Multiply norm(a) by norm(b)
     *                       Multiply a and b elementwise
      %                      Divide a*b by norm(a)*norm(b) elementwise
 [:+/                        Reduce using addition to the sum which is the
                             correlation coefficient and return it

Answer 6

APL 57

Using the dot product approach:

a←1 2 3 4 5 6 7 8 9 10 11

b←6.86 5.92 6.08 8.34 8.7 8.16 8.22 7.68 12.04 8.6 10.96

(a+.×b)÷((+/(a←a-(+/a)÷⍴a)*2)*.5)×(+/(b←b-(+/b)÷⍴b)*2)*.5

0.7690904406         

Answer 7

Python 3, 140 bytes

E=lambda x:sum(x)/len(x)
S=lambda x:(sum((E(x)-X)**2for X in x)/len(x))**.5
lambda x,y:E([(X-E(x))*(Y-E(y))for X,Y in zip(x,y)])/S(x)/S(y)

2 helper functions (E and S, for expected value and standard deviation, respectively) are defined. Input is expected as 2 iterables (lists, tuples, etc). Try it online.

Answer 8

Oracle SQL 11.2, 152 bytes (for exhibition)

SELECT CORR(a,b)FROM(SELECT REGEXP_SUBSTR(:1,'[^ ]+',1,2*LEVEL-1)a,REGEXP_SUBSTR(:1,'[^ ]+',1,2*LEVEL)b FROM DUAL CONNECT BY INSTR(:1,' ',2,LEVEL-1)>0);

Un-golfed

SELECT CORR(a,b)
FROM
(
  SELECT REGEXP_SUBSTR(:1, '[^ ]+', 1, 2*LEVEL-1)a, REGEXP_SUBSTR(:1, '[^ ]+', 1, 2*LEVEL)b
  FROM DUAL
  CONNECT BY INSTR(:1, ' ', 2, LEVEL - 1) > 0
)

Input string should use the same decimal separator as the database.

Answer 9

Python 3 with SciPy, 52 bytes (for exhibition)

from scipy.stats import*
lambda x,y:pearsonr(x,y)[0]

An anonymous function that takes input of the two data sets as lists x and y, and returns the correlation coefficient.

How it works

There's not a lot going on here; SciPy has a builtin that returns both the coefficient and the p-value for testing non-correlation, so the function simply passes the data sets to this and returns the first element of the (coefficient, p-value) tuple returned by the builtin.

Try it on Ideone