Non-normal bivariate distributions with normal marginal distributions

It’s well known that the ratio of two independent standard normal random variables has a Cauchy distribution. Let’s prove it! Suppose our normal random variables are X and Y. The kernel of the density is

p_{X,Y}\left(x,y \right )\propto\exp\left[-\frac{1}{2}\left(x^2+y^2\right ) \right ].

We do this change of variables:

\begin{aligned} C&=\frac{X}{Y},\\T&=Y^2.\end{aligned}

By a happy chance the Jacobian of this transformation is constant and so, with a little harmless disregard for the region of integration, we substitute directly into the joint density function and get

\begin{aligned}p_{C,T}\left(c,t \right )&\propto\exp\left[-\frac{1}{2}\left(1+c^2\right )t \right ]\\&=\left ( \frac{1}{1+c^2} \right )\left \{ \frac{1}{2}\left(1+c^2\right )\exp\left[-\frac{1}{2}\left(1+c^2\right )t \right ]\right \}.\end{aligned}

By inspection we can see that we’ve got the joint density of a Cauchy random variable C and, conditional on C, an exponential random variable T with rate (1 + C2)/2. (The kernel of the Cauchy density and the rate parameter of the exponential density are shown explicitly in the second line; in the first line they have effectively cancelled out.) Thus we have shown that the ratio of two independent standard normal random variables is indeed Cauchy.

We can do a little more with this result though. It shows how to generate two independent normal random variates from a Cauchy and an Exponential (and one more random bit) via the following recipe. Generate, independently:

  • a Cauchy random variable C,
  • a Rademacher random variable S (which takes the value +1 with 50% probability and the value -1 with 50% probability), and
  • an Exponential random variable (with rate 1) U.

Set

\begin{aligned}Y&=S\sqrt{\frac{2U}{\left(1+C^2 \right )}},\\X&=CY.\end{aligned}

Independent standard normal random variables aren’t too interesting; this recipe is worth considering because it gives us a way to generate something a lot more fun: three distinct (and very pretty) bivariate distributions, each non-normal but with normal marginal distributions. (Of course, there’s a boring brute force way to make such a distribution: pick any non-Gaussian bivariate copula and push the marginal uniform random variables through the normal quantile function. Yawn.) We’ll do this by making two independent copies of each of the three random variables in our recipe; that is, we’ll have Ci, Si, and Ui for i ∈ {1,2}. Now we use the i = 1 variables and follow the recipe as above,

\begin{aligned}Y&=S_1\sqrt{\frac{2U_1}{\left(1+C_1^2 \right )}},\\X&=C_1Y.\end{aligned}

And then we mix the indices to create two more marginally normal variables that have a complicated dependence relation to the previous set:

\begin{aligned}W&=S_2\sqrt{\frac{2U_2}{\left(1+C_1^2 \right )}},\\V&=S_2\sqrt{\frac{2U_1}{\left(1+C_2^2 \right )}}.\end{aligned}

So what kind of distributions did we make? Well, we’ve defined four distinct marginally normal random variables which gives rise to six bivariate distributions. The three interesting distributions are those of (W, X), (W, Y), and (V, Y). The other three pairs fail to spark joy: X and Y are independent standard normal by definition; V and W would be too except I didn’t bother to give them different signs so their joint distribution is only defined in the first and third quadrants; and it turns out that the pair (V, X) just has the same bivariate distribution as (V, Y).

Let’s sample some, uh, samples and see what we get.

w_vs_x
w_vs_y
v_vs_y

And here’s the code, ready to go.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

In the Dark

A blog about the Universe, and all that surrounds it

Minds aren't magic

Paul Crowley

Mad (Data) Scientist

Musings, useful code etc. on R and data science

djmarsay

Reasoning about reasoning, mathematically.

The Accidental Statistician

Occasional ramblings on statistics

Models Of Reality

Stochastic musings of a data scientist.

Data Colada

Thinking about evidence and vice versa

Hacked By Gl0w!Ng - F!R3

Stochastic musings of a data scientist.

John D. Cook

Stochastic musings of a data scientist.

Simply Statistics

Stochastic musings of a data scientist.

LessWrong

Stochastic musings of a data scientist.

Normal Deviate

Thoughts on Statistics and Machine Learning

Xi'an's Og

an attempt at bloggin, nothing more...

%d bloggers like this: