3. Linear Regression

Exercise 12: Simple regression without an intercept

a. When is $\hat{\beta}$ the same when we switch predictor and response?

From equation (3.38), $\hat{\beta}$ is the same in both cases when

$\frac{\sum_i x_iy_i}{\sum_i x_i^2} = \frac{\sum_i y_ix_i}{\sum_i y_i^2}$

The numerators are always equal, so $\hat{\beta}$ is the same iff

$\sum_i x_i^2 = \sum_i y_i^2$

b. Generate a counterexample

This is fairly easy to do, since with overwhelming probability

$\sum_i x_i^2 \neq \sum_i y_i^2$

import numpy as np
import pandas as pd
import statsmodels.api as sm

x, y = np.random.normal(size=100), np.random.normal(size=100)

model_1 = sm.OLS(y, x).fit()
model_2 = sm.OLS(x, y).fit()

model_1.params[0] == model_2.params[0]

False

c. Generate an example

This is fairly easy to do cheesily by letting x = y

x = np.random.normal(size=100)
y = x

model_1 = sm.OLS(y, x).fit()
model_2 = sm.OLS(x, y).fit()

model_1.params[0] == model_2.params[0]

True