An Entropic Approach to Constrained Linear Regression

We introduce a novel entropy minimization approach for the solution of constrained linear regression problems. Rather than minimizing the quadratic error, our method minimizes the Fermi–Dirac entropy, with the problem data incorporated as constraints. In addition to providing a solution to the linear regression problem, this approach also estimates the measurement error. The only prior assumption made about the errors is analogous to the assumption made about the unknown regression coefficients: specifically, the size of the interval within which they are expected to lie. We compare the results of our approach with those obtained using the disciplined convex optimization methodology. Furthermore, we address consistency issues and present examples to illustrate the effectiveness of our method.