Binary Logistic Regression

Multiple linear regression is a statistical technique used to predict a continuous dependent variable from multiple independent variables.

The general equation for multiple linear regression, when we have k number of independent variables, is

Y = β₀ + β₁*X₁ + β₂*X₂ + ......β_k*X_k

Where,
Y = Dependent variable (continuous)
β₀ = intercept or constant. It is the value of Y, when all X (independent) variables are 0.
β₁ = regression coefficient for independent variable 1.
X₁ = Value of first independent variable.
β₂ = regression coefficient for independent variable 2.
X₂ = Value of second independent variable.

---

---

However, when dependent variable is not continuous, but binary (a categorical variable having only two possible values: 0 or 1), Binary Logistic Regression (BLR) is the technique to predict the dependent (outcome) variable. The difference in regression equations in MLR and BLR is given below.

MLR:      Y = β₀ + β₁*X₁ + β₂*X₂ + ......β_k*X_k

BLR:     LN(P/(1-P)) = β₀ + β₁*X₁ + β₂*X₂ + ......β_k*X_k

Please note the difference in the formula, with respect to the left side of the equation. In MLR the continuous variable can be directly estimated using the MLR regression equation.

In BLR, the left side is LN(P/(1-P)),

Here P = Probability of event, hence (1 - P) = Probability of no event.

P/(1-P) is ratio of probability of event to non-event. It is also called "odds" of the event. LN stands for "Natural Logarithm".

So, LN(P/(1-P)) = Log Odds, also called as logit.

So, BLR gives the logit for given set of independent variables. From logit, we can easily calculate probability of event. If this probability is equal to or more than 0.5 (or any specified cut-off value), the prediction of event is made. If this probability is less the cut-off value, then prediction of no-event is made.

A question may arise here, why we can not use P in the left side of the BLR equation. Why we need to use logit? The answer is that there are only two possible values for dependent variable (0 or 1). Also P, being probability, must have a value between 0 and 1. However, right side  of equation may give a value less than 0 or more than 1. Such probability values are invalid and impossible to interpret. We want something which can take any value, and can be converted to valid probabiity value, between 0 and 1. This requirement is fulfilled by log odds, which can take any value, and which can be converted back into valid probability value.

Example: Dependent variable is DM (1 for presence of diabetes and 0 for its absence). There are only two independent variables. RBSL_1 (Random BSL) and SBP_1 (systolic BP). N=237

You can copy the data from excel file (along with heading row) and paste into the textbox. Automatically the data is shown in the tabular format. You need not to create dummy variables for categorical independent variables. These will be automatically calculated by the software. Alphabetically, first category is treated as reference category, and for other categories, dummy variables are created.

You can click on "Select dependent and independent variables", to open a popup, wherein you can select a dependent and independent varibles. After filling all the required input, you can click on "Run BLR" to get the output.

---

Model	Chi-square	degree of freedom	p
Enter: All IDV Simultaneously	173.8551	2	0

Above table shows the Omnibus test of model coefficients. If the p value is significant, it means that the given predictors can predict the outcome significantly better than than the null model (without any predictors). Under null model, every outcome is predicted as Yes, if probablity of "Yes" in given dataset is more than probablity of "No". It simply means, every outcome is predicted as "Yes", if "Yes" (1) are more in number than "No" (0).  Needless to say that under Null model, every outcome will be prdicted as "No", if number of "No" are more than number of "Yes" in the dataset.
So, if p is significant, it means adding predictors in the model can significantly improve the prediction of outcome variable.

---

Model: Enter	- 2 Log Likelihood	Cox & Snell R Square	Nagelkerke R Square	McFadden R Square
Enter: All IDV Simultaneously	153.9832	0.5198	0.6938	0.5303

These psuedo R squared values can be considered as equivalent to R squared value in linear regression. These psuedo R squared values tell us about the approximate variance in outcome that can be explained by the given predictors. 

---

IDV	Coefficients (β)	Standard Error	Wald	df	p	Adjusted OR (Exp(B))	Confidence Interval*
							LB of Exp(β)	UB of Exp(β)
Constant	-14.6605	2.9525	24.6553	1	0	0	0	0.0001
RBSL_1	0.0695	0.0097	51.3288	1	0	1.072	1.0518	1.0926
SBP_1	0.0131	0.0164	0.6446	1	0.4221	1.0132	0.9812	1.0463

* (100 - Significance level)% Confidence interval
Above table gives the beta coefficients, their standard errors and significance for each of the coefficient. Significant p value for a coefficient means the coefficient is significantly different than 0. The table also gives te adjusted Odds Ratio and their confidence intervals for each of the predictor.
Here, Adj. OR = e^β
Below is the equation for the "logit". Logit = LN (Odds of Yes) = LN (P/(1-P)). LN = Log to base e. P= Probability of presence of the characteristic (1). Please note that (P/(1-P)) gives us the odds.
LN [P/(1-P)] = -14.6605 + 0.0695 * RBSL_1 + 0.0131 * SBP_1
Probability of presence of a characteristic (P) is given by following formula.
P = e^Logit / (1 + e^Logit)

---

Chi-square	df	p
22.0742	8	0.0048

Hosmer Lemeshow test compares the observed and expected probabilities in subgroups of the observations. Based on these observed and predicted probabilities Chi-square statistics is calculated using formula (O - E)²/E. Please note here that we want the observed and predicted probabilities to be equal to say that the model predicts the outcome correctly. So, we want O and E to be close to each other, and very small chi-square value. If chi-square is non-significant, our LR model is good.Otherwise, with significant p value, we can say that there is significant difference between observed and predicted observations.Hence, with significant p, model should be discarded.
Because of different number of observations in subgroups, the calculated chi-square value by different softwares can be slightly different. This is a major drawback of this test.
In above table, p is significant
-----------------------
Hosmer Lemeshow contigency table.

SubGroup	Number of Observations in the SubGroup	event = 0		event = 1
		Observed	Expected	Observed	Expected
1	24	24	23.8298	0	0.1702
2	24	24	23.3265	0	0.6735
3	24	24	22.1082	0	1.8918
4	24	14	19.6185	10	4.3815
5	24	16	15.586	8	8.414
6	24	10	10.8053	14	13.1947
7	24	13	7.0152	11	16.9848
8	24	0	2.4318	24	21.5682
9	24	0	0.2544	24	23.7456
10	21	0	0.0243	21	20.9757

---

		Predicted		Total	Percentage Correct
		1	0
Observed	1	93	19	112	83.0357
	0	20	105	125	84
Total		113	124	237	83.5443

Above table shows the comparison between predicted outcomes by the model (based on cutoff of probability) and actual observed outcome in the data. It also calculates the overall correctness of the model.

---