Final detector is evaluated on data it has been trained on?

[davidglavas]

The code that creates the detector (linear regression classifier):

detecting-adversarial-samples/scripts/detect_adv_samples.py

Lines 149 to 155 in 2c26b60

	## Build detector
	values, labels, lr = train_lr(
	densities_pos=densities_adv_z,
	densities_neg=np.concatenate((densities_normal_z, densities_noisy_z)),
	uncerts_pos=uncerts_adv_z,
	uncerts_neg=np.concatenate((uncerts_normal_z, uncerts_noisy_z))
	)

Variables values and labels returned by train_lr() represent the training data that the model has been trained on:

def train_lr(densities_pos, densities_neg, uncerts_pos, uncerts_neg):
    # [...data preparing code left out...]
    
    lr = LogisticRegressionCV(n_jobs=-1).fit(values, labels)
    return values, labels, lr

At the end the detector is evaluated on the data it was trained on (line 159 uses values which is the training data returned by train_lr):

detecting-adversarial-samples/scripts/detect_adv_samples.py

Lines 157 to 168 in 2c26b60

	## Evaluate detector
	# Compute logistic regression model predictions
	probs = lr.predict_proba(values)[:, 1]
	# Compute AUC
	n_samples = len(X_test)
	# The first 2/3 of 'probs' is the negative class (normal and noisy samples),
	# and the last 1/3 is the positive class (adversarial samples).
	_, _, auc_score = compute_roc(
	probs_neg=probs[:2 * n_samples],
	probs_pos=probs[2 * n_samples:]
	)
	print('Detector ROC-AUC score: %0.4f' % auc_score)

Evaluating on training data leads to a biased result. Did I miss anything?

[yoheikikuta]

That's what I thought.
It seems that the ROC-AUC evaluation uses the same data that was used in the training of a model.
Is this implementation as the authors intended?