Remove skip building graph check for quantization use case (#2430)

Remove skip building graph check for quantization use case (#2430) #171