Feature Request: An option to disable flushing to zero on CPU

[Zentrik]

I would like a way to disable the code below,

xla/xla/service/llvm_ir/llvm_util.cc

Line 718 in a41222c

function->addFnAttr("denormal-fp-math", "preserve-sign");

Xla supports disabling fastmath (e.g. the --xla_cpu_enable_fast_math flag) and we seem to disable flushing to zeros in the code below. So controlling whether we set denormal-fp-math seems reasonable to me.

xla/xla/service/llvm_compiler.cc

Lines 41 to 51 in a41222c

	// Tensorflow tries to enable the following behaviors in all its threads:
	//
	// - Denormals are zero (DAZ): roughly, operations treat denormal floats as
	// zero.
	// - Flush denormals to zero (FTZ): roughly, operations produce zero instead
	// of denormal floats.
	//
	// In theory enabling these shouldn't matter since the compiler should ideally
	// not leak its environment into generated code, but we turn off DAZ and FTZ
	// to get some defense-in-depth.
	tsl::port::ScopedDontFlushDenormal dont_flush_denormals;

Based on llvm/llvm-project#81204 (comment) and https://groups.google.com/g/llvm-dev/c/TDGKHFU4hzE/m/k-LEa3NvBQAJ I don't believe on x86_64 that setting denormal-fp-math actually sets the floating point register and instead might only affect constant folding.

EDIT: Also when on gpu the xla_gpu_ftz flag can be used to control this.