Renormalize the tensors before SVD for the CTMRG projectors

[Confusio]

For the projector routines, we require the inverse square root of the singular values. However, when the local PEPS tensor has a small norm—or due to symmetry constraints—the SVD performed during the CTMRG projection can yield very small largest singular values. These small singular values create numerical instabilities both in the CTMRG algorithm and during the automatic differentiation (AD) steps in the optimization.

Currently, I address this by renormalizing the enlarged corner tensors within CTMRG when using the HalfInfiniteProjector. While a similar renormalization strategy seems to work for the FullInfiniteProjector in many cases, it is not robust enough under all conditions. I propose that we implement a mechanism to renormalize the tensors immediately prior to performing the SVD. This adjustment should improve the numerical stability and overall performance of both CTMRG and AD in our PEPS optimization routines.

[lkdvos]

I'm definitely okay with this, that seems very sensible. Does this alter the output in any way? @pbrehmer what do you think?

[Confusio]

If this alters the output somewhere else, maybe we can define the following general projector function when we do the truncation on the bond connecting the R (right) and L (left) parts,

function build_projectors(R::AbstractTensorMap,L::AbstractTensorMap)
    R=R/norm(R)
    L=L/norm(L)
    if dim(codomain(R)) > dim(domain(R))
        _,R= leftorth!(R)
        L,_=rightorth!(L)
    end
    U,S,V=tsvd!(R⊙L;trunc=trscheme,alg = TensorKit.SVD()) #⊙ is for the tensor contraction.
    vals=deepcopy(S)
    S=sqrt(pinv(S))
    return L*V'*S,S*U'*R,vals
end

[pbrehmer]

Thanks for the suggestion! This seems very sensible indeed. I don't think it should alter the output since the CTMRG fixed-point equations for the corners and edges are anyway only defined up to a scalar constant (and during the renormalization we anyway normalize the corners and edges).

I can quickly give this a go tomorrow!

[lkdvos]

Just as a heads up, I did check the implementation for TensorKit's SVD, and there at least we do use a relative tolerance, so this would not actually affect that part of the computation. see here

I would have to check what the other SVD implementations do though. In any case, normalizing the (half)infinite environments seems like a convenient choice anyways, since that makes the truncation algorithms more natural. I'm just not sure if it will help with the stability, since there we probably need some broadening instead(?)