Multi-GPU Sharding

[gorlando04]

Summary

Platform

4 NVIDIA GeForce RTX 2080 Super
Ubuntu 20.04
Cuda 12

Faiss version: 1.7.3

Installed from: anaconda

Faiss compilation options:

Running on:

• CPU
• [x ] GPU

Interface:

• C++
• [x ] Python

Reproduction instructions

I am trying to make a knn search using faiss in GPU using the multi-gpu indexes with sharding. But when im trying to execute the code it keeps to get MemoryError.

import numpy as np
from time import time
d = 12 # dimension
nb = 40000000 # database size
np.random.seed(1234) # make reproducible
xb = np.random.random((nb, d)).astype('float32')

import faiss # make faiss available

#Recall func
def recall(arr1,arr2,k):

#Verificação da integridade
if arr1.shape != arr2.shape:
    print("Impossível de fazer a avaliação, as arrays tem tamanho diferentes")
elif arr1.shape[1] < k:
    print(f"Impossível de fazer o recall{k}, já que as array não tem {k} vizinhos")

#Somatório dos k primeiros vizinhos positivos dividido por n*k
acertos = 0

n = arr1.shape[0]


recall_value = (arr1[:,:k] == arr2[:,:k]).sum() / (float(n*k))

return recall_value

ngpus = faiss.get_num_gpus()

print("number of GPUs:", ngpus)

for i in range(20000,1000,-512):
k = 20
t0 = time()
qntd = faiss.IndexFlatL2(d)
cpu_index = faiss.IndexIVFFlat(qntd,d,i,faiss.METRIC_L2)
cpu_index.nprobe = 13
print(f"Nlist = {i} nprobe = {cpu_index.nprobe}")
print("Fazendo o indice")
p = faiss.GpuMultipleClonerOptions()
p.shard = True
p.common_ivf_quantizer = True
gpu_index = faiss.index_cpu_to_all_gpus( # build the index
cpu_index,p
)
print("Training")
gpu_index.train(xb)
gpu_index.add(xb) # add vectors to the index
print(f"Tempo de treino = {time()-t0}")

                    # we want to see 4 nearest neighbors
print("Search")
_, I = gpu_index.search(xb, k) # actual search
print(f"Time = {time()-t0}")

[mdouze]

This is probably because you are calling faiss.index_cpu_to_all_gpus in a loop, this allocates a StandardGpuResources object that uses about 1.5G of GPU RAM.

[gorlando04]

What should I use instead?