Frequently Asked Questions (FAQ)

How does magnetico fetch metadata of torrents from DHT nodes?

Skim through BEP 5: DHT Protocol first.

magnetico basically works by launching a Sybil attack on the DHT network.

• It forges a (fake) node ID for every node it contacts to make the node consider us a neighbour.
  • "Proximity" and "neighbourhood" relies on the XOR difference (in Mainline DHT which is based on Kademlia), so we copy the n = 15 most significant bytes of the ID of the node we are contacting, and append 5 more random bytes to complete the node ID to 20 bytes in total.
• Then it floods the network with get_peers requests, whose responses will
  • ...either return a list of BitTorrent peers that we can connect and fetch metadata from.
  • ...or return a list of more DHT nodes that we can contact.
• Whenever an announce_peer request is received, the BitTorrent peer is contacted to fetch the metadata.

magnetico does "fairly egregious bittorrent spec violations" which is completely unsustainable in the long run. Therefore, I am working on BEP 51: DHT Infohash Indexing which directly addresses our use-case.

How does it perform the first connection? What is the "starting point" to crawl the DHT?

Bootstrapping nodes, which serve at a well-known address (such as a domain), and available almost always.

From v0.7.2:

	bootstrappingNodes := []string{
		"router.bittorrent.com:6881",
		"dht.transmissionbt.com:6881",
		"dht.libtorrent.org:25401",
	}

How big is the DHT?

I have no clue, but some people have reached ten million torrents as I know of.

Is there an "end" to the process whereby it only then needs to load newly posted torrents (like Bitcoin) or is it kind of an endless process?

The latter, "it's kind of an endless process." DHT is not as structured as Bitcoin blockchain so there is no way -at least currently- for anyone to keep track of "new" torrents.

Is there a way to reliably backup and restore the database in case I need to change server and want to use all the content it's already downloaded in advance?

You can always copy the content of your ~/.local/share/magneticod directory (which contains an SQLite database, named "database.sqlite3" [how creative huh]).

Of course this is not a proper solution since you might as well want to migrate between different database engines (such as from SQLite to PostgreSQL), for which there is currently no solution (because only SQLite is supported). An export & import mechanism shall be implemented in future, see #187.

How to "submit" new torrents?

Is it also possible to submit (or start broadcasting) a new torrent (i.e. magnet link) without the use of a central torrent site, so that magnetico can discover it? [...] Do magnet links only become discoverable once it they have been shared and are being used?

I think the answer to the question is that indeed -unfortunately- magnet links become discoverable only once they have been shared and are being used, at least currently. Once we implement BEP 51: DHT Indexing support, I believe it would be much more likely to "traverse" the DHT as a whole and thus discover newly added (even not so active) torrents too, but only time will show as things might now always work as devised in social networks...