Thoughts from running an Admarus node

[jphastings]

Hello! I enjoyed running an Admarus node this weekend after reading about it in the IPFS newsletter. I'd be interested in running one permanently next to my homelab's IPFS node as it was responsive and good at searching the documents pinned to my node!

Here are some notes and suggestions from my experience:

Building & install

I don't spend a lot of time in Rust, so it took me a moment or two to realise that my rustc was too out of date to build Admarus, and then to hunt down the way I installed rustc so I could upgrade it the same way.

Github actions is pretty easy to configure to build & push containers to their ghcr.io container repository (it's also free for OpenSource projects like Admarus, as well as offering a free security audit too). Having a docker one-liner to run would have made things a lot easier, and will make it easier for me to keep a server up-to-date in the future.

If this sounds useful, let me know and I'll put a PR together.

WebUI and operation

The UI is already impressively polished for an MVP! There were some CSS oddities when I ran the locally-built version, but I quickly realised how seemlessly admarus.net integrates with my local instance, so I just used that instead.

My first searches were quite confusing (through my own "fault", more below), but I did notice that pressing "search" again (with no change to the search query) didn't appear to re-query the Admarus server. This meant that search results looked like they persisted even after I'd unpinned on my IPFS server. Perhaps this is some query caching? If so it might be worth allowing the same query to be sent after a short while (perhaps only a locally running instance?)

Search results

When I first searched (for a term that would only be in my own pinned data — the name of my blog) I found the results very confusing! Some slightly less useful search results came first (I would have expected my blog's homepage at the top, but I got seemingly random posts instead — though this seems very reasonable for an MVP!) but lots of identical duplicate pages.

It turns out that I had forgotten that I'd pinned many different recent revisions of my blog, but what was interesting was that some of the duplicate pages had the same CID but were still showing twice, which confused me. I think this is because the root CID was different (even though the entry CID was the same).

`QmCIDforPostVersion1` appears twice in search results, even though it is identical

Pinned blog from May:

QmCIDforBlogPublishedInMay
├── index.html (QmCIDforHomepageVersion1)
└── posts
    └── some-post.html (QmCIDforPostVersion1)

Pinned blog from July:

QmCIDforBlogPublishedInJuly
├── index.html (QmCIDforHomepageVersion2)
└── posts
    └── some-post.html (QmCIDforPostVersion1)

I expect that only returning this result once might be a little difficult to accomplish (which parent version do you link to?) but with multiple versions of, for example, the Wikipedia on IPFS being pinned, search results might get very hard to navigate as the network grows.

Return links

I then got to thinking about setting up my site to show up usefully in an Admarus search. I couldn't figure out how to change the URL shown in the search result from an ipfs://bafy…/posts/some-post.html URL to being the DNSLink address the site exists canonically at. I have appropriate rel="canonical" meta tags in the HTML (which I thought might help, but didn't), and I tried pinning the IPNS reference (instead of the CIDs) too, but I can't figure it out!

I figure building a guide on SEO for Admarus is a long way away (and hopefully very much not tied to advertising optimisation at all) but this felt a little confusing.

Managing pinned items

Admarus was actually incredibly useful for discovering what I had pinned! I had forgotten what a lot of things were, as managing base58 hashes manually is… boring and mostly not worth the bytes of disk space I'd save. However once Admarus had shown me the pins I wanted to remove, it was suprisingly hard for me to remove them.

Admarus (correctly, IMHO) uses CIDv1/subdomain format for its search results, but my pins happened to be in CIDv0 format. I know of tools for converting from CIDv0 to CIDv1, but not easily for the reverse, so I spent a few minutes converting all my local CIDv0s to v1s, so I could see which pins to remove.

It would be nice to see Admarus recognise when the search results are pinned locally, and let me manage the pin directly — or (when they're not pinned locally) easily let me pin something, so I can come back to it later if my net connection is down.

---

Anyway, this is already enormous — I'll stop there; thanks for putting Admarus together, it was fun to play with, I hope I can help it improve!

[Mubelotix]

It would definitely be useful to have a docker container available. I created the issue #97, I would be glad to receive your contribution!

I'm surprised you had CSS issues. Maybe it was broken at that time and fixed later on. Or maybe it's caused by the server software you used; may I ask what it was?

You are right: there is a bug preventing the query to re-run when unchanged, and files that are unpinned after indexing are indeed not removed from the index yet. I'll fix it. (#93 and #96)

There are currently two mechanisms to help Admarus rank different revisions of the same page:

1. Number of peers returning it, which is higher for the most popular (probably the best and latest) version
2. Whether the document is under a domain name or not

The first mechanism was not effective because you were the only node. The second wasn't effective either because Admarus is currently unable to detect domain names automatically. There is however a way to tell Admarus to track some domain names but this wiki item didn't exist at that time. This would probably have resulted in the latest version to be ranked first in your case.

There is currently no way to tell Admarus the url to be used. It uses the one with a domain name, or the shortest. This is a known limitation but I didn't know how to fix it. Now that you tell me about it, using the canonical url of the page is the way to go. This would also allow detecting domain names automatically, and the current dns-pin feature could be removed. Moreover, it would be of a great help to detect and remove duplicate content. (#94)

Results (ie: blog posts) are grouped under their parent (blog homepage) when their urls have the right structure and the parent's title matches the query. In your case, Admarus wrongly used the shortest ipfs urls so no parents could be detected. It should fix itself with #94 too

It would indeed be very convenient to be able to manage locally pinned files from Admarus. It will be done sometime after the other major features are implemented. (#95)

Thank you very much for your detailed input, this is most helpful! I'll use your suggestions to solve current issues and improve the experience of running a node. I hope to count you in the network in a few releases :)

[jphastings]

I'll see what I can do about a PR for docker support! I'm on call this week so I'll either have lots of spare time where I need to be near a computer, or no spare time tending to work issues 😄

---

I was using serve . (from npm install -g serve; in the webui directory after running trunk build --release). See below for a screenshot of how it appears even now on ccbf621. It looks the same way on Arc (screenshot), Safari and Firefox on my macbook.

---

Thanks for those two five new issues you've raised to address! I think they'll make the new developer experience a lot simpler!

---

I suppose my ranking questions are also slightly coloured by the search term I was using (one that exists on every page of my blog, and likely nowhere else), I can see that Admarus already groups results, so I was expecting my whole site to be presented as sub-results of my blog's homepage. (Ahh! You've pointed out this is already handled with #94 — awesome!)

The use-case I'm working towards here is the hosting (and searchability) of IndieWeb blogs — with content like that there's likely to be more "whole site" matches, compared to sites like Wikipieda-on-IPFS, where folks probably aren't searching for "wikipedia" (which, by the way, currently doesn't seem to return a single wikipedia result! Perhaps the domain name should be a part of the searched document? People do like to use search tools as address bars 🙄)

---

I'm glad my canonical URL suggestion was so helpful! I had a couple of questions about --dns-pins, but I'll hold them back if you're planning on removing that feature for canonical parsing 😊

---

You're very welcome! Thank you for engaging with my points so closely — I hope to work more in this space this week (🤞) so I'll keep in touch!