Killing zef corrupts the store (?)

[andinus]

andinus@ ~> zef --update=fez search fornax
===> Updating fez mirror: http://360.zef.pm/
===> Updated fez mirror: http://360.zef.pm/
Use of uninitialized value @short-names-to-index of type Any in string context.
Methods .^name, .raku, .gist, or .say can be used to stringify it to something meaningful.
  in block  at /home/andinus/.raku/sources/32E407EDE579C166DFEF1B180F5930E968008D1A (Zef::Repository::Ecosystems) line 247
Use of uninitialized value @short-names-to-index of type Any in string context.
Methods .^name, .raku, .gist, or .say can be used to stringify it to something meaningful.
  in block  at /home/andinus/.raku/sources/32E407EDE579C166DFEF1B180F5930E968008D1A (Zef::Repository::Ecosystems) line 247
===> Found 0 results

Context

Updating an ecosystem and searching for a module.

Expected Behavior

No warnings expected.

Steps to Reproduce

andinus@ ~> zef --debug --update=fez search fornax
===> Updating fez mirror: http://360.zef.pm/
===> Updated fez mirror: http://360.zef.pm/
Use of uninitialized value @short-names-to-index of type Any in string context.
Methods .^name, .raku, .gist, or .say can be used to stringify it to something meaningful.
  in block  at /home/andinus/.raku/sources/32E407EDE579C166DFEF1B180F5930E968008D1A (Zef::Repository::Ecosystems) line 247
Use of uninitialized value @short-names-to-index of type Any in string context.
Methods .^name, .raku, .gist, or .say can be used to stringify it to something meaningful.
  in block  at /home/andinus/.raku/sources/32E407EDE579C166DFEF1B180F5930E968008D1A (Zef::Repository::Ecosystems) line 247
===> Found 0 results

Your Environment

andinus@ ~> raku -v
Welcome to Rakudo™ v2021.10.
Implementing the Raku® Programming Language v6.d.
Built on MoarVM version 2021.10.
andinus@ ~> zef list --installed                                                              
===> Found via /home/andinus/.raku
App::RaCoCo:ver<1.4.4>:auth<zef:atroxaper>
Base64:ver<0.0.2>:auth<github:ugexe>         
CBOR::Simple:ver<0.1.0>:auth<zef:japhb>
CSV::Parser:ver<0.1.2>:auth<zef:tony-o>   
Cairo:ver<0.2.7>                                                                                         
Cro::Core:ver<0.8.6>    
Cro::HTTP:ver<0.8.6>                 
Cro::TLS:ver<0.8.6>
Cro::WebSocket:ver<0.8.6>             
Crypt::Random:ver<0.4.1>:auth<github:skinkade>
DB::Pg:ver<1.1>:auth<github:CurtTilmes>:api<1>
DBIish:ver<0.6.2>:auth<github:raku-community-modules>                                                    
Data::Dump:ver<0.0.14>:auth<zef:tony-o>
DateTime::Parse:ver<0.9.1>       
Digest::HMAC:ver<1.0.2>:auth<github:retupmoca>
Digest::SHA1::Native:ver<0.04>           
Digest:ver<0.4.0>:auth<Lucien Grondin>           
Docker::File:ver<1.0>:auth<github:jnthn>
File::Find:ver<0.1.1>
File::Ignore:ver<1.1>:auth<Jonathan Worthington <[email protected]>>                                       
File::Which:ver<1.0.1>                    
HTML::Escape:ver<0.0.1>                      
HTTP::HPACK:ver<0.9.2>                 
IO::Path::ChildSecure:ver<1.001011>          
IO::Socket::Async::SSL:ver<0.7.8>:auth<jnthn>                                                            
JSON::Class:ver<0.0.18>:auth<github:jonathanstowe>:api<1.0>                                              
JSON::Fast:ver<0.16>                     
JSON::JWT:ver<1.0>:auth<github:retupmoca>                                                               
JSON::Marshal:ver<0.0.23>:auth<github:jonathanstowe>:api<1.0>
JSON::Name:ver<0.0.6>:auth<github:jonathanstowe>:api<1.0>                                                
JSON::OptIn:ver<0.0.1>:auth<github:jonathanstowe>
JSON::Unmarshal:ver<0.08>                                                                                
LibUUID:ver<0.5>:auth<github:CurtTilmes>     
LibraryMake:ver<1.0.0>:auth<github:retupmoca>                                                           
License::SPDX:ver<3.14.0>:auth<github:jonathanstowe>:api<1.0>                                            
Linenoise:ver<0.1.1>:auth<Rob Hoelz>     
Log::Async:ver<0.0.7>                            
Log::Timeline:ver<0.4>                                                                                  
META6:ver<0.0.26>:auth<github:jonathanstowe>:api<1.0>                                                    
MIME::Base64:ver<1.2.1>:auth<github:retupmoca>
NativeHelpers::Blob:ver<0.1.12>:auth<github:salortiz>                                                    
NativeLibs:ver<0.0.9>:auth<github:salortiz>
OO::Monitors:ver<1.1.1>
OpenSSL:ver<0.1.27>:auth<github:sergot>
PathTools:ver<0.1.1>:auth<github:ugexe>
Red:ver<0.1.52>:auth<Fernando Correa de Oliveira>:api<2>                                                 
Shell::Command
Template::Nest:ver<0.1.1>
Terminal::ANSI:ver<0.0.24>:auth<cpan:bduggan>
Terminal::ANSIColor:ver<0.5>
Terminal::UI:ver<0.1.1>:auth<cpan:bduggan>
Test::META:ver<0.0.17>:auth<github:jonathanstowe>:api<1.0>                                               
Test::When:ver<1.001008>
TinyFloats:ver<0.0.3>:auth<zef:japhb>
URI:ver<0.3.3>
UUID:ver<1.0.0>:auth<github:retupmoca>
YAMLish:ver<0.0.6>
cro:ver<0.8.6>
fez:ver<31>:auth<zef:tony-o>:api<0>
fornax:ver<0.1.0>:auth<zef:andinus>
if:ver<0.1.1>:auth<github:FROGGS>
taurus:ver<0.1.1>:auth<zef:andinus>
zef:ver<0.13.1>:auth<github:ugexe>:api<0>
===> Found via /usr/local/rakudo/share/perl6/core
CORE:ver<6.d>:auth<perl>

[andinus]

I do remember killing zef while it was running zef --update search fornax.

It emits these even when performing a normal search:

andinus@ ~> zef search octans
Use of uninitialized value @short-names-to-index of type Any in string context.
Methods .^name, .raku, .gist, or .say can be used to stringify it to something meaningful.
  in block  at /home/andinus/.raku/sources/32E407EDE579C166DFEF1B180F5930E968008D1A (Zef::Repository::Ecosystems) line 247
Use of uninitialized value @short-names-to-index of type Any in string context.
Methods .^name, .raku, .gist, or .say can be used to stringify it to something meaningful.
  in block  at /home/andinus/.raku/sources/32E407EDE579C166DFEF1B180F5930E968008D1A (Zef::Repository::Ecosystems) line 247
===> Found 0 results

And the previous search should be returning results.

[andinus]

rm -fr ~/.zef

fixed this issue.

andinus@ ~/.zef> zef search fornax
===> Updating fez mirror: http://360.zef.pm/
===> Updated fez mirror: http://360.zef.pm/
===> Updating cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Updating p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Found 1 results
--------------------------------------------------------------------------------------------------------------------------------
ID|From                            |Package                            |Description                                             
--------------------------------------------------------------------------------------------------------------------------------
0 |Zef::Repository::Ecosystems<fez>|fornax:ver<0.1.0>:auth<zef:andinus>|Collection of tools to visualize Path Finding Algorithms
--------------------------------------------------------------------------------------------------------------------------------

[andinus]

Is this expected behavior?

[ugexe]

I'm guessing the json file for the fez ecosystem wasn't written out fully so until it was updated again zef would just see it as empty. At the very least there shouldn't be warnings though.

[andinus]

I see, but then --update should have fixed it, right? Even after multiple update runs it emitted those warnings, it only went away after rm -fr ~/.zef.

[ugexe]

Yeah that is correct.

For the ecosystems it downloads the file to ~/.zef/store/fez.json and then moves it to ~/.zef/store/fez/fez.json (or maybe its the other way around... its early). I'm not really sure what state those files could be in that a update wasn't overwriting those files despite having e.g. curld them though...

[andinus]

I ran into this again, it seems like it did download the file to ~/.zef/store/360.zef.pm and moved that to ~/.zef/store/fez/fez.json.

This was the file it downloaded:

andinus@cadmium ~/.z/s/fez> tail -c 450 ../360.zef.pm
n":"0.1.3"},{"auth":"zef:tony-o","depends":["HTTP::Server","JSON::Fast"],"description":"Middleware to parse JSON for HTTP::Server-s","dist":"HTTP::Server::Middleware::JSON:ver\u003c0.0.1\u003e:auth\u003czef:tony-o\u003e","name":"HTTP::Server::Middleware::JSON","path":"H/TT/HTTP_SERVER_MIDDLEWARE_JSON/24297dbc074dde504ab302a2152fb32a9ed10f31.tar.gz","perl":"6.*","provides":{"HTTP::Server::Middleware::JSON":"lib/HTTP/Server/Middleware/JSON.pm6"},"s⏎

What seems to be happening is that the download gets interrupted and so it writes an incomplete file to 360.zef.pm which gets copied.