Attempt to use os.walk to speedup reading from Directory Store

[Carreau]

usage of os.walk for tree (os.scandir for folders) is faster than
listdir as it avoids many stats call.

This Should make the DirectoryStore faster.

See #562

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• AppVeyor and Travis CI passes
• Test coverage is 100% (Coveralls passes)

[pep8speaks]

Hello @Carreau! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-09-14 16:16:45 UTC

[joshmoore]

Hi @Carreau. Did you mean for this to be proactively merged? I may be missing something but based on https://github.com/zarr-developers/governance/blob/master/GOVERNANCE.md#decision-making-process I'd think this needed a ":thumbsup:" perhaps especially after a force-pushed. Sorry if I'm missing something. After putting the governance doc in place, there likely should have been a discussion about GitHub settings. I'll kick that off separately.

Edit: see zarr-developers/governance#12

[Carreau]

Sorry for the discussion on Friday I thought that it was or to try to move some of these small changes.

The force push was a rebase to make sure this was tested on a more recent version of the repo. I'm happy to get that reverted.

[joshmoore]

You can have a post-hoc zombie 👍 (necro-review?) from me. More a question of the process moving forward. Don't hesitate to ping and holler on a PR if you need motion.