Importing from other tools¶
add takes a reference to data (a URL or DOI); import ingests another
tool's catalog file. Both produce standard manifest entries, and
already-downloaded files are adopted in place — verified against their
checksum and recorded, with no re-download:
datamanifest import pooch registry.txt --base-url URL --cache-dir DIR # adopts pooch's cache
datamanifest import csv files.csv # a CSV with url (+ name, sha256) columns
datamanifest import urls list.txt --base-url URL # a plain list of URLs
datamanifest import intake catalog.yml # an intake catalog ([yaml] extra)
datamanifest import dvc path-or-dir # *.dvc / dvc.lock (+ .dvc/cache)
Per-source detail, and the add-side sources (direct URLs, Zenodo and PANGAEA
DOIs), are on the adding datasets page. Importing is a
CLI operation: the importers are not part of the public
Python API.
Coming from Pooch¶
If you use Pooch, one command converts the registry and adopts your downloaded files in place:
datamanifest import pooch registry.txt \
--cache-dir "$(python -c 'import pooch; print(pooch.os_cache("yourpkg"))')"
datamanifest covers the same fetch-and-verify workflow, with an explicit,
cross-language manifest file, a dataset-lifecycle CLI, and the
@cached decorator for your own computed
results. See related projects for a comparison with Pooch,
intake, and pystow.