Loading a dataset cached in a LocalFileSystem is not supported

🤗Datasets
1

Hey i am new to huggingface and i got an error in this piece of code:
data_files = {
“train”: “/content/drive/MyDrive/fine_tuning_medroberta/train.txt”,
“validation”: “/content/drive/MyDrive/fine_tuning_medroberta/dev.txt”,
“test”: “/content/drive/MyDrive/fine_tuning_medroberta/test.txt”
}

dataset = load_dataset(“text”, data_files=data_files, split={
“train”: “train”,
“validation”: “validation”,
“test”: “test”
})

i get the following error:
NotImplementedError Traceback (most recent call last)
in <cell line: 0>()
5 }
6
----> 7 dataset = load_dataset(“text”, data_files=data_files, split={
8 “train”: “train”,
9 “validation”: “validation”,

1 frames
/usr/local/lib/python3.11/dist-packages/datasets/builder.py in as_dataset(self, split, run_post_process, verification_mode, ignore_verifications, in_memory)
1171 is_local = not is_remote_filesystem(self._fs)
1172 if not is_local:
→ 1173 raise NotImplementedError(f"Loading a dataset cached in a {type(self._fs).name} is not supported.")
1174 if not os.path.exists(self._output_dir):
1175 raise FileNotFoundError(

NotImplementedError: Loading a dataset cached in a LocalFileSystem is not supported.

can anyone help?

1 Like
2

I found several similar cases.

Try like this:

dataset = load_dataset(“text”, data_files=data_files, download_mode="force_redownload", split={
    “train”: “train”,
    “validation”: “validation”,
    “test”: “test”
})

and just in case:

pip install -U datasets
3

the pip install dataset worked, thank you so much!!!

1 Like