Get Error/Applying Univariate Time Series Dataset

[shathaa1983]

Hi, I am trying to use a univariate time series dataset. I got this error:

KeyError Traceback (most recent call last)
~\anaconda3\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
3079 try:
-> 3080 return self._engine.get_loc(casted_key)
3081 except KeyError as err:

pandas_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 521170

The above exception was the direct cause of the following exception:

KeyError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_17036/184118730.py in
9 running_loss = 0
10 with tqdm(total=len(dataloader_train.dataset), desc=f"[Epoch {idx_epoch+1:3d}/{EPOCHS}]") as pbar:
---> 11 for idx_batch, (x, y) in enumerate(dataloader_train):
12 try:
13 print(f'idx_batch {dataloader_train} is {enumerate[dataloader_train]}')

~\anaconda3\lib\site-packages\torch\utils\data\dataloader.py in next(self)
519 if self._sampler_iter is None:
520 self._reset()
--> 521 data = self._next_data()
522 self._num_yielded += 1
523 if self._dataset_kind == _DatasetKind.Iterable and \

~\anaconda3\lib\site-packages\torch\utils\data\dataloader.py in _next_data(self)
559 def _next_data(self):
560 index = self._next_index() # may raise StopIteration
--> 561 data = self._dataset_fetcher.fetch(index) # may raise StopIteration
562 if self._pin_memory:
563 data = _utils.pin_memory.pin_memory(data)

~\anaconda3\lib\site-packages\torch\utils\data_utils\fetch.py in fetch(self, possibly_batched_index)
42 def fetch(self, possibly_batched_index):
43 if self.auto_collation:
---> 44 data = [self.dataset[idx] for idx in possibly_batched_index]
45 else:
46 data = self.dataset[possibly_batched_index]

~\anaconda3\lib\site-packages\torch\utils\data_utils\fetch.py in (.0)
42 def fetch(self, possibly_batched_index):
43 if self.auto_collation:
---> 44 data = [self.dataset[idx] for idx in possibly_batched_index]
45 else:
46 data = self.dataset[possibly_batched_index]

~\anaconda3\lib\site-packages\torch\utils\data\dataset.py in getitem(self, idx)
309
310 def getitem(self, idx):
--> 311 return self.dataset[self.indices[idx]]
312
313 def len(self):

~\anaconda3\lib\site-packages\pandas\core\frame.py in getitem(self, key)
3022 if self.columns.nlevels > 1:
3023 return self._getitem_multilevel(key)
-> 3024 indexer = self.columns.get_loc(key)
3025 if is_integer(indexer):
3026 indexer = [indexer]

~\anaconda3\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
3080 return self._engine.get_loc(casted_key)
3081 except KeyError as err:
-> 3082 raise KeyError(key) from err
3083
3084 if tolerance is not None:

KeyError: 521170

I'd appreciate it if you let me know if your code is suitable for the univariate time series. And how to solve this error?
I used the code in this link:
https://github.com/maxjcohen/transformer

Thanks

[maxjcohen]

Hi, this seems to be an issue with Pandas, as you can see in the last stack of the Traceback. Did you try to feed a pandas Dataframe directly to the trainer ? You most likely need to modify the dataloader class to match your dataset.

I currently use a custom dataset defined here. For an example of dataloader definition, see

transformer/training.py

Lines 52 to 57 in bea1c92

	dataloader_train = DataLoader(dataset_train,
	batch_size=BATCH_SIZE,
	shuffle=True,
	num_workers=NUM_WORKERS,
	pin_memory=False
	)

The documentation for both dataset and dataloader can be found, along with examples, on the PyTorch documentation.

[shathaa1983]

I tried to feed the panda's data frame and also NumPy arrays but without luck. So your suggestion to transfer my time series dataset into supervised then reframed to have input and output data then modify the data loader class to suit the dataset.
Another question, Is this code suitable for time series regression?

[maxjcohen]

It sounds like a lot of words, but don't worry it's not that hard.

Yes, the code was designed for regression, although it can also be used for classification (see #18).