Skip to content

Commit

Permalink
updated README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
cahya-wirawan committed Sep 5, 2024
1 parent 663493e commit 3abca77
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,8 @@ https://huggingface.co/datasets/cahya/simple-wikipedia/resolve/main/simple-wikip

We also created the [json2bin](https://github.com/cahya-wirawan/json2bin) application to convert datasets from JSONL format
into binidx format, a data format used for training RWKV models. It uses multithreading to scale up the performance and
can convert a dataset more than 70 times faster (around 360 MB/s) than the original json2binidx program written in Python.
can convert a dataset more than 70 times faster (around 360 MB/s) than the original
[json2binidx_tool](https://github.com/Abel2076/json2binidx_tool) written in Python.

## Changelog
- Version 0.9.0
Expand Down

0 comments on commit 3abca77

Please sign in to comment.