updated the notebook and bar chart

cahya-wirawan · Jun 9, 2024 · a17fd84 · a17fd84
1 parent b263e39
commit a17fd84
Show file tree

Hide file tree

Showing 3 changed files with 158 additions and 210 deletions.
diff --git a/README.md b/README.md
@@ -39,7 +39,7 @@ We did a performance comparison on [the simple English Wikipedia dataset 2022030
 - The Rust RWKV tokenizer
 
 The comparison is done using [this jupyter notebook](tools/rwkv_tokenizers.ipynb) in an M2 Mac mini. The Rust RWKV tokenizer 
-is around 8x faster than the original tokenizer and 4.8x faster than OpenAI Tiktoken.
+is around 8.2x faster than the original tokenizer and 4.9x faster than OpenAI Tiktoken.
 
 ![performance-comparison](data/performance-comparison.png)
 

diff --git a/data/performance-comparison.png b/data/performance-comparison.png