[Feature Request]: 增加每一次消息的input和output token的数量的计数。

[jiangying000]

Problem Description

希望增加每一次消息的input和output token的数量的计数。

Solution Description

在每一条消息发完的时候。给一个input message的token数量的计数。 Output token可能一般数量不会太大，所以不重要。

Alternatives Considered

No response

Additional Context

No response

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Title: [Feature Request]: Increase the count of the number of input and output tokens in each message.

Problem Description

Want to increase the count of the number of input and output tokens for each message.

Solution Description

After each message is sent. Gives a count of the number of tokens in an input message. The number of Output tokens may not be too large, so it is not important.

Alternatives Considered

No response

Additional Context

No response

[daiaji]

我以前提过，毕竟也是钱，看不见是真的慌。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

I have mentioned it before, after all, it is still money, so if you don’t see it, you are really panicking.

[davyzhu]

The Tokenizer Playground
transformers.js

[Siltal]

+1