[QUERY] How to get amount of token using OpenAI

[aBabken]

Title

I want to get the total amount of tokens using openai for single task

Description

I got from openai debug

'x-ratelimit-limit-tokens': '30000'
'x-ratelimit-remaining-tokens': '18305'

I heard that its difference is exactly the amount of spent token for request, is that true?

Additional Context

But anyway I want to have an ability to get the total amount of spent token for one task

[bolt163]

u can use openai's tiktoken to encode the return result of get_dom_with_content_type() in
ae/core/skills /get_dom_with_content_type.py, which the total process can be encapsulated in a self-defined function

[teaxio]

@aBabken We use Autgen agentic framework under the covers. It makes the calls to openai. It has an accounting component, howeve, it is only partially accounting due to a bug in the way it works. We have filed this with autogen. Hoping they address it soon, if not, we will have to figure out a way to hack through the library. If you figure out a way and want to post a PR, please feel free.