The identifier to rate limit, like a user ID or an IP address
Ratelimit options
Optional requestOptional tokenRun when the chain starts running.
This method is called multiple times during a chain execution.
To ensure it only runs once, it checks and updates a _checked state.
Serialized chain
Chain input values
UpstashRatelimitError - If the request rate limit is reached
Run when the LLM ends running.
If the includeOutputTokens is set to true, the number of tokens
in the LLM completion are counted for rate limiting.
LLM result output
Optional _parentRunId: stringOptional _tags: string[]Error - If the LLM response does not include required token usage information
Run when the LLM starts running.
Serialized LLM
Prompts passed to the LLM
Optional _parentRunId: stringOptional _extraParams: Record<string, unknown>Optional _tags: string[]Optional _metadata: Record<string, unknown>Optional _name: stringUpstashRatelimitError - If the token rate limit is reached
Creates a new UpstashRatelimitHandler object with the same ratelimit configurations but with a new identifier if it's provided.
Also resets the state of the handler.
Optional identifier: stringOptional new identifier to use for the new handler instance
New UpstashRatelimitHandler instance
Generated using TypeDoc
Callback to handle rate limiting based on the number of requests or the number of tokens in the input.
It uses Upstash Ratelimit to track the rate limit which utilizes Upstash Redis to track the state.
Should not be passed to the chain when initializing the chain. This is because the handler has a state which should be fresh every time invoke is called. Instead, initialize and pass a handler every time you invoke.