Type of the acoustic model.

Configuration for the Wav2Vec model.

Configuration for the Wav2Vec model using Triton.

Configuration for the Whisper model.

Configuration for the Whisper model using Triton.

Amazon web service cloud service inference settings.

Deepgram cloud service inference settings.

Google cloud service inference settings.

Microsoft Azure cloud service inference settings.

Configuration for the Parakeet model.

Name of the language model to which to add data. Example: "user_lm_1"

Zip file containing data in the form of text files. Example: A zip file with text files containing sentences or phrases in the target language.

Path to the checkpoint file.

Name of the language model to create. Example: "user_lm_1"

Name of the language model to delete. Example: "user_lm_1"

A pipeline ID. Example: "pipeline_1"

A list of all available language models for the corresponding pipeline ID. Example: ["model_1", "model_2"]

Path to the directory of language models.

Beam size for the search algorithm.

Default language model to be selected if none is given.

Weight for the language model scorer (alpha).

Weight for the word insertion penalty (beta).

Filter for languages

Filter for pipeline owner

domains available. Example: ["medical", "finance"]

List of pipeline IDs to retrieve their available language models. Example: ["pipeline_1", "pipeline_2"]

Response is a list of LanguageModelPipelineId, where each element contains a pipeline ID and its associated language models. Example: [{pipeline_id: "pipeline_1", model_names: ["model_1", "model_2"]}, {pipeline_id: "pipeline_2", model_names: ["model_3"]}]

Filter for domains. Example: ["medical", "finance"]

Filter for pipeline owners. Example: ["ondewo", "partner_company"]

available languages

Optional. Define the language.

Required. Representation of a list of normalization pipelines configurations. Retrieved by ListS2tNormalizationPipelines, containing the configurations of normalization pipelines with the specifications received in the ListS2tNormalizationPipelinesRequest.

Filter for languages

Filter for pipeline owners

Filter for domains

If true, return only registered pipelines. Default false: return registered and persisted (from config files) configs.

A list of Speech2TextConfig message instances containing the configuration of each pipeline. Example: [{id: "pipeline_1", description: {language: "en"}, active: true, ...}, {id: "pipeline_2", description: {language: "fr"}, active: true, ...}]

Type of logging.

Path for logging.

Optional. The API key used to authenticate with the OpenAI API.

Optional. The organization ID associated with the OpenAI account, used for billing and access control.

Optional. The project ID associated with the OpenAI account.

Optional. The webhook secret used to validate incoming webhook events from OpenAI.

Optional. The base URL for the OpenAI API. Overrides the default endpoint, useful for proxies or compatible third-party providers.

Optional. The base URL for OpenAI WebSocket connections. Overrides the default WebSocket endpoint.

Optional. The timeout in seconds for requests to the OpenAI API. Applies to the entire request lifecycle including connection, sending, and receiving.

Optional. The maximum number of retries to attempt when a request fails due to a transient error. Defaults to 2.

Optional. Default HTTP headers to include with every request to the OpenAI API.

Optional. Default query parameters to append to every request URL sent to the OpenAI API. Values can be of any type (string, number, boolean, list), hence the use of Struct.

Optional. If true, enables strict validation of response payloads returned by the OpenAI API.

Required. The name or identifier of the OpenAI model to use for chat completion (e.g., "gpt-4o", "gpt-4o-mini", "o3").

Optional. A number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the likelihood of the model repeating the same line verbatim.

Optional. Modifies the likelihood of specified tokens appearing in the completion. Maps token IDs (as strings) to bias values from -100 to 100. Mathematically added to the logits before sampling.

Optional. Whether to return log probabilities of the output tokens. If true, returns the log probabilities of each output token in the response.

Optional. An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.

Optional. The maximum number of tokens that can be generated in the chat completion. Deprecated in favor of max_completion_tokens.

Optional. Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.

Optional. The number of chat completion choices to generate for each input message. Note that costs are multiplied by the number of choices generated.

Optional. A number between -2.0 and 2.0. Positive values penalize new tokens based on whether they have already appeared in the text, increasing the likelihood of the model discussing new topics.

Optional. Used by OpenAI to cache responses for similar requests to optimize your cache hit rates. Replaces the `user` field. [Learn more](https://platform.openai.com/docs/guides/prompt-caching).

Optional. Constrains effort on reasoning for [reasoning models](https://platform.openai.com/docs/guides/reasoning). Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response.

Optional. If specified, the system will make a best effort to sample deterministically given the same seed and parameters, enabling reproducible outputs.

Optional. Specifies the processing type used for serving the request. - If set to 'auto', then the request will be processed with the service tier configured in the Project settings. Unless otherwise configured, the Project will use 'default'. - If set to 'default', then the request will be processed with the standard pricing and performance for the selected model. - If set to '[flex](https://platform.openai.com/docs/guides/flex-processing)' or '[priority](https://openai.com/api-priority-processing/)', then the request will be processed with the corresponding service tier. - When not set, the default behavior is 'auto'. When the `service_tier` parameter is set, the response body will include the `service_tier` value based on the processing mode actually used to serve the request. This response value may be different from the value set in the parameter.

Optional. Not supported with latest reasoning models `o3` and `o4-mini`. Up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.

Optional. Whether to store the output of this chat completion request for use in model distillation, evals, or the stored completions dashboard.

Optional. What sampling temperature to use, between 0 and 2. Higher values (e.g., 0.8) make the output more random, while lower values (e.g., 0.2) make it more focused and deterministic.

Optional. An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. Requires logprobs to be true.

Optional. An alternative to sampling with temperature, called nucleus sampling. The model considers only the tokens with top_p probability mass. Ranges from 0 to 1.

Optional. This field is being replaced by `safety_identifier` and `prompt_cache_key`. Use `prompt_cache_key` instead to maintain caching optimizations. A stable identifier for your end-users. Used to boost cache hit rates by better bucketing similar requests and to help OpenAI detect and prevent abuse. [Learn more](https://platform.openai.com/docs/guides/safety-best-practices#safety-identifiers).

Optional. Constrains the verbosity of the model's response. Lower values will result in more concise responses, while higher values will result in more verbose responses. Currently supported values are `low`, `medium`, and `high`.

Optional. Additional HTTP headers to send with the request. These are merged with and override default_headers for this specific request only.

Optional. Additional query parameters to send with the request. These are merged with and override default_query for this specific request only.

Optional. Additional JSON properties to include in the request body. Useful for accessing new or undocumented API parameters.

Name of the Triton model.

Version of the Triton model.

Timeout for checking model status.

Host name of triton inference server that serves the Parakeet model

Port number of triton inference server that serves the Parakeet model

List of names of active post-processors.

Post-processor configurations.

Whether to use spelling correction

Whether to disable normalization

Post-processing configuration specifying the active post-processors in the pipeline, as well as their individual configuration. If not set, all values are replaced by the ones in current pipeline.

Whether to disable LLM post-processing

Configuration of the SymSpell spelling correction.

Configuration of the normalization object.

Configuration of the LLM post-processing.

Path to the PT files.

Step for the PT files.

Full name of the Pyannote model.

Minimum audio size for processing.

Fill inactive regions shorter than that many seconds. Example notebook

Remove active regions shorter than that many seconds. Example notebook

Host name of triton inference server that serves the Pyannote model

Port number of triton inference server that serves the Pyannote model

Optional. Configuration for Amazon web service speech-to-text provider.

Optional. Configuration for DeepGram speech-to-text provider.

Optional. Configuration for Google speech-to-text provider.

Optional. Configuration for Microsoft Azure speech-to-text provider.

Optional. Enables or disables partial_results_stabilization feature. More details at: https://docs.aws.amazon.com/transcribe/latest/dg/streaming-partial-results.html#streaming-partial-result-stabilization

Optional. You can use this field to set the stability level of the transcription results. A higher stability level means that the transcription results are less likely to change. Higher stability levels can come with lower overall transcription accuracy. Acceptable values: ["low", "medium", "high"]. Defaults to "high" if not set explicitly. More details at: https://aws.amazon.com/blogs/machine-learning/amazon-transcribe-now-supports-partial-results-stabilization-for-streaming-audio/

Optional. The name of your customize language model you want to use. More details at: https://docs.aws.amazon.com/transcribe/latest/dg/custom-language-models.html

Optional. The name of your customize vocabulary you want to use. More details at: https://docs.aws.amazon.com/transcribe/latest/dg/custom-vocabulary.html

Optional. Enables or disables punctuate feature of Deepgram to add punctuations to the resulted transcript. More details at: https://developers.deepgram.com/docs/punctuation

Optional. Enables or disables smart_format feature of Deepgram transcription result to improve readability. More details at: https://developers.deepgram.com/docs/smart-format

Optional. Enables or disables numerals feature of Deepgram to convert numbers to numeric form in the resulted transcript. More details at: https://developers.deepgram.com/docs/numerals

Optional. Enables or disables measurements feature of Deepgram to convert measurement units (i.e. Kilogram) to abbreviated form (i.e. Kg) in the resulted transcript. More details at: https://developers.deepgram.com/docs/measurements

Optional. Enables or disables dictation feature of Deepgram to convert spoken dictation commands into their corresponding punctuation marks. More details at: https://developers.deepgram.com/docs/dictation

Optional. Enables or disables automatic_punctuation feature of Google s2t to add punctuations to the resulted transcript. More details at: https://cloud.google.com/speech-to-text/docs/automatic-punctuation

Optional. Enables or disables word_time_offsets feature of Google s2t to add word-level timestamps (time-offset) to the resulted transcript. More details at: https://cloud.google.com/speech-to-text/docs/async-time-offsets

Optional. Enables or disables word_confidence feature of Google s2t to add word-level confidence scores to the resulted transcript. More details at: https://cloud.google.com/speech-to-text/docs/word-confidence

Optional. Enables or disables transcript_normalization feature of Google s2t to automatically replace parts of the transcript with phrases of your choosing. More details at: https://cloud.google.com/speech-to-text/v2/docs/reference/rpc/google.cloud.speech.v2#transcriptnormalization

Optional. Maximum number of recognition hypotheses to be returned, may be returned fewer than max_alternatives. Valid values are 0-30. A value of 0 or 1 will return a maximum of one. If omitted, will return a maximum of one.

Optional. Enables or disables the Microsoft Azure fast transcription API. It is faster than SDK but is in preview version. More details at: https://learn.microsoft.com/en-us/azure/ai-services/speech-service/fast-transcription-create

Optional. Enables or disables the detailed format for the result of Microsoft Azure s2t service to add timestamps and confidences to the resulted transcript.

Language of the audio to transcribe by Amazon web service s2t cloud service. It should be 4-letter language code (BCP-47) e.g. 'en-US' or 'de-DE'.

Specifies if streaming mode of Amazon web service speech to text is available for the selected language, otherwise batch mode transcription is used. See the list of languages and available transcription modes at: https://docs.aws.amazon.com/transcribe/latest/dg/supported-languages.html

Enables or disables partial_results_stabilization feature. More details at: https://docs.aws.amazon.com/transcribe/latest/dg/streaming-partial-results.html#streaming-partial-result-stabilization

You can use this field to set the stability level of the transcription results. A higher stability level means that the transcription results are less likely to change. Higher stability levels can come with lower overall transcription accuracy. Defaults to "high" if not set explicitly.

The name of your customize language model you want to use. More details at: https://docs.aws.amazon.com/transcribe/latest/dg/custom-language-models.html

The name of your customize vocabulary you want to use. More details at: https://docs.aws.amazon.com/transcribe/latest/dg/custom-vocabulary.html

Model name from one of the speech-to-text models provided by Deepgram for the desired use-case. Provided model names and details at: https://developers.deepgram.com/docs/model

Language of the audio to transcribe by Deepgram s2t cloud service. It should be 4-letter language code (BCP-47) e.g. 'en-US' or 'de-DE'.

Enables or disables punctuate feature of Deepgram to add punctuations to the resulted transcript. More details at: https://developers.deepgram.com/docs/punctuation

Enables or disables smart_format feature of Deepgram transcription result to improve readability. More details at: https://developers.deepgram.com/docs/smart-format

Enables or disables numerals feature of Deepgram to convert numbers to numeric form in the resulted transcript. More details at: https://developers.deepgram.com/docs/numerals

Enables or disables measurements feature of Deepgram to convert measurement units (i.e. Kilogram) to abbreviated form (i.e. Kg) in the resulted transcript. More details at: https://developers.deepgram.com/docs/measurements

Enables or disables dictation feature of Deepgram to convert spoken dictation commands into their corresponding punctuation marks. More details at: https://developers.deepgram.com/docs/dictation

Model name from one of the speech-to-text models provided by Google for the desired use-case. Provided model names and details at: https://cloud.google.com/speech-to-text/docs/transcription-model

Language of the audio to transcribe by Google s2t cloud service. It should be 4-letter language code (BCP-47) e.g. 'en-US' or 'de-DE'.

Enables or disables automatic_punctuation feature of Google s2t to add punctuations to the resulted transcript. More details at: https://cloud.google.com/speech-to-text/docs/automatic-punctuation

Enables or disables word_time_offsets feature of Google s2t to add word-level timestamps (time-offsets) to the resulted transcript. More details at: https://cloud.google.com/speech-to-text/docs/async-time-offsets

Enables or disables word_confidence feature of Google s2t to add word-level confidence scores to the resulted transcript. More details at: https://cloud.google.com/speech-to-text/docs/word-confidence

Enables or disables transcript_normalization feature of Google s2t to automatically replace parts of the transcript with phrases of your choosing. More details at: https://cloud.google.com/speech-to-text/v2/docs/reference/rpc/google.cloud.speech.v2#transcriptnormalization

Maximum number of recognition hypotheses to be returned. The server may return fewer than max_alternatives. Valid values are 0-30. A value of 0 or 1 will return a maximum of one. If omitted, will return a maximum of one.

Language of the audio to transcribe by Microsoft Azure s2t cloud service. It should be 4-letter language code (BCP-47) e.g. 'en-US' or 'de-DE'.

Enables or disables the Microsoft Azure fast transcription API. It is faster than SDK but is in preview version. More details at: https://learn.microsoft.com/en-us/azure/ai-services/speech-service/fast-transcription-create

Enables or disables the detailed format for the result of Microsoft Azure s2t service to add timestamps and confidences to the resulted transcript.

Language of the speech-to-text system.

Owner of the pipeline.

Domain of the speech-to-text system.

Comments about the system.

Version number based on semantic versioning, e.g. "4.2.0".

Configuration for the acoustic models.

Configuration for the language models.

Configuration for the inference backend.

Optional. Configuration options for the OpenAI client used for post-processing.

Optional. System prompt used to guide the LLM model for post-processing.

Optional. Ending prompt appended to the accumulated prompts of the active post-processing tasks.

Optional. Configuration of the options to casing task in LLM post-processing.

Optional. Configuration of the options to punctuation task in LLM post-processing.

Optional. Configuration of the options to spelling-correction task in LLM post-processing.

Optional. Configuration of the options to semantic-correction task in LLM post-processing.

Optional. Configuration of the options to translation task in LLM post-processing.

Optional. Configuration of the options to inverse-normalization task in LLM post-processing.

Optional. Configuration of the options to normalization task in LLM post-processing.

Optional. Configuration of the options to summarization task in LLM post-processing.

Optional. Configuration of the options to user-prompt task in LLM post-processing.

Optional. Indicates if the inverse-normalization task of LLM post-processing is active.

Optional. Configuration of the inverse-normalization of email address sub-task.

Optional. Configuration of the inverse-normalization of phone number sub-task.

Optional. Configuration of the inverse-normalization of date and time sub-task.

Optional. Configuration of the inverse-normalization of credit card number sub-task.

Optional. Configuration of the inverse-normalization of social security number sub-task.

Optional. Configuration of the inverse-normalization of time zone sub-task.

Optional. Indicates if the normalization task of LLM post-processing is active.

Optional. Configuration of the normalization of email address sub-task.

Optional. Configuration of the normalization of phone number sub-task.

Optional. Configuration of the normalization of date and time sub-task.

Optional. Configuration of the normalization of credit card number sub-task.

Optional. Configuration of the normalization of social security number sub-task.

Optional. Configuration of the normalization of time zone sub-task.

Optional. Indicates if this sub-task is active.

Optional. Custom prompt to guide the LLM for this sub-task. Overrides the default prompt when set.

Optional. Indicates if the summarization task of LLM post-processing is active.

Optional. Custom prompt to guide the LLM for summarization. Overrides the default prompt when set.

Optional. Minimum number of characters of the summary generated in summarization task of LLM post-processing.

Optional. Maximum number of characters of the summary generated in summarization task of LLM post-processing.

Optional. Indicates if the translation task of LLM post-processing is active.

Optional. Target language of the translation task of LLM post-processing.

Optional. Custom prompt to guide the LLM for translation. Overrides the default prompt when set.

Language for normalization of transcriptions.

List of names of active normalizations.

id of the model that will generate audio

Full name of the Silero model.

Minimum audio size for processing.

Speech probability, in [0, 1], above which a frame counts as speech. A run of speech ends at the hysteresis threshold threshold - 0.15, as it does upstream, so this sets both the onset and - through that offset - the release point. Optional, and explicitly so: 0 is a legitimate value here, and without presence tracking it would be indistinguishable from an unset field and silently replaced by the default.

Speech runs shorter than this many milliseconds are discarded. A run still open at the end of the buffer is kept regardless, since more audio may extend it. Optional for the same reason as threshold: 0 means 'discard nothing'.

Silence, in milliseconds, that must follow the last speech before an utterance is declared to have ended. Optional for the same reason as threshold: 0 means 'end the utterance as soon as the speech stops'.

Padding, in milliseconds, added on each side of the detected boundary. Optional for the same reason as threshold: 0 means 'no padding'.

Host name of triton inference server that serves the Silero model

Port number of triton inference server that serves the Silero model

Unique identifier for the configuration.

Description of the speech-to-text system.

Indicates if the configuration is active.

Configuration for inference models.

Configuration for the streaming server.

Configuration for voice activity detection.

Configuration for post-processing.

Configuration for logging.

Hostname of the streaming server.

Port number of the streaming server.

Output style for the streaming server.

Configuration for streaming speech recognition.

Indicates whether to transcribe non-final results.

Decoding method for speech recognition.

Sampling rate for audio input.

Minimum audio chunk size for processing.

Timeout between audio chunks; if exceeded, the stream will be stopped.

Configuration of the options to turn-detection in utterances

Path to the dictionary used by symspell

The maximal edit-distance to consider for spelling correction (affects performance - bigger number takes longer to process!)

The length (number of characters) of the prefix to consider for filtering

Name of the language model to train. Example: "user_lm_1"

Order n of the ngram. Example: 3 (for trigram model)

wav file to transcribe

The configuration to override the default configuration

List of transcriptions with confidence level

The time the transcription took

id of the transcribed audio file

Required. id of the pipeline (model setup) that will generate audio

Optional. decoding type

Name of the language model

The postprocessing options

The utterance detection options

Voice activity detection with pyannote

The transcribe return options

Optional. Specify language of transcription to return

Optional. Specify task of s2t model, e.g. 'transcribe' and 'translate'

Optional. s2t_service_config provides the configuration of the service such as API key, bearer tokens, JWT, and other header information as key value pairs, e.g.,

MY_API_KEY='LKJDIFe244LKJOI'

• aws_access_key_id (required) Access key id to access Amazon Web Service.
• aws_secret_access_key (required) Secret access key to access Amazon Web Service.
• region (required) Region name of Amazon Server.

• api_key (required) API key of Deepgram account to access Deepgram S2T service.

• api_key (required) API key of Google cloud to access its S2T service.
• api_endpoint (optional) Regional API endpoint of Google cloud S2T service. (Defaults to 'eu-speech.googleapis.com')

• subscription_key (required) Subscription key to access Microsoft Azure Service.
• region (required) Region name of Microsoft Azure Server.

Optional. Defines the cloud provider's specific configuration for using speech to text cloud services. The default value is None.

wav file to transcribe

if it's the final chunk of the stream

The configuration to override the default configuration

Whether or not to mute the audio signal. Defaults to false.

List of transcriptions with confidence level

The time the transcription took

Whether or not this transcription is final (transcribed texts might change if transcription is started before the end of an utterance).

is audio bytes of the utterance in response

audio bytes of the transcribed utterance

is it a start of the utterance

id of the transcribed audio file

The configuration for the transcription

The transcribed text

The corresponding confidence score. The confidence estimate between 0.0 and 1.0. A higher number indicates an estimated greater likelihood that the recognized words are correct.

List of the words of transcription with their confidence scores and probable alternatives

List of alternative transcriptions, confidence scores, words timings and alternative words

The alternative transcribed text

The corresponding confidence score to the alternative transcript.

A list of word-specific information for each recognized word, including word timings, confidence score of the word and alternative words.

should server make response indicating that the beginning of the speech was detected

should s2t server return audio bytes of transcribed utterance

Whether or not to return confidence scores

Whether or not to return alternative results from beam-search

Optional. Number of alternative transcriptions results from beam-search or greedy-search

Whether or not to return alternative results from beam-search

Optional. Number of alternative words to results

Optional. Whether or not to return timestamps of start and end of the words. Only used in TranscribeFile.

Optional. Indicates if the turn-detection feature is active.

Optional. Whether to transcribe the whole utterance when turn moment is detected. It is helpful to increase accuracy of transcriptions in cost of drop in speed. If deactivated, it just transcribe from last short silence period and concatenates the transcriptions of small audio chunks between tiny silences.

Optional. System prompt used to guide the turn-detection model.

Optional. User prompt used as input to the turn-detection model.

Optional. Configuration options for the OpenAI client used for turn detection.

Return also immediate transcription results

if time between audio chunks exceeds next_chunk_timeout, stream will be stopped

The turn detection options

Deprecated in favour of vad_method, but still honoured so that configurations written before that field existed keep working unchanged. One of 'pyannote' or 'silero'. Read only when vad_method is VAD_METHOD_UNSPECIFIED.

Sampling rate for voice activity detection.

Configuration for the Pyannote model. Read when Pyannote is the resolved VAD or TSD method.

Configuration for the Silero model. Read when Silero is the resolved VAD method.

Configuration for WeSpeaker target-speaker detection. Read when WeSpeaker is the resolved TSD method.

Which model splits the stream into utterances. Takes precedence over active; leave unset to keep using active.

Which model decides whether an utterance came from the main speaker. Independent of vad_method, except that TSD_METHOD_PYANNOTE requires Pyannote to also be the active VAD. Leave unset to keep the behaviour of older configurations.

Path to the model.

Indicates if GPU is used.

Path to the processor.

Name of the Triton model.

Version of the Triton model.

Timeout for checking model status.

Host name of triton inference server that serves the Wav2VecTriton model

Port number of triton inference server that serves the Wav2VecTriton model

Indicates if target-speaker detection is active.

Full name of the WeSpeaker model.

Host name of triton inference server that serves the WeSpeaker model

Port number of triton inference server that serves the WeSpeaker model

Cosine similarity, in [-1, 1], above which a candidate utterance is judged to come from the same speaker as the reference. Optional, and explicitly so: 0 sits in the middle of the valid range, and without presence tracking it would be indistinguishable from an unset field.

Utterances shorter than this many seconds carry too little speaker information to judge, and are treated as undecided rather than rejected. Optional for the same reason as similarity_threshold: 0 means 'judge every utterance, however short'.

The reference audio is cropped to its most recent this-many seconds before being embedded.

Path to the model.

Indicates if GPU is used.

Default language of the model.

Default task of the model.

Path to the processor.

Name of the Triton model.

Version of the Triton model.

Timeout for checking model status.

Default language of the model.

Default task of the model. E.g., transcribe, translate, etc.

Host name of triton inference server that serves the WhisperTriton model

Port number of triton inference server that serves the WhisperTriton model

The recognized word corresponding to this set of information.

The corresponding confidence score to the alternative word.

The start time of the spoken word relative to the beginning of the audio. The accuracy of the time offset can vary, and this is an experimental feature.

The end time of the spoken word relative to the beginning of the audio. The accuracy of the time offset can vary, and this is an experimental feature.

The recognized word corresponding to this set of information.

The corresponding confidence score to the word.

List of alternative words and confidence scores of each.