
What is Whisper?
Whisper is an open-source speech recognition model released by OpenAI, used to convert audio into text, translate speech into English, and automatically detect language, including Vietnamese. According to the GitHub API (read on 04/11/2026), the openai/whisper repository was created on 16/9/2022, has about 111,900 stars, is licensed under MIT, written in Python, and the most recent code push was on 31/8/2026. According to the model card in the repository, the model was trained on 680,000 hours of audio collected from the internet, a figure announced by OpenAI and not yet verified. This post answers: what Whisper can do, how many model sizes exist, what hardware is needed, how to install it, what its limitations and risks are, and when a solo worker should use it.

Whisper takes audio, processes each 30-second window and returns text. Diagram: OPCAIOS.
What is Whisper and who created it?
Whisper is OpenAI's multitask speech recognition model, released in September 2022 with source code and weights under the MIT license. A single model that does three things: transcribe speech, translate speech into English, and detect language, replacing multiple steps of traditional speech processing pipelines.
Technically, the repository's README describes Whisper as a sequence-to-sequence Transformer model. The encoder reads the audio spectrum, the decoder generates a sequence of tokens, and "special tokens" tell the model what task it is doing: transcribing, translating, or just identifying the language. Because of this training approach, Whisper also learns to detect whether there is speech in an audio segment. According to the model card (read on 04/11/2026), the training data consists of 680,000 hours, about 65% English, with the rest spread across 98 other languages. OpenAI also published the paper "Robust Speech Recognition via Large-Scale Weak Supervision" on arXiv (code 2212.04356). Release milestones according to the model card: original version September 2022, large-v2 December 2022, large-v3 November 2023, and turbo September 2024. The most recent release on GitHub has the tag v20250625 (26/6/2025).
What can Whisper do?
Whisper can do three main things: transcribe speech from audio files into text, translate speech in other languages into English, and detect the language being spoken. Users choose one of six model sizes depending on their hardware and required accuracy. The table below summarizes according to the README (read on 04/11/2026).
| Feature | Description from README |
|---|---|
| Transcribe | Reads entire file, processes each 30-second window, returns text with timestamps by segment |
| Translate to English | Use the --task translate option with multilingual model; turbo version does not support translation |
| Language detection | detect_language function returns probability for each language |
| Six model sizes | tiny (39M parameters, ~1 GB VRAM), base (74M, ~1 GB), small (244M, ~2 GB), medium (769M, ~5 GB), large (1550M, ~10 GB), turbo (809M, ~6 GB) |
| English-only versions | tiny.en, base.en, small.en, medium.en: more accurate for English at smaller sizes |
| Command line and Python | whisper command and import whisper library |
| Input formats | Any format that ffmpeg can read: mp3, wav, flac, m4a, including video |
The README notes that turbo is an optimized version of large-v3, about 8 times faster than large with slightly reduced accuracy; speed measured on A100 GPU with English, your machine will differ. For Vietnamese, we have not yet measured it ourselves.
Is Whisper free? License and hidden costs
Whisper is free: both the source code and model weights are under the MIT license, usable for commercial purposes. The real cost lies in hardware and time: large models need GPUs with sufficient VRAM, while running on CPU is much slower. The table below summarizes it (read on 04/11/2026, may change).
| Item | Details |
|---|---|
| Software fee | 0 đồng, MIT license for code and weights |
| GPU | Not mandatory but nearly needed for medium and above; VRAM from ~1 GB (tiny) to ~10 GB (large) per README |
| Server rental | If you have no GPU, rent GPU servers by the hour; price decided by providers, we don't list it because we haven't read any official pricing table in this post |
| API key | Not needed: Whisper runs entirely locally, no calls to OpenAI |
| Download size | Model weights downloaded on first run, large version around several GB |
| Supporting software | ffmpeg (free), Python 3.8 to 3.11, PyTorch; may need Rust if tiktoken prebuilt is missing |
OpenAI also sells transcription service via API; we haven't read their pricing page so we don't mention it.
How to install Whisper and test run it?
Installing Whisper requires Python, ffmpeg, and one pip command. According to the README (read on 04/11/2026), the process has four steps.
- First, install ffmpeg because Whisper uses it to read audio files. On macOS:
brew install ffmpeg
On Ubuntu or Debian: sudo apt update && sudo apt install ffmpeg. On Windows you can use Chocolatey (choco install ffmpeg) or Scoop.
- Next, install the Python package from PyPI:
pip install -U openai-whisper
The README states the code is expected to run with Python 3.8 to 3.11 and recent PyTorch. If pip reports a tiktoken-related error, install Rust per official instructions then add ~/.cargo/bin to PATH; if setuptools_rust is missing, run pip install setuptools-rust.
- Then, transcribe your first file using the command line. The default model is
turbo:
whisper audio.mp3 --model turbo
For Vietnamese speech, you should specify the language so the model doesn't have to guess: add --language Vietnamese. To translate to English, use a multilingual model like medium or large with --task translate; turbo will return the original language because it wasn't trained to translate.
- Finally, if you need to call it from Python code, the README gives a three-line example:
whisper.load_model("turbo"),model.transcribe("audio.mp3")then readresult["text"]. The first run will download model weights to your machine.
Example (hypothetical): a solo podcaster records 40 minutes each week, runs whisper ep12.m4a --model small --language Vietnamese on a laptop without a discrete GPU, waits several tens of minutes, then uses the transcript to write episode description and blog post. Actual time depends on the machine; we haven't measured it.
Limitations, risks and legal notes when using Whisper
Whisper has three main limitations per OpenAI's own model card: it can "hallucinate" text not in the audio, accuracy varies across languages, and it's not suitable for real-time transcription out of the box. Users need to review the transcript before publishing or using it as evidence.
More specifically, the model card (read on 04/11/2026) states: output text may contain sentences the model makes up, especially in silent sections or noise; error rates are higher for languages with less training data; errors can vary by accent, gender, age; and the model sometimes repeats a phrase many times, which beam search and temperature adjustment only partly reduce. OpenAI also lists out-of-scope uses: transcribing someone else's recording without their consent, inferring human characteristics from voice, and use in high-risk decisions where errors could cause serious harm.
On personal data: the plus is Whisper runs locally, audio never leaves your machine, unlike cloud transcription services. But meeting recordings, customer call recordings are still personal data of others. Ask permission before recording and transcribing, store files with control, and delete when no longer needed. This section is for reference, not legal advice, and we haven't checked specific Vietnamese law or GDPR text.
The project releases infrequently (latest release 6/2025 per GitHub), so don't expect new features soon.
Whisper versus other options
Original Whisper is the reference model; many popular tools actually run Whisper weights themselves but optimized. The table below is a general description, not our own measurements.
| Option | How it differs from original Whisper | Good for |
|---|---|---|
| Original Whisper (openai/whisper) | Reference implementation in PyTorch, easy to understand, easy to modify | People who want to try quickly, Python programmers |
| faster-whisper, whisper.cpp | Same weights, runs faster or runs on CPU and Mac | People who need speed or don't have NVIDIA GPU |
| OpenAI transcription API | Pay per minute, no need for powerful machine, audio sent to server | People who want simplicity, willing to send data |
| Commercial transcription service (with interface) | Has editing, speaker separation, collaboration | People who don't know how to code |
| Other voice models (open source) | Different architecture, Vietnamese quality needs your own testing | People who want to compare |
In this same series of posts, we also write about Voice Pro and AI Video Transcriber, two tools with interfaces built on transcription models.
When should you use Whisper?
Use Whisper when you need to transcribe Vietnamese or English in large quantities, want data to stay on your machine, and are willing to manually fix some parts. Don't use it when you need real-time subtitles or there's no one to review the transcript.
| Situation | Recommendation |
|---|---|
| Transcribe your own podcasts, YouTube videos to write articles | Use Whisper size small or medium, specify --language Vietnamese |
| Internal meeting notes, want to keep data on your machine | Use Whisper, ask participants for permission first |
| Real-time subtitles for livestream | Not suitable; model card says Whisper is not ready for real-time |
| Transcribe hundreds of hours, no GPU | Consider faster-whisper, whisper.cpp or rent a GPU server |
| Don't know how to code, need an interface | Use an app built on Whisper or a commercial service |
| Transcript used as legal or medical evidence | Not suitable; model card classifies it as out of scope |
OPCAIOS experience
We added Whisper to the Repo list in the Utilities section on opcaios.org after reading the README and checking that the repository is still being maintained; we haven't tested it yet, so this section outlines the criteria we use to consider it, not an evaluation after using it.
The first criterion is whether the repository is still being updated: the most recent code push was 31/8/2026 according to GitHub API, the latest release in June/2025, meaning the project is in maintenance status rather than active development; for a model that's already stable, this is acceptable. The second criterion is the license: MIT for both code and weights, suitable for incorporating into internal processes or products. The third criterion is hidden costs: no software fees or API keys, but needs GPU or waiting time; we do use a rented GPU server for our OmniVoice video generation workflow, so Whisper could run on it when we need transcription. The fourth criterion is whether we can replace tools we're already using: currently we transcribe using services built into AI tools; Whisper is attractive because it keeps audio on the machine. Finally, there's risk: no issues with platform terms, only risks like hallucinations and personal data as mentioned.
Read more: What is Ollama? to run language models locally on your machine, Turn YouTube video into 10 blog posts, What is n8n? to automate transcription steps, 6 best speech-to-text tools in 2026.
Frequently Asked Questions
Does Whisper support Vietnamese?
Yes. Whisper is a multilingual model, Vietnamese is in the language list in the tokenizer.py file of the repository, and the model card publishes error rates by language on Common Voice 15 and Fleurs. However OpenAI clearly states that accuracy is uneven across languages and lower for low-resource languages. With regional Vietnamese accents, specialized terminology, or background noise, you should use medium size or larger, specify --language Vietnamese, and review the transcript. We have not measured Vietnamese quality numerically.
Does Whisper require a GPU?
Not required, but highly recommended if using medium, large, or turbo sizes. The README states VRAM needs around 1 GB for tiny and base, 2 GB for small, 5 GB for medium, 6 GB for turbo, and 10 GB for large. Running on CPU is possible with smaller sizes, just much slower. If your machine doesn't have an NVIDIA GPU, you can rent a GPU server by the hour, or use optimized versions like whisper.cpp which run better on CPU and Mac machines. Actual processing time depends on your machine and language; the README only gives relative numbers measured on an A100.
Does Whisper send my audio to OpenAI?
No. The openai/whisper repository runs entirely on your machine: model weights are downloaded once, then all computation happens locally, no API key is needed, and there are no calls to OpenAI servers. This is the main difference from OpenAI's paid transcription API, where audio files are sent to the cloud. If you use Whisper through a third-party application, check where that application sends data, since their policies are not covered by Whisper's MIT license.
How does Whisper differ from faster-whisper and whisper.cpp?
Original Whisper is the PyTorch reference implementation released by OpenAI. faster-whisper and whisper.cpp are community projects using the same weights but rewriting the model execution to be faster, use less memory, or run on CPU and Apple Silicon. For transcription quality, they share the same foundation so they should be equivalent in principle, but segmentation and default parameters may differ. This is a general description, not our measurement; if you need speed, try both on your own files.
Can someone without coding knowledge use Whisper?
Yes, at the command line level: after installing ffmpeg and running pip install -U openai-whisper, you just need one command whisper file.mp3 --language Vietnamese to get a transcript in txt, srt, or vtt format. If the command line is still a barrier, choose an application with a UI built on Whisper, or ask an AI agent like Claude Code to install and run it for you, then review the result. This approach, using AI to operate technical tools without writing code yourself, is what we teach in OPCAIOS's AI application training course.
Source: GitHub openai/whisper (README, model-card.md, API, release page, read 04/11/2026); not read/not verified: Whisper introduction blog post on openai.com (page returns 403 error when accessed), OpenAI transcription API pricing page, arXiv paper 2212.04356; star count and pricing may change. Experience is from OPCAIOS.
Translated from Vietnamese with AI. Read the original.
Want to apply this in your business?
Book a free 30-minute call (Vietnam time, in English or Vietnamese).


