A Chinese AI lab just released the best open source LLM: DeepSeek v3. I ran it through my private CFA level 1 exams to compare it against OpenAI o1, gpt-4o and Anthropic claude-sonnet.
The CFA exams were run without access to the CFA curriculum and with zero-shot inference.
Overall DeepSeek v3 delivered solid gpt-4 class performance. It even outperformed both gpt-4o and claude-sonnet in “Quantitative Methods” and “Corporate Finance” chapters.
These are amazing results because DeepSeek v3 is more than 10x cheaper to train and deploy.
DeepSeek v3 was trained on mere 2048 Nvidia H800s at a cost of $5.576M. This is in stark contrast to the 16k GPUs used to train Meta Llama 3.1, the 20k GPUs used to train Grok 2 and the 100k GPUs used to train Grok 3.
It was estimated that GPT-4 was first trained at a cost of $60-70M.
The inference costs of DeepSeek v3 are $0.27 per 1M input tokens and $1.1 per 1M output tokens. These are ~9x cheaper than gpt-4o ($2.5 per 1M input tokens, $10 per 1M output tokens) and ~13x cheaper than claude-sonnet ($3 per 1M input tokens, $15 per 1M output tokens).
We should expect the same Chinese AI lab to release a competitor to OpenAI o1 in the near future. 2025 will be a watershed moment in AI development.