29 The Open-Source Community Is Competing With Private Models Challenging closed-source models from OpenAI and Google, the open-source community and its corporate champion, Meta, are democratizing access to generative AI. On balance, the performance of open-source models is improving faster than that of closed-source models, helped recently by models from China. Open Source vs Private Models 5-Shot MMLU Performance E Private Open Source C Private (Doesn't Outperform Previous Models on 5-Shot MMLU) Open Source (Doesn't Outperform Previous Models on 5-Shot MMLU) N E 1.0 G GPT-4 (OpenAI) I L Flan-PaLM 2 (Alphabet) Gemini Ultra (Alphabet) L ance 0.9 E Claude 2 (Anthropic) T m N r 0.8 PaLM-2 (Alphabet) I o f Qwen-72B (Alibaba, China) r Claude 1.3 (Anthropic) AL e I P 0.7 Flan-PaLM (Alphabet) C I U PaLM 540B (Alphabet) Yi-34B (01.AI, China) F L I M 0.6 Grok-1 (X.ai) T M Chinchilla 70B (Alphabet) GPT-3.5 (OpenAI) AR r 0.5 o Mixtral 8x7B (Mistral) r r GPT-3 (OpenAI, Fine-Tuned) E 0.4 Falcon 180 (TII, UAE) g LlaMA 2 70B (Meta) o GPT-2 1.5B (OpenAI, Fine-Tuned) L LlaMA 65B (Meta) e 0.3 t Flan-T5-XXL (Alphabet) u l Average Human Performance o 0.2 s Ab 0.1 - 10/27/2018 5/15/2019 12/1/2019 6/18/2020 1/4/2021 7/23/2021 2/8/2022 8/27/2022 3/15/2023 10/1/2023 4/18/2024 Note: The chart’s trendlines are fit to the most performant open- or closed-source models on 5-Shot MMLU (Massive Multitask Language Understanding) at the time of their release. Sources: ARK Investment Management LLC, 2024. This ARK analysis is based on a range of data sources as of Jan. 9, 2024, which are available upon request. Forecasts are inherently limited and cannot be relied upon. For informational purposes only and should not be considered investment advice or a recommendation to buy, sell, or hold any particular security. Past performance is not indicative of future results.