Wulf's Various Things

User Tools

Site Tools

Trace:
ai:generalinfo

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
ai:generalinfo [2024/07/12 21:47] – [Terminology] Wulf Rajekai:generalinfo [2024/07/23 19:56] (current) – [Hardware] Wulf Rajek
Line 216: Line 216:
 GPT-3 175B model: Microsoft built a supercomputer with 285,000 CPU codes and 10,000 Nvidia V100 GPUs [[https://news.microsoft.com/source/features/innovation/openai-azure-supercomputer/|exclusively for OpenAI]], hosted in Azure.  Researchers calculated that to train GPT-3 OpenAI could have needed 34 days using 1024 A100 GPUs costing $5M just in compute time based on 175 billion parameters if the A100 had been available at the time. GPT-3 175B model: Microsoft built a supercomputer with 285,000 CPU codes and 10,000 Nvidia V100 GPUs [[https://news.microsoft.com/source/features/innovation/openai-azure-supercomputer/|exclusively for OpenAI]], hosted in Azure.  Researchers calculated that to train GPT-3 OpenAI could have needed 34 days using 1024 A100 GPUs costing $5M just in compute time based on 175 billion parameters if the A100 had been available at the time.
  
 +Llama 3.1 used 16,000 Nvidia H100 GPUs to train the [[https://venturebeat.com/ai/meta-unleashes-its-most-powerful-ai-model-llama-3-1-with-405b-parameters/|405B model]]. 
  
 +===== Evaluation =====
 +
 +https://www.philschmid.de/llm-evaluation
ai/generalinfo.1720817229.txt.gz · Last modified: by Wulf Rajek