Summary: Meta, led by CEO Mark Zuckerberg, is investing billions in Nvidia’s H100 graphics cards to build a massive compute infrastructure for AI research and projects. By end of 2024, Meta aims to have 350,000 of these GPUs, with total expenditures potentially reaching $9 billion. This move is part of Meta’s focus on developing artificial general intelligence (AGI), competing with firms like OpenAI and Google’s DeepMind. The company’s AI and computing investments are a key part of its 2024 budget, emphasizing AI as their largest investment area.

  • simple@lemm.ee
    link
    fedilink
    English
    arrow-up
    36
    arrow-down
    1
    ·
    6 months ago

    Who isn’t at this point? Feels like every player in AI is buying thousands of Nvidia enterprise cards.

    • 31337@sh.itjust.worksOP
      link
      fedilink
      English
      arrow-up
      15
      ·
      6 months ago

      The equivalent of 600k H100s seems pretty extreme though. IDK how many OpenAI has access to, but it’s estimated they “only” used 25k to train GPT4. OpenAI has, in the past, claimed the diminishing returns on just scaling their model past GPT4s size probably isn’t worth it. So, maybe Meta is planning on experimenting with new ANN architectures, or planning on mass deployment of models?

      • qupada@kbin.social
        link
        fedilink
        arrow-up
        16
        ·
        6 months ago

        The estimated training time for GPT-4 is 90 days though.

        Assuming you could scale that linearly with the amount of hardware, you’d get it down to about 3.5 days. From four times a year to twice a week.

        If you’re scrambling to get ahead of the competition, being able to iterate that quickly could very much be worth the money.