I bought a datacenter GPU that doesn't fit in a normal motherboard, macgyvered the fan with jumper wires, and now I'm running a model that ties with Claude Sonnet 4.6 on benchmarks, all for £200.
  • melfie@lemmy.zipEnglish
    3·
    1 month ago

    Good point, that should theoretically be double the decode speed, though not sure how the prefill would differ since it’s mainly compute-bound.