• Miss Brainfarts@lemmy.blahaj.zone
      link
      fedilink
      arrow-up
      3
      ·
      10 months ago

      Mixtral 8x7B seems fairly capable to me so far. It’s just that I need to wait a few minutes for it to reply, given I’m running it on a 1st gen Ryzen…

      Any models you’d reccommend that fit into 4GB of VRAM?

      I’ve tried Deepseek Coder, and it certainly works well for quickly churning out bash scripts for whatever purpose I can possibly think of

      • WaxedWookie@lemmy.world
        link
        fedilink
        arrow-up
        2
        ·
        10 months ago

        Similar story here with an old 10xx GPU. I’ve just started tinkering with dolphin-mixtral-8x7b, but it’s early days.

        • Miss Brainfarts@lemmy.blahaj.zone
          link
          fedilink
          arrow-up
          3
          ·
          10 months ago

          Dolphin is just peak amusement to me. I’ve asked so many weird things of it that it’s basically just cowering in a corner now, afraid of answering any question for fear of getting another kitten killed.

      • WaxedWookie@lemmy.world
        link
        fedilink
        arrow-up
        2
        ·
        9 months ago

        An nvidia card with as much memory as possible - the newer the better.

        I’m also beginning to look into the dedicated accelerators like the coral, but at first wash, it looks like lack of onboard memory will be a massive bottleneck.