• 0 Posts
  • 14 Comments
Joined 11 months ago
cake
Cake day: June 4th, 2025

help-circle






  • It’s an MoE (Mixture of Experts) approach. An 80B-A3B model has 80B parameters total, so that dictates the size of the model and the VRAM+RAM you need to have to hold it, but only 3B of those parameters are active at any given time. This reduces the intelligence of the model compared to an 80B dense model, but improves the speed. In the end it’s the size of an 80B model, with the intelligence of a ~40B model, that runs at the speed of a 3B model.

    Pretty much all state of the art models either have already, or are in the process of switching to an MoE design, since it significantly reduces the hardware required to run big models at usable speeds. You can often get usable speeds on MoEs without a GPU at all.






  • Nah I’m on that guy’s side. His experience lines up with my own, namely that vibe coding is not useful for people who don’t know how to program, but it can be useful for people who do know how to program, and simply aren’t familiar with the specific syntax used in a language they’re not an expert in.

    In that case, the queries to the AI model aren’t, “write me a program that can do X”, it’s more like “write me a function in this language that can take A, B, and C as inputs, do operation Y with them, and return Z”, or “what’s the best way to find all of the unique elements in an array and sort it alphabetically in this language”. Then the programmer can take those pieces and build up a proper application with them. The AI isn’t actually writing the program for you, it’s more like a customized Stack Overflow generator, without having to wade through a decade of people arguing back and forth in the comments about inane bullshit.

    Does it save a ton of time? No, but it’s still helpful, and can get you up and running in a new language much faster than the alternative.


  • Context Switching

    It’s why I hate when middle managers get a hold of my time allocation. “You have 8 hours a day, so you can spend 1 hour each on these 8 different projects and move them all forward together!” Sprinkle 3-4 pointless meetings throughout the day, and then they wonder why nothing gets done.