Feel like half of lemmy is waiting for forgejo federation, lol.

It’s just another species of jellyfish. I was hoping for something crazy interesting like a tropical leopard seal or a manatee with tentacles.

Truth always moves slower than lies and always for the same reason. Were newspapers a terrible invention for the people because they came with yellow papers or did the good outweigh the bad?

I think we think of AI differently at least in part because I don’t treat it as something owned by someone else. It doesn’t matter if the rich stop subsidizing it, I can and do run it on my own machine. Sure, I can’t train it, but I can’t code an entire operating system either and I still think of my computer as mine. There is certainly something to your point about biased training and the costs related to it, but that just seems like reason to favor open source local AIs. It’s the same reason I don’t use Windows or Apple operating systems – it’s terrible to have that sort of dependency and bias built into your tools. Of-course, I have to admit Linux users are a minority (for reasons I don’t really understand), and if all the AI hate basically amounts to an inability to run linux / local LLMs… well, I understand it a little more.

Honestly, good conversation. Don’t agree with you, but you definitely widened my perspective.

Your point about technology enabling people is well made. While I think it’s pretty clear that AI can have positive effects, your point about it’s use in propaganda and misinformation is legitimate. Even if I don’t think it’s the underlying issue, I can’t deny that it enables it. That said, this comment chain is long enough that I’m losing the plot. I tried to read back, but I don’t know what your stance in all this is.

Screaming regulate AI

Who said anything about regulating AI? I want nothing short of a third AI winter.

I don’t know what “a third AI winter” means. I can’t tell if you’re saying you want AI to die out or if you’re sarcastically saying you want an AI nuclear apocalypse or maybe both. If you genuinely aren’t calling for regulation, I don’t know what you’re calling for. Like for regulation – like gun control – I sympathize with your concerns; however, there’s not a snowballs chance in hell I’m going to advocate for only the government (have you seen governments these days?) to choose who does or doesn’t get AI.

If you genuinely don’t want regulation, are you just saying it shouldn’t have been made in the first place because it enables the evil people more than the good people? I’m not sure that’s true in regards to what the actual technology does and I’m not comfortable blaming the technology for the grifts associated with it. Propaganda and misinformation have always been an aspect of class warfare and typically the rich and elite have had bigger budgets and a greater effect. Every single communications advancement has pushed that more and more towards the people, and I think AI will still favor the people over the elites for propaganda - I can literally put Trump’s face on a chicken with no skills. Misinformation is clearly a bit more built into the technology, but even with that it gives people access to a lot more information a lot more easily (e.g. finding algorithms or new terms to search). The tech itself seems to favor the people.

You don’t seem like the type that’d be advocating teaching kids how to use it, but I actually think that’d be a great idea in some second year coding course or something both to build their resistance to the negatives and to give them more capabilities once people get over the nonsense drama with it.

What do you think we should do about it?

That’s way smaller than I thought – or perhaps space is just way bigger than I thought.

I’m biased as a biologist, but I don’t think extraterrestrials will reach out to humanity until we’ve solved both global peace and biological immortality (the second being much easier and closer than scifi makes it seem and likely a prerequisite for the first). Until we have a population that’s basically capable of responsibly managing themselves, I think humanity looks like infants or maybe teens throwing tantrums. Unstable, unreliable, and in need of the time to work itself out.

That’s assuming of-course that they’re near by in the first place, which is unknown.

CAR-T is rapidly advancing, and I’m a bit biased, but I’d say cell-therapies are very much the technology of the future.

How does text prediction have nothing to do with programing? And how is executing a series of functions often dependent of previous output not a pipeline? Okay, whatever, we’re on the same page that AI doesn’t take jobs and is used as an excuse – that’s great.

despite that insist that that alone is worth dealing with all the problems that come with it

No, I don’t think the problems should be tolerated.

we should just hold the bad people using it for bad things accountable, as if it were that easy

Yes, this. Do this. No, it’s not easy, but it never has been because…

How is AI itself not part of the problem here? … Yes, a lot of the problems associated with AI are just worse versions of problems that already existed.

It has nothing to do with AI. Literally every problem “associated” with AI is just a pre-existing problem repackaged. They’re hiding behind an excuse and for reasons I do not understand you’re buying into it, making yourself look like a Luddite, and letting them claim to be on the side of historical progress. You and I both know it’s nonsense to let them claim that, but rather than be dismissive of the technology and critical of the people you’re blaming it:

But the existence of AI makes these problems worse and harder to solve.

Only if you buy into the idea that it’s something we have to move the world for. You claim you can see how little an effect it has had, but you simultaneously seem to think it’s a big bad demon. AI has done very little in any direction. Aside from all the hype, the actual effect good or bad has been marginal. Something like 3% of job cuts occurred due to automation, and I’m not even sure if that’s lower or higher than on a normal year.

You want to complain about data centers, fine, but complain about political corruption and talk about how we shouldn’t centralize power in singular mayors rather than full consensus councils, or how we need multi-candidate ranked lottery voting systems with recall mechanisms, or how we should heavily tax political donations and political advertising and use a portion of those funds to provide the public with factual information campaign ran with a similar ranked lottery electorate consulting with technical experts, or how we should regulate or socialize utilities. – None of it’s easy, but if you want a world worth living in that’s the work you have to put in.

Screaming regulate AI leads to politicians creating internet ID laws and mass surveillance because you didn’t bother to build consensus about what you meant when you said regulate AI and politicians doing as they do took it upon themselves to reinterpret what you think you meant into something that they or their billionaire donors wanted.

AI is a god damn programming package

No, it’s not. Other than a few specific products, it’s not marketed like that, it doesn’t have an interface that suggests that it’s that and its functionality (or veneer thereof) isn’t limited to that. What a strange thing to say.

No, it literally is a programming package + some data. There’s nothing “agentic” AI can do that isn’t done just as well by a generic pipeline – except perhaps ignore garbage information and interface with non-coders. If you’re being automated away, it was going to happen anyway.

And here we get to the crux of the matter…

First of all let’s not act like my exasperation with the world’s stupidity is some sort of lapse in my judgement. I’m getting the ability find algorithms and debug faster to write better code, and occasionally asking it for terms I can search for to find information. That’s it. I’m not revealing some deep political bias you can use to ignore my underlying argument.

Honestly I don’t understand what’s so hard about arguing for labor laws or demanding proper accounting in SEC filings or any of the other zillion things we should’ve been doing for the last fourty years, but if you’d rather act like fancy program ruined world I guess whatever man. I don’t think it’s gonna help anyone, but you do you.

Yeah, unfortunately it seems this can’t be converted to a llama.cpp compatible format yet, and that’s pretty big a tradeoff right now. Not surprising with how new it is, but we’ll have to wait to combine it with other improvements. Pretty exciting for the future though.

Update: I actually couldn’t get this to run even on HuggingFace Transformers. I made a bug report, but basically I’m getting some torch incompatibilities with flash-attn. Maybe this is a known issue for more experienced folks, but I couldn’t solve it.

I was just about to make a post asking for the best small model after finding out Qwen3-27B was way too slow, so Orthrus-Qwen3-8B looks like a pretty appealing option.

If you believe there’s no world in which the majority of people are thoughtful and well intentioned, I don’t think we’re on the same page regarding what would make the world a better place. AI, like all technology, enables people who exercise their agency. I’m saying “if only people would just” blame the actors instead of the technology we might be able to have the benefits of the technology and hold the actors accountable for the detriments they create. I understand your argument that there are always going to be bad people, but I believe in holding people accountable for their actions.

AI is a god damn programming package that the whole world has decided to flip out over when there’s only been like five niche applications for it. I don’t get it. Why do I have to lose a programming package because everyone - for no real reason - started blaming it for the reason they’re assholes to each other as if they hadn’t been jerks to each other before. It’s stupid.

AI doesn’t have to be directed to make the world a worse place. That’s a choice made by people. There are tools that don’t have good applications (e.g. nukes), but AI certainly isn’t one of them. Blaming the tool when you should be blaming the people gives the people an out. Shall we ban hammers because they can be used as weapons?

Yeah, I had seen the OT-2 before. I’m not sure it quite reaches the standard I’m looking for, but it might be the best I can hope for. The user manual seems to document a lot even if I’d like a more details on a lot of things. I’m certainly not paying 3k for a pipette and a subscription to over priced tips, but maybe someone has documented some mix of casting and 3D printing that to make a more sensible pipette. Ideally, we’d have open blueprints and repair manuals, but maybe there’s enough of a community around it to make it the best starting place.

Good. If you can’t be bothered to read your own papers, you shouldn’t be in academia.

My point abut the environment was more that people don’t have perspective on how good or bad it is. The thing you linked had to go to multiple image and video generation to be 3hrs of microwaving, and I think it’s pretty clear to anyone text is cheaper. For the same amount of time, running a microwave is worse than text based chat with an LLM. You can make up your own mind as to whether that’s unacceptably horrible, but I think a lot of people would be fine with that level of damage for what they get out of it.

Data centers are a separate issue to me. Those are just stupid. Neither sensible business nor the population at large are going to buy into SaaS and training even more models that have clearly reached diminishing returns is utterly pointless, but again that’s more a why are our corrupt politicians taking bribes for this? I strongly agree with the anger at the government for wasting tax payer money and otherwise being a bunch of corrupt bastards – we need better democratic systems. You cannot consent to being governed if you cannot say no, and our elections don’t have a none of the above or lottery option. That makes them illegitimate.

Honestly, I’m glad you care about the environment enough to criticize me for mitigating the issue. You’re right for a lot of points AI is probably a little closer to running a microwave than gaming for pre-existing models, but either is far better than leaving your car idling for the same time. My point was more that people don’t seem to have good a sense of the scale of the effect. You’re also right that a lot of people are rushing to use more AI in stupid pointless wasteful ways that will cause mass waste for no good reason, but again perspective: the vast majority of people don’t agree with the fuckcars communities and there’s really no denying that the mass adoption of personal vehicles has been, is, and will continue to be far worse than AI. Unless, perhaps, if some moron hooks an AI up to nukes – but in that case I’d still be pissed at the moron. I agree that AI sucks for the environment, and it’s fair to dislike me mitigating it. My point is more about the consistency of the criticism compared to other things in the discussion of what is worth what.

Coding landed on the right solution pretty early with this: Use whatever tools you want, but you’re responsible for what you publish.

I work on protein design, and generative AI for proteins is very capable of helping us make medicines, but if take some half baked garbage output and inject it into someone and they die, I’d still be held responsible. I have to filter out a lot of junk and test that the designed protein works the way they’re supposed to work. I have to be responsible for my usage of AI.

Same for anyone. If AI helps you re-phrase hard to parse sentences and you’ve verified the re-phrasing accurately describes what you’re trying describe, then that’s just making science is more accessible. If you’re using it responsibly, that empowers the reader and the writer by enabling communication.

I’m unimpressed by the anti-AI arguments, and I think they should be more honest. I grew up with online piracy arguing ideas can’t be owned, so I’m unimpressed by the intellectual property arguments, and I seriously doubt the vast majority of people care about this. I’m sure artists care when they lose work, and I’m sure people who consume the art care when direct garbage output is used – but be mad about that if that’s what you’re mad about. Be mad about CEOs who are willing to cut corners and produce shabby products using AI. Be mad about the capitalists incentives, and work to fix them instead of being mad when someone who’d’ve gone without has a good-enough picture.

Also people have never cared about the environment and the impact of AI is vastly overstated. Running an existing model isn’t any worse for the environment than gaming, and training them produces tools that can be useful as a coding package, like numpy for language. Are they bad for the environment? Yeah, everything is bad for the environment, but I didn’t see this level of outrage about bitcoin and people still like to pretend plastics recycling works. There is not a meaningful portion of the population that cares about the environment, and anyone who does care knows I’m right.

People aren’t actually mad at AI as a tool. People are mad about shabby work and increased spam. People are mad at losing their job when some jackass CEO fires them and being stung along when they apply for a new job. I don’t understand why we’re taking this “it’s the AI’s fault” excuse seriously. The more we blame AI, the more legitimacy we give the stupid AI hype bubble, the stupid uses of AI, and the policy outcomes hiding behind AI. If we were brutally honest and insisted that AI has had little substantive effect on the economy (which is what the data actually says), CEOs and politicians would be forced to take responsibility and be held to account for their stupid decisions.

Best I can tell, this anti-AI crap is a distraction and excuse not to talk about workers rights, education, and good democratic systems.

Oh, okay. Python bindings are like llama-cli where once it’s loaded it persists until the process ends, and the server approach is for when I’m bouncing between python processes and don’t want to re-initialize. That’s good that’s more like normal code. With context, you could just use the already loaded large model and just not send the same context/history (i.e. Have the larger context generate the smaller context), but for more narrow contexts simple/small and complex/large LLMs work pretty much the same and the smaller ones are faster, so you use them. That makes sense.

Would I be correct in saying that since I’m just starting out using smaller models is a risk since I may not know how to sufficiently narrow the context and the larger model is more capable of catching oversights?

I just installed llama yesterday, and I was under the impression the reason you’d use a llama server as opposed to python bindings was to initialize the model once then use it repetitively. It seems like multiple models would require initializing and killing models as you switch (or maintaining multiple, which seems hard on consumer hardware). Is that re-initialization still faster than just using a larger model due to the time it takes for larger models to process the requests and can I assume switching between models is relatively infrequent?

Also, doesn’t this seem backwards? In my experience solving real problems in life as a human the overall plan is pretty easy to come up with, but getting the details right takes careful methodical thought; though, I suppose if the model is specialize for a particular type of task that makes sense.