• 0 Posts
  • 72 Comments
Joined 2 年前
Cake day: 2024年5月20日
  • BioMan@awful.systemstoTechTakes@awful.systems•Stubsack: weekly thread for sneers not worth an entire post, week ending 8th November 2026English
    9·
    5 天前

    Oh my god

    So why, at Proton, does it not feel like the revolution has come? Why has productivity not increased dramatically if everyone is using LLMs all the time already?

    The first is a change in engineering culture and engineer mentality. Your job has fundamentally changed. It is no longer to write code, and LLMs are not here to help you write better code faster. Your job is to help the LLM write better code faster.

    The second reason we have not been able to take full advantage of the AI revolution is that we have been almost completely stuck at phase 1 — using LLMs to speed up tasks as part of existing workflows. But writing code is not the only part of the software development lifecycle, and what happens when you eliminate a bottleneck? Another takes it is place.

    The third reason is a collective action problem. Using LLMs effectively to write code is something an individual engineer can do for themselves… But to retool the company at large requires amounts of both organization and resourcing that we simply have not invested in yet.

  • BioMan@awful.systemstoTechTakes@awful.systems•Stubsack: weekly thread for sneers not worth an entire post, week ending 1st November 2026English
    8·
    5 天前

    You can have a system that just gives you a relevance-sorted list, or a system that then ingests the chunks of those documents that are most relevant and gives a human readable summary in addition to the list.

    The indexing step requires a lot of processing, by which you chunk reference documents into couple hundred word chunks and cook each of those chunks into a high dimensional embedding vector. Once you have done that and saved the database of vectors, you can put in your query, embed it, check its dot product against all the other embedding vectors, and get your ranked list (potentially quite fast) without need to produce text out the other end.

    Making those embedding vectors works much faster on a GPU rather than a CPU, but can be done on either. Since I have one system set up to ingest a bunch of web documents and embed them and search them, I have that system already set up using the GPU to go faster, so there’s no real disadvantage to having it also set up to spit out a small natural language summary using a shitty tiny local LLM to go along with the list. When running it on local files, you can totally ingest the local document stash slowly on even just a CPU and just store that and run the semantic search on CPU, no LLM required.

  • BioMan@awful.systemstoTechTakes@awful.systems•Stubsack: weekly thread for sneers not worth an entire post, week ending 1st November 2026English
    61·
    5 天前

    Am I part of this category? I have a rather powerful home server that, among other things, I use to:

    • Run a decent coding LLM that helps me vomit up simple python functions for moving around lots of data between CSVs and graphing stuff, set up new docker containers with less trial and error, and run ML experiments on my GPU that previously took me hours to set up. I’m a biologist and previously I could run a commandline without breaking a computer but was very slow and non-native, now I can do simple non-security-critical things faster and actually understand novel error messages. Of note I never have that hooked up in a way that lets it run things just produce text that I can then do things with and maybe run myself.

    • Run my own local RAG system with embedding models and a small shitty LLM to search a hundred gigabytes of scientific papers and other PDFs semantically for things relevant to roughly paragraph length queries.

    • Run my own web metasearch engine with RAG and embedding models pointed at the internet, such that when I remember a paper I read fifteen years ago and provide a three sentence description it reliably finds it or finds lots of relevant old research (or old news articles I remember) when I give it a query and let it churn for five minutes.

    I feel like the important thing here is that I almost exclusively use them as information channels that take something at one end and spit it back out the other in a different or filtered form rather than making stuff, or effectively a piece of reference material for something that is formally verifiable and NOT CRITICAL FOR A SYSTEM TO FUNCTION. I also feel like all the actual innovations are more than a year old and do not understand people hanging excitedly on every little development.

  • BioMan@awful.systemstoTechTakes@awful.systems•Stubsack: weekly thread for sneers not worth an entire post, week ending 1st November 2026English
    6·
    8 天前

    It’s not like it’s a verboten hypothesis. The 1977 flu pandemic definitely WAS leaked from a freezer, probably Chinese or less likely Soviet, probably from a study on the effectiveness of H1N1 flu vaccines on young 18 year olds who were just coming through the demographic pipeline who had no immunity to H1N1 because it had been pushed out of the population by the 1957 H2N2 pandemic. It was a reassortment between a known human circulating strain that from 1950 and a second laboratory workhorse strain from the late 40s, paused for 27 years with zero evolution since their isolation, and re-emerged right when you would have human subjects available to test the effects of challenging adults who had no immune memory against H1N1 in an area that was heavily militarized and the earliest publicly known cases among military recruits.

    (Amusingly enough this flu pandemic was the one and only reason that the 2009 swine flu pandemic was a nothingburger, the population had immune memory against a strain similar enough to prevent severe disease)

    It just seems like calling the covid pandemic artificial became a community shibboleth along with suspicion about EVERY FREAKING DISEASE specifically and only because it gave them something to point to about the Dangers of Research!!!oneoneone