I also use it for a chatbot assistant for customer onboarding. Gemini deserves some kudos for their willingness to allow entry level subscriptions access to API-keys to build solutions with.
This is so true. Every junior sysadmin I have trained over the years (including myself) has had a “are we being attacked?!” moment when tasked with WAF report analysis, monitoring fail2ban logs, etc.
Monitoring WAN traffic really gets the paranoia juices flowing.
I remember when you could stand up a website and no bots would scrape it or scan it. It was a lovely time. No one had firewalls or antivirus and things were working fine until the worms and viruses started coming. You could be confident that your guests were real, so much so we had guest counters on many public sites.
Just build your website yourself as deep in the stack as you can instead of piling up 50 abstractions on top of each other. Some decisions like having your page be accessible by IP can only happen if you use technology like generic http servers (like apache or nginx) from the 2000s instead of implementing the lower stacks and actually thinking about whether that makes sense for a second.
If when you build a website or a backend, your server responds to requests by IP address (for example), you are building a bottom 90% product, and considering most software markets are super top-heavy, (say 1% win), that's ngmi land.
This would run into the same deal as VINs in the real world being tied to licenses, or serial numbers to guns. But the car equivalent of VMs/open hardware/custom firmware (imagine a $7 pi zero flashed with lineageOS “overage phone”) then becomes equivalent to a gun without a serial number, and suddenly open source/hardware people are felons and there is insane amount of control on hardware and software like it’s 1982.
This assumes that the government would be able to verify independently a phone serial number so that people’s IDs aren’t leaked. If not, then you’re back to the same thing as before since “drivers licenses” are stored by sites and shared around with advertisers
Can confirm, my experience in “loop engineering” was “this is neat” for 45 minutes until a daily ration of tokens was evaporated. The quadratic cost trap is prohibitive to experimentation.
As a localLLM evangelist, I am hopeful this will bring more attention to the joys of rolling your own sovereign AI.
Yeah, i'm hoping that gets smoother. I've been experimenting with omlx and opencode on my m5x64gb and keep running into issues w/ Qwen3.6-35B-A3B-MLX-8bit exceeding it's memory limit at the most inopportune times. Playing with 12B gemma4 (8bit) more today.
Maybe I should be aiming for something targeting 48gb of memory?
It depends what your goals are and what you are using it for. This space is fluid and my answer last week would be different than my answer today! That said there’s no substitute for hard work, here are some resource to get you up to up to speed:
Personally I find myself swapping models depending if I am engaged in “trad-development” vs building agentic probes or apps involving imagery. Tailscale the LLM to your deployments and ta-da!
The more sensational the headline the less I believe that the authors were present in technology 15-20+ years ago. People forget that Reddit used to be 2 parts programmer-humor 1 part snuff.
Show me an abliterated frontier model that is able to breakthrough the surrounding supporting models and actually hold state to produce contraband and I’ll gladly supply my personal image making making a silly face in a compromising position if it wouldn’t make the testers feel better.
Do they need to be tested like this? Yes. But it would take the carbon footprint of a commuter air terminal and the land rights of am small town in the high Sierras …. all converted settlers of Catan style into tokens …. just to lobotomize a fine tuned model to get close.
20 or so years ago while working for a Startup in the Home-Health EMR Space - it was my job to develop and integrate the proper processing of incoming visit forms. After an outage, I performed an audit of our incoming forms and noticed some anomalies in the billing patterns of doctors belonging to one clinic. In other words, these doctors either had the highest concentration of extremely sick patients - or they were committing Medicare fraud.
At the end of the post mortum with the CMO, as I was getting ready to leave I decided to bring this to his attention. I’ll never forget the change of mood preceding the dressing down I received: “do not ever put yourself in a position to make clinical decisions.”
3 months later, the charting anomalies were so egregious that the CMO’s spot-checks led him to sit the medical director of that physicians clinic down for a chat. They were good doctors, but they were over-billing. A year and a half later their practice goes under pre-payment review, and four years after I wrote a script that noticed an anomaly - the head MD of the practice was sent to prison for 4 years after collecting millions of dollars in over-billed house calls.
I loved working in healthcare, and I still miss it to this day. I don’t know where I am going with this, but right now I believe there is a diagnostic technology out there that is being used in veterinary science or piloted in some other country that could save a statistic level of lives …. However, due to the fact that doctors practice medicine and we don’t, as a group they act as defacto gate-keepers (which they are entitled to be as clinicians), the best thing you can do is to incentivize them with money (like Obama did) with Medicare bonuses for using an EMR that logged CCRs and alerted the doc if the patient didn’t have certain vaccine information in the elderly.
If the first guy to wash his hands was seen as a lunatic, the first geriatric practitioner to give over an iota of their clinical practice to automate Rx dispersal while navigating poly pharmacology concerns will go to jail for a narcotics crimes or will be labeled to heretic until Medicare pays them all for it.
If your name is John Smith, and you prompt Gemini to act as a robust OSINT tool, you will experience the vicissitudes of Chaps firsthand as the tool normalizations of vector space yields an answer that will not only never get close to identifying you because it has lumped several John Smiths together like asking a dog to fetch your shoes and it comes back with a Footlocker in its mouth.
No, not really. That's pretty basic stuff. You would do well in reading up on the shared responsibility model. Customers are responsible for setting up their own infrastructure, and platform/service providers are only responsible for the services they manage. Even then, stuff like persisted data is still recoverable by design.
But you are absolutely responsible for the service you put together. This is a basic principle for around two decades. Infrastructure as code tools are pervasive and ubiquitous for over a decade.
Correct me if I’m wrong, but from my experience in this space in order for a model to exercise judgment it must force itself to operate in a strict chain of thought mode. Since all LLMs are predictive creatures, I started to care a lot more about my judgment settings, the transparency of them, and the presence of a judgment loop in either the development or functionality of an application built these days.
Not exactly sure where I’m going with this, but my work with creating penetesting tools for LLMs, the way that I use judgment is critical to the core functionality of the application. I agree with your concern and I will just say that the more time I spent concerned with chain of though where now I will make multiple versions of the same app using a different judge set a different “temperaments” and I found it to be incredibly enlightening as to the diversity of applications and approaches that it creates.
Even using BMAD or superpowers, I can make five versions of an app without judges involved and I feel like I’m just making the same app five times because the API begins to coalesce around the business problem you want to solve. The vicissitudes of prediction tools always want to take the safest bet for the greater good, but with the judge involved we can make the agent force itself to actually be hostile about what exactly we’re trying to do, which has produced interesting and fun results.
