Hacker Newsnew | past | comments | ask | show | jobs | submit | kriskrunch's commentslogin

Doheny was approved shortly after the Huntington Beach desal plant was killed. Update from last month: https://www.ocregister.com/2025/11/26/landfill-trash-could-h...

Poseidon currently runs a desal plant in Carlsbad. My understanding is that the water the plant releases into the ocean requires exemptions for how concentrated it is. Additionally, the plant draws plankton filled water. Not really what we want in California.

There are better desal solutions out there like OceanWell. They have a deep water desalination solution that solves many of the problems of conventional desal. They just signed a project in Nice, France in the past few days. Also, they are working with the city of Las Virgines over the past few years.

If I remember correctly, the new desal plant in Doheny has a slightly different approach to draw water in from beneath the sand, using the sand as a prefilter. But I'm not sure how that works better than drawing water in from near the surface. I can't imagine how the plankton can possibly escape the suction forces drawing them into the sand.


I wasn't aware of the Doheny project, thanks! But it seems that that project timeline is at least as long as the one CCC killed.

With these kinds of timelines and that kind of regulatory risk, I don't think large-scale desalination is going to fix California's drought issues, regardless of whether it could.


Anyone use SmartTube? Or is it something I should remove?


Just make sure you're using the new version (old one was compromised):

https://github.com/yuliskov/SmartTube


I use it. Works great.


Check out the discussion[0], looks like there are submissions in several languages. Go, Rust, Python, and C++, to name a few

[0] https://github.com/gunnarmorling/1brc/discussions


It looks like the problem is dominated by reading in the data file. Some fast solutions just read the whole file into memory.


It would be a more interesting challenge if the data file were larger than the memory. I would love to see what people would come up with on some baby vm with 512 mb of ram.

Even more interesting would be small ram, little local storage and a large file only available via network, I would like to see something other than http but realistically it would be http.


As the file-memory ratio changes the problem becomes more and more stream processing, right? If the number of cities becomes too much to keep in memory then it becomes a database with "let's see who can find a better index data structure fitting for these I/O patterns and HW" game.


For this problem, changing the file size to not fit in RAM doesn't really make the optimal solutions more interesting


Sixteen thousand Excel 97 spreadsheets?


Rather than read the file into memory, memory mapping can be used.


Memory mapping (at least on Linux) isn't actually faster than reading the file manually. Especially if you use appropriately sized buffers.

(Of course, the five times in a row might mess with that.)


Right, it's the five times in a row thing that makes an in-memory solution faster. Otherwise, this is a purely sequential one-pass problem, which is how you'd do it in practice.

Parallelism with edge effects is pretty common. Weather simulation, finite element analysis, and big-world games all have that issue. The middle of each cell is local, but you have to talk to the neighbor cells a little.


You wouldn't want to do this for a huge file. A very fast solution would use a small number of buffers and io_uring (or equivalent), keeping the page table and cache footprint small.


Yeah so I had a discussion on Twitter about this, turns out 12GB is small enough to fit into memory, and the author runs submissions by running a solution 5 times in a row, so using direct IO actually hurts because having the kernel cache is a way to enforce the file is in memory for the 4 runs after. I have a direct IO solution with SIMD string search and double parsing, just in C++ (using libraries). It runs in 6 seconds on my 24 core linux box (NVMe).

Code: https://github.com/rockwotj/1brc

Discussion on Filesystem cache: https://x.com/rockwotj/status/1742168024776430041?s=20


I missed the "5 times in a row." If you do that, yeah, keeping the whole thing in memory is far better.


> double parsing

In case you haven't noticed yet, the input format guarantees exactly one fractional digit, so you can read a single signed integer followed by `.` and one digit instead.


Yeah I missed this originally, and stuff could be faster with this assumption without a full double parser. The fastest java solution dies some near branchless decoding for these


could you just add the character values eg 49 for ascii 1, and then subtract off the offset once at the end instead of doing atoi on each line?

edit: doh that works for min and max but the average overflows.


Yes. I'm not sure it'll help, but def worth a try.


Wow, that's pretty fast considering how simple main.cc looks. I do love c++. Nice use of coroutines, too.


So you are basically at the mercy of the OS caching algorithm. That sounds like a bad plan for a benchmark. You are not measuring what you think you are (your code), you are measuring the OS caching policy.


Dumb question. With io_uring, how do you handle lines that straddle between chunks? I'm asking since, AFAIU, the submitted requests are not guaranteed to be completed in order.

(The easiest I can think of is submitting reads for "overlapped" chunks, I'm not sure there is an easier way and I'm not sure of how much performance overhead there is to it.)


You have to handle it manually. Remember the partial lines at the beginning/end of your chunks and merge them when their mates become available.


What is the downside of memory mapping in this scenario? Shouldn't the page table properly handle the case of doing a single sequential read over a range of pages? Accessing the contents of a file doesn't seem like something caching would matter for. Do you mean that reading of sequential pages will keep adding to the cache compared to reading from a single page? That seems like a similar thing as before where they will be the first things gone from the cache anyways.


Caching and paging almost always matter, even on things like this. The core problem is that the filesystem won't prefetch for you, and you will be waiting to page fault several times over the length of the file. Another problem of the size of the working set is that you will be seeing several slow calls to (essentially) malloc in the kernel to hold all of that data, while using a small, preallocated structure will give you none of that trouble.


> Shouldn't the page table properly handle the case of doing a single sequential read over a range of pages?

That's what I used to think, too. But the kernel ain't that smart.


But would that really be faster when you need to read every byte of a file?

I thought memory mapping solved a different problem.


> No external dependencies may be used


The irony is Wozniak and Jobs manufactured and sold devices, called Blue Boxes, that hacked the phone system to provide free phone calls.

> "If it hadn't been for the Blue Boxes, there would have been no Apple. I'm 100% sure of that." -Steve Jobs

For your Apple collection:

https://www.bonhams.com/auction/24495/lot/109/wozniak-steve-...


Sure, but no one was arguing that ought to have been legal or that the phone system would be in the wrong to attempt to patch exploits found by hacking.

Their Blue Box efforts were very cool! So is the coding that let Beeper do this.

But it is mystifying to me that people are arguing it is illegitimate (under our current system) for Apple to try to secure its system.


The irony is people are rooting for a company (Apple) which only exists because it did something similar like Beeper early on, but root against Beeper. Apple is its present and history.


Aluminum was once rare too. It was considered a precious metal. The obelisk at the Washington Memorial is capped with Aluminum... I've heard stories that it was used for engagement rings in that era as well!


Yes! The movie is a good one. My wife enjoyed it and she doesn't know anything about pinball. There's a little love story woven in. Highly recommend it.


Get new machines if you can afford them. Stern or Jersey Jack.

90s machines are great, I love them, but they are 30 years old and are not where I recommend anyone to start... My first pin was a fixer upper Twilight Zone. Learn from my mistake! Haha

The most challenging part is the maintenance, especially the classics. Owning several machines taught me to fix them. My newer machines broke less often...

Every time I threw a decent party I'd end up buying minimum $200 in parts and spending half a day fixing them.

Lord of the Rings is one of my all time favorites. I highly recommend it. It's not ancient, and the game is really deep. It's pretty easy to work on and fix. Make sure you protect the plastics, I couldn't find after market ones. The Wizard mode is extremely satisfying because it is super difficult to achieve... There are a couple fun special modes you can unlock.

On modern solid state pins, you can adjust the difficulty settings to make the game more fun for friends.


Thanks for the great advice. LotR is actually my single favorite pinball game and I've played it a ton, mostly at Pinball Pete's in Ann Arbor. But, I've played it so much that I would probably get bored of it quickly if I purchased it. Twilight Zone is great too! I'm in Seattle now, and so lots of places have the new Foo Fighters pin which I'm enjoying a lot.


> he said he’d been walking around the world all this time and as soon as he clicked with people, he’d always be saying goodbye.

I feel the same way when I take my little vacations... I meet amazing people, connect, and then move on. There is a beauty to it though. I really try to enjoy those moments. They are so brief.

This was a touching story, especially the fact that the dog he rescued might be the first dog to have walked around the world!

In elementary school, around 2nd grade, I attended a talk by Dave Kunst, the first man to walk around the globe. I'm surprised the Guardian didn't mention him. His story has often surfaced in my memory. I never had the guts to do anything remotely so adventurous... and he did it in the very wild 1970s. Around the same time my mother hitchhiked from South America to the US.

Dave told us he searched for sponsors before embarking on his trip, but no one would take him seriously. Not even shoe companies!

He walked to the edge of each continent, to dip his toes in the ocean at the start and end of each continent to make sure he was going as far as possible. I think there were a handful of countries he wasn't able to pass through for some reason or another.

Dave's brother was killed on the journey by thugs who thought Dave and his brother were personally taking donations and carrying the money. He said he came to peace with his brother's murder because he died doing what he loved. If I remember correctly, another brother joined Dave to complete the trip. In all his journey took around four years. About half the time it took Tom.

He closed the talk by showing us all photos of his last pair of shoes after he had completed his trip, and then a photo of his bare feet -- and we all screamed in disgust!


Wow the Kunst story is incredible!

His brother's tragic death, his own recovery in the hospital, the people he met who helped him along the way (including his future wife!).

This would make an incredible biopic: https://en.wikipedia.org/wiki/Dave_Kunst


> and then a photo of his bare feet -- and we all screamed in disgust!

I'm really curious about the reason and the state of his feet, having walked so much.


He'd have lost a toenail or two and might have a bit of callusing.


Capitalist societies did more to improve the lives of everyone on Earth than Marxism ever did or ever could.

Have you walked the streets of Havana? I have. Talk about oppressive... And depressing.


Please, being under economic sanctions for longer than most people on this board have been alive has done more to damage Cuba than any kind of political/economic system could have ever managed. Especially given their main industry was tourism and agriculture. The rich Americans that poured dollars into Havana, were cut off along with the supply of manufactured goods, and being a resource poor island made it hard to build an industrial base.

Compared with their more capitalist neighbors who aren't doing great either, might point you to reasons why a whole region, which includes a resource rich continent have been struggling for the past couple hundred years. I will start you off with the wikipedia page on banana republic: https://en.wikipedia.org/wiki/Banana_republic


Please! :) The Castro regime made the decision to seize any land from anyone with 50 acres or more. What sort of outcome did Fidel, Che, and Cienfuegos expect? Well, they expected to build thriving trade economy with the Soviets... whoops!

My point is Marxism/Socialism trades one form of oppression for another. Marxism fails to identify poor leadership as oppressive. Socialist systems are run by error prone leaders that crave power and fortune... They are the same people that run corporations under a different flag. Che was killed trying to overthrow another nation to put in Cuban friendly socialist leadership... How is that different from your banana example?

At least in capitalism we can vote with our dollars. We can spend our money on things we believe in. Not with socialism. Good luck finding choice or innovation that the state doesn't support. It literally crushes innovation.

Besides these arguments, what great things came from Marxism that we enjoy today?


I sorta think your conflating authoritarian socialism (aka communism) with democratic socialism which is as large a mistake as conflating pure democracy with corrupt representative democracy where a minority of the population is used to rubber stamp an oligarchy.

Similarly, capitalism in its purest forms is basically feudalism because the people won't have any dollars to vote with. And time and time again it seems the benevolent Henry Ford style capitalists are proven to be the minority. So, one might say that without the socialist scare in the late 1800's early 1900's which forced the hands of the capitalists all those quality of live improvements everyone likes to go on about probably wouldn't exist. We might all be working 80 hour weeks and scraping by in company hovels eating whatever gruel the company store gives us. But then again, a large portion of the US population is wildly unaware of how the bottom 30% live, while ignoring their own debt slavery. So enjoy your bread/circuses while the rich take an even larger slice of the economic pie.

edit: How about some George Carlin, RIP https://www.youtube.com/watch?v=Nyvxt1svxso


I love Carlin. Thanks

Perhaps you're changing the subject? My original post was in response to an invocation of Karl Marx, the father of Communism... as I understand it, Communism is about as authoritarian as it gets.

I mean an intermediate stage from Capitalism to Communism is called "Dictatorship of the proletariat"...


>At least in capitalism we can vote with our dollars. We can spend our money on things we believe in.

The end state of the current US capitalistic system is a rental economy where "you'll own nothing and be happy"(WEF terminology, not mine). We are seeing this in the loss of the ability to buy real estate but also in the push to move towards a subscription model for everything. Subscription cars vs ownership, subscription services vs ownership, even subscriptions for hardware such that you rent hardware for a monthly fee and "trade up" every other year. Capital has exhausted every other avenue for growth so this is what is left. Combine that with wage stagnation and inflation eating away what little value is left. How is it possible to follow your plan in the world?


The end state of socialism and communism is also "you'll own nothing and be happy". All political/economic systems suffer from corruption and abuse of power at the top. The only difference is the ability for the masses to control their own destiny. Market economies give them a little, control economies give them none.


True, yeah both extremes suck. My optimistic hope is that the US self corrects to something like what the European countries have: a flavor of Democratic Socialism. Right now its not looking good, the US is moving further to extreme capitalism with no real end in sight.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: