As one of the original creators of sudo (https://en.wikipedia.org/wiki/Sudo) I've witnessed it getting nearly totally rewritten and then incrementally bug-fixed over the last 43 years. It must take the prize for the UNIX command most highly-scrutinized for security flaws. Flaws which have been identified and fixed.
Thousands of developers and security experts have gone over it. So part of me wonders - how is it possible for a single dev team to totally reimplement it without unknowingly
introducing at least a bug or two? Is there something to this Rust language which magically eliminates all chances of any bug being introduced?
I can only thank you for the work you've done in creating sudo, I think it's an invaluable tool in the general day to day use for so many people. As someone working on sudo-rs, our goal with creating it never was to invalidate any of the work previously done, and we are very much aware that our implementation will not be bug free, especially not at the start.
For me personally, creating this Rust version allowed me to work on something that I would normally not be able to work on, given how I would not rate my confidence in writing relatively safe C code very high. If nothing else, at least we already found a few bugs in the original sudo because of this work. Despite the 43 years of bugfixing, such a piece of software is unlikely to ever be free of bugs, even if just for the changing surroundings.
Other than that, having some alternatives can never hurt, as long as we keep cooperating and trying to learn from each others work (and from each others mistakes).
In a very literal sense, you could write this same code, in unsafe Rust, so one could argue that Rust does not prevent it.
Some may argue that if this program was written in Rust in the first place, "concatenate all command line arguments into one big string for processing" wouldn't be the way you'd go about escaping command line arguments. The issue here is about misplacing a null terminator, Rust strongly prefers a "start + length" style of representing strings instead of null terminators, so you'd never really end up in this situation in Rust in the first place.
I'm sure there's other ways to evaluate the situation as well. Which one you find compelling is up to you.
Yes buffer overflows are one of the explicitly addressed vulnerabilities of Rust's bounds checker, which is always on, if memory serves. I haven't touched Rust in a year.
You can, but unsafe code is discouraged in general, even given a slight performance cost.
For performance-insensitive, security-critical code, there really shouldn't be any such code in the entire program—and it would be easy to verify that with a presubmit.
In an ideal world, no, there shouldn't. But `unsafe` is not just a performance hack; people do find things that they legitimately need to do that Rust can't statically verify. Probably the most trivial example is interacting with code that is not itself written in Rust.
This does imply that some of the stronger claims about Rust's level of static safety guarantees that float around on the Internet can't really be true unless substantially everything you might want to do has a version that's been completely written in Rust. Whether you feel that means that achieving the desired level of safety implies you've still got to rely on some dynamic analysis tools just to be sure probably depends on how much safety you really want, and how much faith you're willing to place in the skills of the authors of the libraries you use.
And even then, if we really want to go least common denominator, if you're running your program on Windows or a Unix or basically any other OS that isn't Redox, then you've got unsafe code executing every time Rust's own standard library needs to make a syscall to achieve something.
Which I don't say by way of criticizing rust Rust. It's got to live in the same crappy world we all have to live in, and it's arguably doing a better job of de-crappifying it than any other systems programming language. I'm just trying to illustrate how an unqualified statement along the lines of "there shouldn't be any unsafe code in the entire program" is kind of a self-strawman, precisely because Rust has to live in said crappy world, and I think that it might be unsafe to lose sight of that fact.
Look into the crates you use and you’ll find tons of unsafe code, especially around custom data structures doing buffer pointer arithmetic and stuff. If it’s wrapped in a safe interface, you’d never know.
The borrow checker does not prevent out of bounds access of arrays (or vectors or whatever you want to call them).
The borrow checker is intended to protect against "temporal memory unsafety". It can tell you that you are using something that has already been freed, or something that could be freed while you are using it for example.
Bounds checking is a "spatial memory unsafety" problem, it has nothing to do with borrowing and exclusive references.
Bounds checking is a trivial problem, for an array that has N length, like a char[N], something tried to access a value past the end of the array (like char[11] if N=10).
Rust doesn't really do anything special here and protecting against buffer overflows does not require any novel technology.
An implementation of bounds checking is as simple as an "assert(I >= 0 && I < N)", where I is the index and N is the length of the array.
In C this is difficult to do because arrays or are just pointers (or decay to) and pointers do not carry any information about the length. Keeping a separate variable containing the length around but this apparently is too unergonomic since virtually all C software does not check every array access in all parts of the program.
In Rust, its very rare to use raw pointers to work with "arrays". Instead there is a "slice" type that models the concept of a contiguous sequence of values in memory. The important part is that the slice type is a "fat pointer", and the fat pointer contains the length or the array. The slice type is able to check every access in all parts of program.
So all slice accesses are checked by default. and you can't "turn it off". If you really want to disable bounds checking for some reason, there is an unsafe "get_unchecked" function.
There are some sequence types other than slices, arrays for example (array is a specific type here). They are still checked but they don't have to store the length information because it's encoded into it's type.
The Vec type is another one. It is a resizable "array", and it stores 3 things, a pointer to the allocation, the capacity and the length. That's mostly not relevant here though. It checks every access like the other types.
Bounds checking is a very easily preventable error. It should not be happening in $CURRENT_YEAR.
Advanced type systems and borrow checking/memory safe languages DO go a long way, but obviously, No. The best developers can do pulling a RiiR is try to follow best practices and learn from past mistakes. We've certainly come a long way in 43 years. Ditching C string handling eliminates a ton of bugs before you factor in the memory safety. Heck, you have to admit: someone setting out to make a secure sudo replacement could do a lot better nowadays even using just C. The OpenBSD project does a pretty good job demonstrating this imo. If you make a programming language that doesn't have many of the sharp edges OpenBSD code avoids, you could probably get yourself a head start, but clearly it also is going to take plenty of care and experience too, and a programming language can't really grant you that.
I think it's at least worth humoring. It probably shouldn't be shipping as a default any time soon, though...
While I don't negate your experience and I genuinely anticipate that this project is going to rediscover some pain, there's something to be said about the fact that we don't have to replicate the life work of Newton, Leibniz, Maxwell, etc to really "get" classical physics. It fits now into the high school curriculum, and if you pass it, you can be fairly decent at it; with a little additional effort, you can get real freaking good at what took those people their whole freaking lifetimes.
This is because we can stand on those giants' shoulders and have the benefit of hindsight and not have to also repeat each and every of their blunders, and have better technology and learning methodology to boot.
So I presume if you yourself wanted to rewrite sudo from the first principles, you, with all your experience and knowledge already there, would spend a lot less time doing it, and it would be way cleaner and simpler.
So while I'm not dunking on your effort and experience, I'm just pointing out that it's not impossible to take your experience and turn it into something better over a smaller timespan.
"Nothing" is too strong. It does not solve logic bugs, but type systems stronger than C can solve some logic bugs too.
Even something as simple as having some concept of "private" and "public" and some boundaries between them can help. I'm writing some code right now in Go, hardly a super strong type system, but I've still put some basic barriers in place like, you can have a read-only view of the global state, but the only way to write to it is to a per-user view of that state, and the only way to witness the changes to the underlying value is through one of those per-user write handles. This eliminates a large class of logic errors in which one accidentally reads the original global state when you should be using the per-user modified state or vice versa. This is a rewrite of some older code, and this error is so rampant in that code as to be almost invisible and probably in practice unfixable in the original code. (Which was solved in practice by only every dealing with one user at a time, and if there was multiple users, it simply ran the process completely from scratch once per user. It tried to cache its way out of repetition of the most expensive stuff, but, the cache keys had some of the same conceptual underlying problems, so it just wasn't as good as it should be.)
You can't solve everything this way. Rust's stronger type system offers more options, but you can't solve everything with that either. But with good use of types, there are still classes of mistakes you can eliminate, and classes of other mistakes you can inhibit.
(There are some tradeoffs, though; with bad types you can alse mandate incorrect usage. But I think in the case of something like a sudo replacement we can reasonably assume fairly high skill developers and that there will be a lot of high skill oversight, as evidenced by the fact they've already sought out a third-party security review.)
C does have some notion of visibility: put private declarations into the .c file instead of the .h file and declare static linkage. You could have a function that returns a pointer to const for read only data. Obviously they can cast that away, but other languages have unsafe escape hatches too. C also has static analyzers to help with some classes of bugs.
Cowboy code might be common, but you don't have to do that. If using something C-like, C++ definitely gives you a lot more tools to write safe code (or hang yourself, up to you) though.
C has "some notion" of a lot of things. That doesn't make them particularly usable at scale. C has the worst static typing of a language that can even plausibly call itself statically typed in the modern world.
C++ is an option to obtain the sort of thing I talked about, yeah, but in 2023 you need to use something memory safe for something as important as sudo, and C++ on its own is not. C++ and a great static analysis tool would be the minimum I would consider acceptable, but there is something to be said for things like Rust that build the analysis all the way in to the compiler rather than relying on external tools, and then future Rust external tools can build on that even more solid foundation if even more assurance is needed.
Enums, Option and Result types, absence of null, not to mention that the type system, borrow checker, and static everything by default, rewards encoding application state and state transitions using all these mechanics, such that they can be verified at compile time. I'd say the language does quite a lot to address logic bugs as well as memory safety. It can't protect a determined developer from themselves, but it provides incredibly useful tools to anyone who can work out how to use them.
Even if I don't like the design of Rust's borrow checker I still do appreciate how Enums/Option/Result types and pattern matching can make your code more robust. Really wish I can bring some of them to C++... I frequently use a poor-man's version of Result types with a `TRY()` preprocessor macro, but I'm often jealous of what Rust has in its toolbelt.
Generally the same idea, yes. Your parent mentioned a key difference though: "and pattern matching." enums in Rust have much stronger language support.
But there are also differences, for example, errors must be absl::StatusCode, whereas enums in Rust allow for arbitrary error payloads.
Also don't discount ecosystem usage: everyone uses Result in Rust, abeseil isn't used by most things, and std::expected has its own issues (though I can appreciate how tough making those calls is) and only landed in C++23, so it's not as widely used as Result either.
Sibling comment mentioned pattern matching, but didn’t point out the important point that the rustc compiler makes sure all patterns matches are exhaustive.
To use a C example, if you add a new definition/variant to an enum, suddenly all switch statements over that enum will fail to compile (unless there is a default: branch).
This does eliminate a large swatch of logic errors, though by no means all.
Static-everything is such a gimmick in my opinion. It sounds great until you try to do something useful with your code. It's almost never the case that people actually want to hard-code stuff in the source code.
Almost always you read configuration files at run-time (like sudo does) and change your behavior depending on run-time information - so you will have run-time errors.
I use rust, and it does have static by default in many places (for example it's hard to do the traditional OOP virtual polymorphism or to keep objects of various types in one container) and it makes it pretty hard for me to write "nice" looking code.
It usually devolves into a lot of nested if-else and switch (match) instructions.
I haven't run into that so much myself. What I have run into is trying to write C-but-in-Rust, for which the compiler yells at me to please knock it off. It got way easier when I gave up and committed to doing things the Rust way.
Not saying you haven't done that, just sharing my personal experience with it.
One of the habits I struggled to get rid of at first was the habit of making lots of things in structs refs for no reason, where it would be typical kn C for C reasons.
This led to a lot of unnecessary friction with the borrow checker while I was still getting to understand it.
Once I started always asking myself "does this really need to be a ref?", things became much easier. And this in turn revealed itself as a useful rule of thumb for keeping coupling between subsystem in check.
Then I was doing some C work and I realised I was asking the same question to myself about pointers too. And I found it could often reduce cognitive load down the line, because I'd end up in much fewer situations where I'd have to figure out "who should free this, and when?".
That's when I realised this cognitive load of keeping the borrow-checker happy was always there in C too. It was just more diffuse, and invisible until it blew up in my face. And when it did it was in the form in nasty runtime problems, not a helpful compiler error that happened before the code was even checked in.
If your program doesn't have a way of reloading its configuration at runtime, then even that first object created by reading the configuration from file can be immutable.
Yep! What I mean, though, is that the loading function itself will need to mutate the object as it reads settings from disk and updates the in-memory data structure. Once that's done, you can pass that around as a read-only object.
This gets said a lot, but I am coming to believe that the case is overstated. For two reasons:
1. Valgrind exists. It's not perfect, but it does arguably do a pretty good job as long as you're writing modern C. The biggest gap I'm aware of is that it can't really help you with global pre-allocated buffers. But I don't think that any language or tool can effectively protect you from information leakage if you're doing that sort of thing, not even Rust.
2. Memory-safe is not the same thing as secure. Programs written in memory-safe languages are rotten with security vulnerabilities, too. Rust's happening to be a memory-safe language that doesn't use garbage collection does not render it immune to this situation. It has some protections around concurrent usage of data that do add additional safety under certain circumstances (assuming you don't switch them off), but I doubt it's a panacea. I worry, though, that the Rust community's tendency to pitch this stuff as a security panacea could breed a culture of complacency that negates the advantages that Rust does bring to the table for systems programming languages. People tend to take unnecessary risks when they believe they're invincible.
The fundamental problem with valgrind is it only looks at what happened, not what could happen. Valgrind is great at making sure you don't have memory safety issues for "normal" inputs, but is basically useless at making sure your code doesn't have memory safety vulnerabilities when fed atypical inputs.
It's true that it doesn't eliminate all bugs in general, but it can completely eliminate buffer overflows for example.
There is no excuse to not at least have bounds checking. This is one of the most basic memory safety problems and it's trivial to prevent.
Just preventing this small issue will prevent a non-trivial fraction of bugs. I don't have sudo's bug list on hand but I wouldn't be surprised if 25% or more are caused by buffer overflows.
So even if it doesn't prevent all logic bugs, it cuts out a pretty big chunk of the bug list.
>assuming you don't switch them off
You can't switch them off.
>Rust community's tendency to pitch this stuff as a security panacea
I’d rather have safety default on with an opt-out, rather than the inverse that C gives you with -Werror -Wall -Weverything -Wyesireallymeanteverything. Compile it again one two different architectures, compile yet another time with clang-tidy and then static analysis with Coverity just to be sure. Run it with valgrind, asan and thread sanitizer. Sprinkle some fuzz testing on top.
Yet you still don’t the same level of confidence as a rust program that may have a small unsafe block in one corner of the code.
>It’s important to understand that unsafe doesn’t turn off the borrow checker or disable any other of Rust’s safety checks: if you use a reference in unsafe code, it will still be checked.
Unsafe rust basically just lets you use raw pointers, mutate static variables, use C-style unions, and do FFI calls, but otherwise it's exactly the same, and the safety checks are not in any way disabled.
The main thing is that pointers let you access whatever memory you want, and borrow checking the pointer value itself doesn't prevent this.
I don't think I would describe this as "switching them off", I would describe it as, "using raw pointers" or something along those lines.
Many vulnerabilities rely on crafting very particular inputs that trigger memory corruption in programs. Unless you happen to have fed that same input to your program when running it under Valgrind then Valgrind is useless for this case.
> It's not perfect, but it does arguably do a pretty good job as long as you're writing modern C.
There's no such thing as modern C. C code that's written neatly and meticulously looks the same today as it did 30 or 40 years ago, except for language changes such as the move from K&R declarations to function prototypes. C the language hasn't changed since 1989 except for minor things like mixed functions and declarations, the introduction of the long long type, restrict pointers, designated initializers, compound literals, and threads being in the standard library.
A lot of the most serious security vulnerabilities are memory safety because e.g. remote code execution is very often along the lines of "LOL, I smash buffer with machine code, it gets executed" and that's a memory safety problem.
For sudo you have potential for some very serious logic bugs, where the program does exactly what the programmer wrote, but what they wrote was not what they intended.
Rust's type safety makes it less vulnerable to these mistakes than some languages, but there is no magic. In C obviously a UID, a PID, a duration, an inode number, a file descriptor, a counter are all just integers. In Rust you could make all those distinct types (the "New type idiom"), and out of the box the Duration and the File Descriptor are in fact provided as distinct types. So, some improvement.
> In C obviously a UID, a PID, a duration, an inode number, a file descriptor, a counter are all just integers. In Rust you could make all those distinct types
For various kinds of IDs you can do that in C, too:
It may be not as nice as other languages, but it isn’t bad, either. If you use C++, it can be made a bit nicer, and you could also have such structs that you can calculate with.
You can technically do this but then you have to write wrapper functions for all relevant syscalls or libc functions to unpack the structure and call the actual thing. Lots of work.
Rust aside, one thing to consider is that a reimplementation of an existing piece of software does offer the benefit of being able to test the old version and the new version side by side for consistent behavior. You could have an entire class of test cases that is just "do X with the old version, and then do X with the new version, and just make sure the result is the same." There is also the entire bug history of the old version that can be investigated during reimplementation. If the old version has specific tests for each resolved bug, those can also be run against the new version to ensure it has consistent behavior.
In this case though, it's only a partial reimplementation: "Leaving out less commonly used features so as to reduce attack surface", which would complicate that approach.
Of course it's not. But hubris is possible and does not take years to master. And honestly statements like my overnight-rust-sudo is better than some poor peoples 40 years of work... well they don't really help Rust becoming more popular, but actually contribute to everything rust becoming even more irritating. Most rust tools are released with this pathos of "we fix what oldies couldn't get right with C". Not a great attitude to approach giants whose shoulders we're standing on indeed.
Who said anything about "overnight"? This project has been worked on for a year, implemented a test suite that found bugs in the original sudo, and have generally been respectful of the original work.
I think you might be projecting something here, there's no evidence for for your assertions.
I think a better question might be whether it prevents categories of bugs that are more likely to be exploitable than, say, the logic errors that no language could ever prevent?
Also, it sounds like your seasoned eyes would be valuable in reviewing this code.
It can eliminate many bugs, but it certainly wouldn’t eliminate all bugs. During implementation they realized they were not implementing sudo’s (undocumented) feature of failing to run if the sudoers file is world-writable: https://ferrous-systems.com/blog/testing-sudo-rs/.
Of course they did find and fix the bug, but in general Rust isn’t going to protect you from bugs like this that are essentially logic errors.
That is documented. Since the mercurial web interface isn't very nice to use I picked a random version. sudo 1.8.6 from 2012 writes in the man page "The sudoers file must not be world-writable,".
This is also a very common behaviour for security sensitive applications to check config file permissions. Another example I remember are ssh private keys.
I might be to harsh but it is not so trustworthy they still made this error and still miss the documentation.
I’m not sure why people are downvoting you. I suspect they may be clicking the link and thinking ‘that’s not documentation it’s source code’, not realizing it actually _is_ documentation.
Interesting, the posting I linked to indicated this behavior wasn’t documented. It’s certainly not surprising and as you mentioned, it’s equivalent to openssh requiring specific permissions on private key files.
I might be too harsh, but it is not so trustworthy that they found bugs within the original sudo after less than a year of effort. those other devs had over 40 years to find it
On the surface, sudo seems fairly straightforward, so it’s interesting to hear how much work has gone into it! Do you have any interesting facts or anecdotes you’d care to share?
great story! also, TIL that I've been pronouncing `sudo` wrong, I was 100% sure that it was supposed to be like pseudo, but I guess that is a myth :)
It's so great to be able to listen and learn from the people that invented these important building blocks themselves, I feel lucky. Thanks for sharing.
The key is in "on the surface". While the common usage of sudo is fairly straightforward, you me and most people use like 5% of it. The trick is in all the side shows.
Makes you wonder then why it does so much, if those rarely used features increase the surface area of possible exploits? This is just a question I’ve had about *nix utilities in general, since sudo is hardly the only tool with obscure flags and features
Because the long tail of features is useful to someone. Mind, I like doas for this reason, but having the more feature rich option available makes sense.
Yes, have a read of the sudoers man page and marvel at the complexity of the configuration, and wonder about your chances of getting it right if you are not well-experienced. This is the config file with the infamous paragraph:
The sudoers grammar will be described below in Extended Backus-Naur Form (EBNF). Don’t despair if you are unfamiliar with EBNF; it is fairly simple, and the definitions below are annotated.
OpenBSD replaced sudo with their own "doas" command a few years ago; the doas.conf manual page is about 100 lines; sudoers is over 2,000.
> Is there something to this Rust language which magically eliminates all chances of any bug being introduced?
no, altough it has features that prevent or reduce the probability of some types of bugs - one example of this being memory safety bugs. rust can't prevent logic bugs.
the rust reimplementation probably has more bugs than the original, but a theoretically better chance to achieve fewer bugs in the long run.
is rewriting mature linux infrastructure in rust a good idea? many people agree that no, it's probably not a good idea outside of special use cases.
> how is it possible for a single dev team to totally reimplement it without unknowingly introducing at least a bug or two?
As someone with over three decades of C programming experience (so not as much as you), maintaining widely used stuff written in C for decades, that has recently switched from C and C++ as main languages for systems programming to Rust, I'd instead ask this: How is it possible, even given 43 years of working the problem, to create a program in C that does what it's supposed to, and only what it's supposed to?
But also, one of the answers from the article is "Leaving out less commonly used features so as to reduce attack surface". Most security bugs in sudo are in features I don't use.
Rust isn't just memory safe. It's also orders of magnitude harder to accidentally make other mistakes, such as race conditions.
> How is it possible for a single dev team to totally reimplement it without unknowingly introducing at least a bug or two
This is possible if every bug fixed has an associated test. If they use this battery of tests to test their new implementation it should be as good as the original implementation.
I’m sure there is a logic bug or two in the new implementation. Whether you want to take the risk of new logic bugs for the benefit of removing several whole categories of bugs (both known and unknown!) is the question and tradeoff in each case like this. This too requires scrutiny, but I’d be a lot more comfortable running a Rust program with 2 years of scrutiny than any C program with 40.
You gotta start somewhere right? I mean its not like you got it right the first time. Don't everyone go switching over just yet, but people can't scrutinize something that doesn't exist.
Thank you for working to create one of the tools which is obviously on the list of the most valuable and beneficial to computer security. Perhaps only second to netfilter.
And I'm really sorry so many people have decided they're going to imply something is wrong or broken with it for their own clout. Or because they've bought into the lie that no code written in C can be safe or correct.
For what it's worth, I and all the engineers I willingly associate with (read: the ones who I respect) all have said the exact same thing. Switching to rust here, just 'cause, isn't going to meaningfully increase anyone's security. But what are you gonna do. Other than ask people to be honest?
Annoying fanboys aside... again, *thank you*! The computer security world is meaningfully better because of your work, and that's something the RIIR fad will never be able to replace :)
The link you cite says it was worse in the Rust version:
> During the audit, it came to light that the original sudo implementation was also affected by [CLN-001: relative path traversal vulnerability], although with a lower security severity due to their use of the openat function.
Re: I am unsure where you got this. It's the same vulnerability.
> During the audit, it came to light that the original sudo implementation was also affected by this issue, although with a lower security severity due to their use of the openat function.
> Like building an entirely new car company around only making side-impact collisions safer
...which still results in safer cars overall, so I don't see the problem. Especially if those cars are almost completely immune to side-impact collisions and if it's actually not a car company but a technology every manufacturer can use for future products.
You don't see the problem in starting an entirely new car manufacturer from scratch just to fix one safety issue?
> if it's actually not a car company but a technology every manufacturer can use for future products
In that case it's like every single manufacturer changing their engine design in order to have a different wiring harness with a slightly thicker shielding around a single cable. The amount of work and cost involved, and risk to every other part of the process, just to fix one tiny thing, makes no sense. It is an insane amount of work for extremely little benefit.
> it's like every single manufacturer changing their engine design in order to have a different wiring harness with a slightly thicker shielding around a single cable.
Car manufacturers improve their engine designs all the time.
It's also not uncommon that programs get rewritten in order to achieve better results, and recently Rust happens to be a popular choice where both speed and security are important. There's nothing extraordinary about it really.
Thousands of developers and security experts have gone over it. So part of me wonders - how is it possible for a single dev team to totally reimplement it without unknowingly introducing at least a bug or two? Is there something to this Rust language which magically eliminates all chances of any bug being introduced?