I have a Dodge Ram. Last night I had a 400km drive to do after a very long day. I wasn't exhausted, but I certainly felt like I did not want to drive an extended period of time.
I have a Comma 3x in the truck and felt way more confident, alert and comfortable for the entire drive. OpenPilot/Sunnypilot/Frogpilot are not FSD, but they are hands off driving assistance. The 2020 Ram performs incredibly well. The latest driving models are very smooth as well, no ping-ponging and they handle passing and traffic extremely well.
A legacy car maker would be smart to acquire Comma if its for sale. They would be extremely close to a viable assisted driving capability with it.
I used Open Pilot for ~4 years. According to connect I have 8,000 miles on Comma 2, 20,000 on Comma 3 and 2,000 on Comma 3x. I recently sold my Rav 4 and went to a Tesla. Open Pilot is actually better in a lot of ways than default Tesla auto pilot, especially because it doesn't do crazy fantom braking on freeway. Open Pilot is also way ahead of pretty much every lane assist / adaptive cruise control systems.
Obviously, FSD is way ahead of e2e open pilot with navigation, but since Open Pilot can apply very little torque to the wheel, it can't do anything gnarly. I actually trust Open pilot more at this point but I guess I just need more time with FSD. Some of that is because longitude was Toyota controlled until I used the e2e longitude model more.
Even on "chill" mode, FSD will make random quick lane changes to turn only lanes to try to get around traffic. This is 12.5.2. Even so FSD can get me from point A to B with no interventions 98% of the time.
There should be an option in FSD to have it not pass on the right and to change what speed difference it will wait to pass for (for example only pass when the car in front is [5] mph slower than what I want to go). These are separate options to look into than chill mode, and could also fall short to Comma, but thought I’d share in case you didn’t know they were there.
Mind boggling to me that a non-ping pongy lane keeping is not standard in cars. Is it standard in luxury cars? Seems like an obvious thing to add/upsell.
Non ping-ponging lane following assist is already available in many cars including KIA and Hyundai models. They're very conservative and disengage very easily. I think it's by design to minimise their legal accountability
Not just legal accountability, but actual safety. They are designed so that they do not give the user a false impression of the extent of their capabilities.
I've been incredibly surprised to see that lane assist in my Kia is significantly better than that of most other (legacy non-hi-tech, think nicer hondas and lexus ICE/hybrid) cars I get a chance to drive.
I unfortunately don't have radar cruise control on my Kia, though, which would make highway driving even in traffic completely effortless, and this seems to be standard on themore expensive cars. Maybe it's for the better, though, because it does force me to be much more attentive on the road.
Hyundai actually has two systems, LKA and LFA. LKA just tries to bounce the car back when it detects lane edges, LFA actively keeps the car in the middle of a lane.
All Hyundai models in Europe have LKA, some (more expensive) also have LFS.
My 2019 Audi S5 was excellent at this. It would ping pong at most once then auto-correct itself to be perfectly centered in the lane.
It did some weird things like if the car in front of you was driving a bit too far to the left/right of a lane, it would copy them. Other than that it was nearly perfect, though. Never had it take an exit by accident, etc.
Their tuning on when to accelerate/brake and make it smooth needed a fair bit of work, but I found that switching the drive mode from Dynamic (Sport) to Comfort changed the eagerness of the system and smoothed things out.
> It did some weird things like if the car in front of you was driving a bit too far to the left/right of a lane, it would copy them
Wouldn't that conceptually be the right thing for the software to do, copy the human in front of it (unless it has demonstrably better information)? OT1H, "lemmings," but OTOH unless the whole line of cars were all on openpilot my life experience has been that the person in front of me by definition has more visibility than I do since their car is not blocking their view as it is mine
I am totally talking out of school, because I'm not in that space and my poor BMW chose to do its own thing[1] so it doesn't work with openpilot[2] -- although they have a dedicated #flexray channel[3] so hope springs eternal
> Wouldn't that conceptually be the right thing for the software to do, copy the human in front of it (
I see people failing to follow the rules for bad reasons far more often than for any good reason. I don't want my car driving off to the side of the lane just because the car in front isn't centred. It should assume the right thing to do is to follow the rules, and hand off to me in cases that are more complicated.
Ugh. So I’m working on a fork of openpilot and the way the OP model is designed, it has its own rules that is not rooted in any legal driving framework for any state. The simple one is staying right. My state says your vehicle must stay on the right side of the road including roads without markings. OP will try to drive in the middle of the road. Another one is how OP does not distinguish people from parked cars or how oncoming cars are not tracked but simply an object the car should try to avoid (though it does not do this very well and experiences frequent disengagements due to it)
Obviously a model which manage these conditions would fair better but the comma hardware is fairly underpowered for any stronger use case.
I have added dedicated compute to my car to handle a lot of driving rules but now my solution is independent of comma. I tie into the LVDS display on the console so the integration is immersive, but it also means I don’t need comma for the hardware. My fork is also starting to diverge from OP so I may have a competing (but tangential) product!
I also notice this phenomenon in Audi. It’s as if the steering motor is applying inputs after the steering setting has been applied. So if your steering wheel is in sport mode then the motor requires additional force to turn.
I run my own forked copy of openpilot and the car cannot keep up with turns in dynamic mode. When set to comfort it can handle all turns with ease.
Also, unless I am misunderstanding the situation, since the code is MIT they don't need to acquire Comma to take advantage of the situation. I'd strongly suspect they all want to roll their own implementation for liability reasons, not strictly technical ones
I can’t wrap my head around the fact that 275 car models include all the actuators needed for self driving driving and there’s some kind of port third party software can hook into.
Level 2 driving assistance is commonplace on many new vehicles, often as a standard feature. They are just significantly more conservative in their functionality compared to the level 2 offerings from Tesla, and marketed as safety features rather than "self driving" features.
It's important to note that nothing we're talking about here is actually "self driving" per SAE standards. Openpilot, Tesla's Autopilot/FSD, Honda Sensing, Toyota Safety Sense, Hyundai SmartSense, etc are all level 2 driving assistance features.
This turns level 2 driver assistance features into ... nicer level 2 driver assistance features.
"Fully self driving but a human has to watch" is nonsensical. Responsibility for observing the road and making decisions is a very important and inseparable part of driving. Mechanistically manipulating the steering, brakes, and accelerator is not the hard part.
Vehicles with levels 0-2 of driver support are never self-driving at any moment of operation. These systems might be able to manipulate the controls of the vehicle, but they are not good enough to make decisions without constant supervision at all times.
Level 3 is partial self-driving, where the vehicle can assume responsibility for driving under some narrow circumstances.
Full self driving doesn't happen until level 4.
I think the reason that there's no decimal points is because J3016 is really a road-map of the milestones on the way to full automation, not a buyers guide.
In which case the number should be multiplied by the percentage of cases it can be activated in, if you can Level 3 self drive in, eg "up to 37 mph but only on highways" so you multiply 3 by 0.001 because it's only useful in 0.1% of conditions, and then you can add that to the times level 2 works.
That's not possible to calculate because the total denominator of possible "cases" approaches infinity. Level 3 doesn't define what the cases for operation are, that is up to each individual implementor of such a system to define. The criteria is likely to be more much more complicated, like "the output of all 12 sensors when applied to this algorithm, and referenced against this mapping data, delivers a confidence greater than x"
Things don't really get that simple until level 5.
Even at level 4, we're going to have vehicles that pull over when they can't figure out how to navigate successfully.
I’ve been saying this since Tesla released autopilot because I personally think teslas implementation is the best but there isn’t an easy way to compare it to something like radar cruise with lane keeping other than saying they’re both level 2.
They also lie about models that are supported and won't assist you when you run into that. Had to return one myself. Found no evidence anywhere that my model/year was ever actually supported and anyone was using it either.
You can find a few attempts of people trying to get it to work in their discord with no clear positive outcome, discord is unfortunately not search indexed.
Using discord, etc for any open source project discussions is really unproductive.
Get it that people like the immediacy of chat style communication. But it seems to encourage way more noise and less thoughtful dialogue. The worst aspect is that it locks away a lot of useful information. Trying to dig through chat histories for information is a horrible experience.
> But it seems to encourage way more noise and less thoughtful dialogue.
But compare it to something like Microsoft support forums and you immediately realize why people are using it. It might not be the best implementation of chat, but the immediacy and lack of formality are undeniably valuable.
Yeah it was on the supported list when I bought it in October of 2023. The website asks your make/model/year when you buy if I recall correctly, that's how it tells you which harness to get.
I found out I needed to update some files in the firmware, followed the guide for that. Asked for help in the Discord. I could never get it working though and returned it after a week. I was going to hang on to it and harass people on Discord, but didn't want to lose track of time and go beyond the return window. Believe me, I really wanted it to work.
Looks like it was added to the supported list before the explicit support I guess
I suspect Openpilot sends packages to CAN bus to do steering but what makes the physical wheel rotates? I'm confused. Or when a human makes a correction to auto steering, perhaps there could be some kind of full duplex synchronization mechanism
It’s cool to see that it’s possible to innovate sustainably, in a niche. So refreshing to see something stick around rather than become a bubble with a billion dollar valuation that either takes over the world and enshittifies or implodes and takes the product with it.
Make business chill.
That said: The larger they get, the more regulator attention they’ll attract. If some government entity wanted to, they could probably easily kill them.
IIRC the actual model is private and only downloads once you activate the device so presumably if the company was shutdown you couldn’t use the device anymore unless someone shelled out for training
For those similarly curious, I went sniffing around the PRs and found that, yes, there are two .onnx (via git-lfs) in https://github.com/commaai/openpilot/tree/v0.9.7/selfdrive/m... - one for the forward camera and one for watching the driver. The driver one had a lot more features than I expected, but I'd guess they are related to the NTSB certification I saw mentioned elsewhere in the thread here. Regrettably, because I don't have a Panda-compatible car, I actually can't try out the self-hosting setup but it certainly seems plausible
I chose the Hyundai Ioniq 5 as my current car specifically because it’s compatible with OpenPilot. It’s been a total game-changer for my driving experience. Just like their tagline says, “make driving chill,” and for me, it truly delivers on that promise.
This reminds me of Waymo's approach to self-driving cars. Paraphrasing, but basically they found that progressively adding self-driving to help human drivers is bad, because it leads to the humans becoming complacent and not paying enough attention. Therefore they decided on an all or nothing approach, where their cars would be only and entirely self-driven.
This always seemed like a bit of bull from waymo. It's not an easy problem to work with existing manufacturers to give a better and or cheaper solution... Especially when there are established competitors with efficient verification and validation processes (that every manufacturer requires).
They decided it wasn't worth explaining that their techniques don't generalise to a driver assist. It would not be good or cheap enough to be worth developing the compliance and integration frameworks.
The first thing Waymo tried building (way back when, circa 2010 or so) was highway based driver assist in the style of Autopilot. They did a ton of testing with it and didn’t like how quickly their testers stopped paying attention despite a ton of instruction not to. I’ve seen clips from these tests.
It’s also possible they shifted direction because the long term vision of robotaxis is much more lucrative.
The thing is, you can tune driver assist to whatever form you like; but then you have a true comparison of function with established competitors.
I just don't believe the approaches for high autonomy (especially at the time) actually could make a cost effective assist system.
And for whatever reason they decided to push the we didn't like the driver behaviour message, rather than actually talking about what was actually plausible to achieve in the driver assist space.
Unless you assume that self-driving software is perfect, no, it really isn't. That's the whole problem - the drivers would get complacent, so when there's an issue, they'd be caught by surprise and wouldn't be able to react.
Isn't the point to pay _no_ attention? The difference is when an accident occurs, was the person in the car at fault for not vigilantly watching everything.
You want the curve of total attention to be always above a baseline human in an unassisted car. The car can do some attention and the human can do some. But if the sum of the two falls below the threshold, you’re in trouble.
You can already do that by just closing your eyes and letting Jesus take the wheel. No, the point is doing so while maintaining safety.
It is materially less safe to operate a ADAS while distracted than driving manually. Humans are exceptionally good drivers on average, only encountering minor crashes on timeframes measured in years to decades. As such, if safety critical ADAS errors occur more frequently than every ~100,000 miles and you are attentive in less than 100% of all such occurrences, you are operating your vehicle multiple times more dangerously than the average driver (which is a number that includes drunks and distracted drivers).
That is why it is critical to deliberately downplay the capabilities, to avoid wishful over-reliance, and enforce strict driver awareness (through techniques such as driver monitoring) to avoid operating multi-ton killing machines in ways that are multiple times more dangerous to the occupants, other drivers, and pedestrians. Without that, people are prone to over-generalization of safety capabilities, extrapolating that a single success means robust, continued success thousands to tens of thousands of times in a row.
That statement from Waymo always struck me as deeply uninsightful because it was really just a more complicated way of saying “self driving systems need to be good enough to drive the car on their own” which isn’t just obvious, but tautological.
"THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY. THIS IS NOT A PRODUCT. YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS. NO WARRANTY EXPRESSED OR IMPLIED."
> this can be used wherever the driver deems it safe and useful
With the disclaimer that this depends on the location of course. For example, I think in Spain (and probably EU wide?) modifications that affect steering and throttle control would need to undergo local homologation before you're legally allowed to drive with that on public roads at all.
Which, to be honest, makes a lot of sense. I don't think anyone would be happy if cars start using software MVPs automatically controlling throttle and steering while in real traffic.
I mean let’s be frank here. The sort of people who are enthusiastic about this also aren’t going to be stopped by the law especially with the low risk of getting caught.
If I were to personally install this and use it anyways, my biggest worry would be the additional punishment if I cause an accident. Not sure exactly what the punishment is for driving a vehicle you know isn't road worthy, but I bet it aint pretty.
While the chances of getting caught are low, I wouldn't want to risk this for insurance. If you are involved in a crash for any reason I don't think they are going to cover it if you modified this stuff on purpose.
Suppose I deem it safe and useful for my 6yo to drive for a while, using his Xbox controller from the passenger seat.
It is illegal in many countries for a device (or anything else) to obscure any part of the driver's forward view (area swept by wipers). So even without actually controlling the car, we have an unlawful vehicle.
It really bothers me when people make statements like this. I propose a bet:
If you’re so confident this will drive my car into a ditch, then front the money for me to buy a compatible car & this kit. If it drives my car into a ditch, I’ll pay you back double that money.
If a piece of software that has the capacity to kill people is provided without any kind of assurances or warranty with the sole responsibility placed solely on the user, then that already tells me all I need to know, no bet needed to settle it.
Would you buy an angle grinder that's specifically been designed to not have a guard so it can be more useful, made by random contributors on the internet without any official certification? I'll stick with the ones that needed to pass the CE mark and you can sue the company responsible if it chops your arm off thank you very much. They at least have a required level of anxiety needed to patch anything serious knowing the level of responsibility they carry and what's coming if they mess up.
They’re just trying to scare away people who thing they can chuck this on their car and suddenly have a self-driving robot that they don’t have to pay attention to.
It’s my understanding that in addition to the cameras it also uses the sensors already built in to the car which would include blind-spot detection, no?
Some cars that have BSD it will work with. My car uses it, but don't forget the lane changes are not automated by default. A user must turn on the blinker and nudge the wheel by default. Positive BSD sensors read on CAN-BUS will be read by OP and it will not perform the rest of the lane change. This is how it works on my car (albeit, I don't run default so I just need the blinker).
It says my car is supported and my car doesn't have any blind-spot detection, nor does the requirements list that as needed, so maybe it's optional but not required?
There's an element of using these systems that people don't usually discuss. One of the forks, Sunnypilot, enables a mode where you control the pedals but don't have to do the steering. What I've found is you do pay attention in this mode, and since you're controlling the brakes you can easily avoid most issues that you would get with self driving. But, not having to have your hands on the steering wheel makes the experience a lot more pleasant. Also, with traditional lane keeping systems, the so-called "longitudinal" control (accelerator and brakes) is where in my experience the system makes the most mistakes.
I think this mode is something car manufacturers should enable in general. I actually suspect it's significantly safer than completely hands- and feet-free driving modes, and you get most of the benefit of lane-keeping assist.
2. They discuss functional safety, and lots of testing, which is great. But I’d want to see some data on the test
results — maybe this exists and I just didn’t see it?
3. It makes me uncomfortable that the anecdotal videos are easily findable, but bulk data/statistics are not. Anecdotes can easily be cherry picked. I get that they’re necessary for marketing, but I don’t feel they’re sufficient given the product’s purpose
This is a modern ADAS system, but a lot more stable. The driver is always liable for what their vehicle does. It's not claiming to be FSD and it's very apparent when the system will need additional inputs from the driver.
I have 15k miles on it. Was able to retrofit a friend's 2015 car as well with a bit of additional hardware, and he likes it. He also has FSD on a model3. But OP or FSD, driver always has to pay attention and add their inputs.
Most likely not legal to invalidate in EU. There is laws that’s say that ev everything you can do manually, you are allowed to automate. Any rules against that is null and void.
The “horse winning race” case is a known one where they go into this.
Geico will cover you. You can disclose ahead of time that you have an aftermarket ADAS if you want. If it drives you off the road, it will be as if you drove off the road and you will be declared at-fault, of course.
Been following Openpilot for awhile, would totally use it if I had a supported car. Though these days most cars come with 'pretty good' ADAS, even 'hands free' in some situations, so I wonder how much it's "worth it" to DIY compared to factory default.
I do think that within the next 5-10 years most cars will be able to hands free highway as reliably as OP. However, cars keep getting more and more expensive. One can buy a 2015-2024 used car for much cheaper and get some very good highway cruising out of it. That's what I did and am very happy about it.
Without taking away anything from the substance or achievement of this release, I find phrases like "openpilot is an operating system for robotics." always quite fishy.
No, it's not an OS for robotics. You can't do actual robotics stuff with it, like drive actuators to control limbs or grippers, do motion control or SLAM or perception or any of the usual robotics stack.
Their website correctly says openpilot is an open source advanced driver assistance system that works on 275+ car models of Toyota, Hyundai, Honda, and many other brands. Should've stuck to that.
Thinking about it some more, it's probably just another engagement baiting strategy to get attention and I'm their gullible puppet. Well played.
George Hotz says: "we developed a proper successor to ROS. openpilot has serialization (with capnp) and IPC (with zmq + a custom zero copy msgq). It uses a constellation of processes to coordinate to drive a car."[1] And Comma sells a robot that runs Openpilot: https://comma.ai/shop/body
> You can't do actual robotics stuff with it, like drive actuators to control limbs or grippers, do motion control or SLAM or perception or any of the usual robotics stack.
A lot of the "usual robotics stack" is not going to be relevant for the new wave of consumer robotics that is coming soon. It will be enabled by end-to-end machine learning and stuff like traditional SLAM methods will not be a part of that. The Bitter Lesson[2] is coming for robotics.
I enjoy Hotz as a hacker, but I'm really allergic to this kind of oversold language. "[W]e developed a proper successor to ROS" is a past tense statement, as if they've already done this thing. In reality, at best they have presented a roadmap for a thing that could approximate ROS one day.
In the robotics community, the stuff coming out of George Hotz has always been considered a kludgy mess, and unsuitable for serious work. Dude is a talented hacker, but the idea that this will replace ROS is kind of a joke.
The point of the bitter lesson is "leverage compute as best you can" not "use DNNs everywhere just because". Oftentimes your available compute is still a crappy ARM machine with no real parallel compute where the best DNN you can run is still not large nor fast enough to even be viable, much less a good fit.
And well some classical algorithms like A* are mathematically optimal. You literally cannot train a more efficient DNN if your problem needs grid search. It will just waste more compute for the same result.
Besides, the nav stack is not really the point of ROS. It's the standardization. Standard IPC, types, messages, package building, deployment, etc. Interoperability where you can grab literally any sensor or actuator known to man and a driver will already exist and output/require the data in the exact format you need/have, standard visualizers and controllers to plug into the mix and debug. This is something we'll need as long as new hardware keeps getting built even if the rest of your process is end to end. It doesn't have to be the best, it just needs to work and it needs to be widely used for the concept to make sense.
The future of consumer robotics will not be built on "a crappy ARM machine with no real parallel compute". Traditional robotics has failed to produce machines that can operate in the real world outside of strictly controlled environments, and more of the same isn't going to change that. Fast hardware for running DNNs will be a hard requirement for useful general purpose robots.
I agree that it'll be needed, but that hardware that can provide enough compute at acceptable wattage is yet to materialize. Only once that changes the equation will change. Today you'd be surprised how many production UGVs run off an actual Pi 4 or something in a comparable compute ballpark.
With due respect, this has to be one of the most ignorant takes on robotics I have read in a while. Yes, you can always slap serialization and ZMQ on your framework. That doesn't make it an OS.
And no, the usual robotics stack is not going away anytime soon. Maybe develop some actual useful robots before posting like an expert on robotics topics.
I believe the idea is that openpilot replaces the usual robotics stack with an end to end neural net.
While I agree operating system is usually a marketing term, it does feel correct in this case as it is the operating system for the Comma Three, which can operate cars but also this thing: https://www.comma.ai/shop/body
Could someone explain the joke? I've been dabbling with learning robotics and I've been confused by how ROS and ROS2 both appear to be actively developed/used. Is ROS2 a slow-moving successor version (like Python 3 was) or a complete fork?
Slow-moving successor, which the community isn't exactly going wild over. It offers modest improvements in exchange for a painful upgrade process, and many of the original issues with ROS1 remaining unsolved.
The other half of the joke is that ROS was never an operating system either.
Well there is one thing that ROS 2 does better, you can declare params directly inside nodes and reconfigure them all without building extra config files. And it doesn't stop working if your local IP changes.
But the rest are firmly downgrades all around. It's slower (rclpy is catastrophically bad), more demanding (CPU usage is through the roof to do DDS packet conversions), less reliable (the RMWs are a mess), less compatible (armhf is kill). The QoS might count as an improvement for edge cases where you need UDP for point clouds, but what it mostly does on a day to day basis is create a shit ton of failure cases where there's QoS incompatibility between topics and things just refuse to connect. It's lot more hassle for no real gain.
Config generally feels more complex though, since there isn't a central parameter server anymore. The colcon build system also just feels more complex now, which I thought was already impressively complex with catkin.
Yep it takes super long to get parameters from all nodes cause you need to query each one instead of the DDS caching it or something.
And yeah I forgot, there's the added annoying bit where you can't build custom messages/services with python packages, only ament_cmake can do it so you often need metapackages for no practical reason. And the whole deal with the default build mode being "copy all" so you need to rebuild every single time if you don't symlink, and even that often doesn't work. The defaults are all around impressively terrible, adding extra pitfalls in places where there were none in ROS 1.
No it’s much worse, python3 was all round better, it just took a while to get all your dependencies ported which made the transition hard. Judging by the comments it doesn’t seem like people agree that ROS2 is even all round better from ROS.
It's funny this topic came up today because I have a group of students working on a ROS2 project and at our meeting this afternoon they had a laundry list of problems they've been having related to ROS2. I'm thinking our best option is to use ROS1...
You're right ROS2 isn't all round better than ROS so the transition will never happen fully.
FWIW I'm working on an actual replacement for ROS, I'll post it to ShowHN one day soonish :P
Isn’t the software for training end-to-end NN to be used in automation?
Just a first version that it’s used for cars, and they have been using it for their own robot.
This seems to be a mix of C++ and Python, including a script called "realtime.py" (oxymoron?). So am I now exposed to other people using Python on the roads to operate heavy machinery?
So what, you want everything written in RUST on a linux kernel with hard real-time patches? It uses machine vision anyway, which has no hard guarantees at all. The software it uses to detect lanes or cars is probabilistic by it's very nature.
Python does pretty good at soft real time if you manage your own event loop and disable the garbage collector, and you're a lot less likely to get "crash the entire stack" style memory allocation bugs. Sure, GO or RUST would be better, I think CPP could be worse if handled inexpertly.
Python has segfault issues, surprising exceptions and version incompatibility.
I've been using Linux/BSD for over a decade now. No C or C++ application has ever crashed, I cannot say the same about Python applications. Outright segfaults are rare but happen. Rogue exceptions are much more common and could basically have the same detrimental effect on a self-driving system as a segfault. And let's not talk about logic bugs due to version incompatibility and the obsessive rewriting of those who took control over CPython.
Ahh, you've been running some grad students first python project as if it was a serious project like curl with 20 years of history, and expecting it to have the same quality. But you've somehow avoided the tons of grad-student CPP programs with similar quality issues, or the broken code pushed by companies like crowdstrike or IBM.
Fair enough, your experience may vary. I'd suggest not judging the language by the standards of some hobbyist code that just so happened to end up on github. I've had tons of bugs in c/cpp programs over the years, some more critical than others.
I've seen a lot of shitty and unreliable python code, and a lot of good and mature C/CPP projects. I've also seen really bad security issues and crashes with bad C code, heartbleed, crowdstrike, etc.
For what it's worth I've never had youtube-dl hard crash on me, and I could argue that it's a more complicated problem to solve than what curl is solving. In an apples-to-apples comparison I think it does pretty well.
No matter what language you use for this you're going to be relying on an AI vision model with no hard guarantees.
Actually Python was insufficient for the sort of grad student bugs I wanted to write, I was able to just wrap everything up in giant try blocks and then,
except:
print(“Something happened”, i)
(Where I might be an index. Or an element).
Fortran is able to generate better bugs, because it has allocate/free.
You have much more control over a pure C/C++ application because it does not involve the Python runtime. Crowdstrike etc. are exploits that don't really matter here: If you are on the CAN bus it's game over already.
That said, I'm pretty sure CPython has exploits, too. They'll be harder to find and trigger though.
Sure, runtimes exist and have engineering trade-offs. You avoid a whole class of memory related bugs but you lose a lot of control over memory allocation. You can do soft real-time as long as you manually manage the garbage collection and accept that there will be some (bounded) jitter on memory allocations.
The first rule of the tautology club is the first rule of the tautology club. Things have trade-offs. Python removes (or at least significantly reduces) a whole class of bugs that appear when using lower-level languages, that's part of why it's a good glue language.
The interface between the openpilot and the car is a standalone device (the panda) that provides and enforces the safety model. All code is written in C to automotive safety standards including ISO26262, ISO11270, ISO15622, and MISRA-C. 100% line coverage for all safety unt tests.
They also run pretty extensive tests (regression, unit, hardware/software-in-the-loop, mutation, and vehicle specific) on every commit and have actual hardware devices continually running real routes looking for regressions.
Just so we're all clear here, there is a lot of gobbly-gook in this answer which is either off target or irrelevant.
> a standalone device (the panda) that provides and enforces the safety model
What the actual safety model is that is being enforced is far more important here. The safety model could be "there is no safety guarantee whatsoever" and this sentence would still be true.
> All code is written in C to automotive safety standards including ISO26262, ISO11270, ISO15622, and MISRA-C.
26262 says practically nothing about software, what you really want is 21448. And 11270 and 15622 are super low targets for the amount of control authority available here.
MISRA-C is mostly a waste of time when it comes to safety. It gives software developers the warm blanket of having a checklist they can tick items off of, but does little to prevent unsafe systems from being built. Programmers have gotten pretty good about at least using tests and other analysis tools to make sure they're not doing the wildly stupid things that MISRA tries to prevent.
> 100% line coverage for all safety unt tests
100% like coverage is also rather trivial to achieve and doesn't say much. Branch coverage would be better, but being able to make some claims about state space coverage with exposure numbers would be what I'm expecting here.
> So am I now exposed to other people using Python on the roads to operate heavy machinery?
yes in the sense that python is running the ML models and deciding what the vehicle should do, but it is heavily bounded in what it can do by the safety model which is implemented in bare-metal MISRA C running on the microcontroller that interfaces between openpilot and the CAN bus (panda). It enforces things like accel/braking limits and steering rate limits along with consistency checks, heartbeats, vehicle status checks, etc.
Level 2 self driving is already only a best effort system so if python caused an issue it would just fall back to the safety model on the panda and ultimately the driver to operate the vehicle safely.
Sort of. It operates after your vehicle safety systems, so yes, python is controlling the car to an extent, but only within the limits of the built in safety systems.
I’m surprised to see so many people disliking Open Pilot on HackerNews. I have one of these, and it’s a total game changer on long trips. I drove from Texas to California using my Comma 3 and I didn’t have to overtake it a single time on the interstate.
Sure you have to actively be alert your entire drive, but it’s still significantly better than actually doing the work of driving.
I've only driven a driver-assist system once and kind of hated it, but I've always assumed that the mental work of remaining attentive as a passive participant would actually be more mentally taxing than just driving. It seems like you're inevitably going to lose attention.
I always wondered who would pay $1000 for a device you have to babysit, just so that you can become an unpaid beta tester and data drone for SV millionaires. I guess all you have to do is tell people they're a part of a revolution?
Or want your car to drive itself. I only drive long distance and my Comma has completely changed how often I visit family and improved my safety coming home from exhausting weekends.
I have a Dodge Ram. Last night I had a 400km drive to do after a very long day. I wasn't exhausted, but I certainly felt like I did not want to drive an extended period of time.
I have a Comma 3x in the truck and felt way more confident, alert and comfortable for the entire drive. OpenPilot/Sunnypilot/Frogpilot are not FSD, but they are hands off driving assistance. The 2020 Ram performs incredibly well. The latest driving models are very smooth as well, no ping-ponging and they handle passing and traffic extremely well.
A legacy car maker would be smart to acquire Comma if its for sale. They would be extremely close to a viable assisted driving capability with it.
I used Open Pilot for ~4 years. According to connect I have 8,000 miles on Comma 2, 20,000 on Comma 3 and 2,000 on Comma 3x. I recently sold my Rav 4 and went to a Tesla. Open Pilot is actually better in a lot of ways than default Tesla auto pilot, especially because it doesn't do crazy fantom braking on freeway. Open Pilot is also way ahead of pretty much every lane assist / adaptive cruise control systems.
Obviously, FSD is way ahead of e2e open pilot with navigation, but since Open Pilot can apply very little torque to the wheel, it can't do anything gnarly. I actually trust Open pilot more at this point but I guess I just need more time with FSD. Some of that is because longitude was Toyota controlled until I used the e2e longitude model more.
Even on "chill" mode, FSD will make random quick lane changes to turn only lanes to try to get around traffic. This is 12.5.2. Even so FSD can get me from point A to B with no interventions 98% of the time.
There should be an option in FSD to have it not pass on the right and to change what speed difference it will wait to pass for (for example only pass when the car in front is [5] mph slower than what I want to go). These are separate options to look into than chill mode, and could also fall short to Comma, but thought I’d share in case you didn’t know they were there.
Mind boggling to me that a non-ping pongy lane keeping is not standard in cars. Is it standard in luxury cars? Seems like an obvious thing to add/upsell.
Non ping-ponging lane following assist is already available in many cars including KIA and Hyundai models. They're very conservative and disengage very easily. I think it's by design to minimise their legal accountability
Not just legal accountability, but actual safety. They are designed so that they do not give the user a false impression of the extent of their capabilities.
I've been incredibly surprised to see that lane assist in my Kia is significantly better than that of most other (legacy non-hi-tech, think nicer hondas and lexus ICE/hybrid) cars I get a chance to drive.
I unfortunately don't have radar cruise control on my Kia, though, which would make highway driving even in traffic completely effortless, and this seems to be standard on themore expensive cars. Maybe it's for the better, though, because it does force me to be much more attentive on the road.
I am addicted to radar cruise + lane assist in my Kia. I use it all the time in traffic.
Hyundai actually has two systems, LKA and LFA. LKA just tries to bounce the car back when it detects lane edges, LFA actively keeps the car in the middle of a lane.
All Hyundai models in Europe have LKA, some (more expensive) also have LFS.
Also Honda. In my Accord 2018 it lane kept but didn’t even play a sound when it lost tracking.
My 2019 Audi S5 was excellent at this. It would ping pong at most once then auto-correct itself to be perfectly centered in the lane.
It did some weird things like if the car in front of you was driving a bit too far to the left/right of a lane, it would copy them. Other than that it was nearly perfect, though. Never had it take an exit by accident, etc.
Their tuning on when to accelerate/brake and make it smooth needed a fair bit of work, but I found that switching the drive mode from Dynamic (Sport) to Comfort changed the eagerness of the system and smoothed things out.
> It did some weird things like if the car in front of you was driving a bit too far to the left/right of a lane, it would copy them
Wouldn't that conceptually be the right thing for the software to do, copy the human in front of it (unless it has demonstrably better information)? OT1H, "lemmings," but OTOH unless the whole line of cars were all on openpilot my life experience has been that the person in front of me by definition has more visibility than I do since their car is not blocking their view as it is mine
I am totally talking out of school, because I'm not in that space and my poor BMW chose to do its own thing[1] so it doesn't work with openpilot[2] -- although they have a dedicated #flexray channel[3] so hope springs eternal
1: https://en.wikipedia.org/wiki/FlexRay
2: https://github.com/commaai/openpilot/issues/44#issuecomment-...
3: https://discord.com/channels/469524606043160576/533838492443...
> Wouldn't that conceptually be the right thing for the software to do, copy the human in front of it (
I see people failing to follow the rules for bad reasons far more often than for any good reason. I don't want my car driving off to the side of the lane just because the car in front isn't centred. It should assume the right thing to do is to follow the rules, and hand off to me in cases that are more complicated.
Ugh. So I’m working on a fork of openpilot and the way the OP model is designed, it has its own rules that is not rooted in any legal driving framework for any state. The simple one is staying right. My state says your vehicle must stay on the right side of the road including roads without markings. OP will try to drive in the middle of the road. Another one is how OP does not distinguish people from parked cars or how oncoming cars are not tracked but simply an object the car should try to avoid (though it does not do this very well and experiences frequent disengagements due to it)
Obviously a model which manage these conditions would fair better but the comma hardware is fairly underpowered for any stronger use case.
I have added dedicated compute to my car to handle a lot of driving rules but now my solution is independent of comma. I tie into the LVDS display on the console so the integration is immersive, but it also means I don’t need comma for the hardware. My fork is also starting to diverge from OP so I may have a competing (but tangential) product!
I also notice this phenomenon in Audi. It’s as if the steering motor is applying inputs after the steering setting has been applied. So if your steering wheel is in sport mode then the motor requires additional force to turn.
I run my own forked copy of openpilot and the car cannot keep up with turns in dynamic mode. When set to comfort it can handle all turns with ease.
> A legacy car maker would be smart to acquire Comma if its for sale
My impression is that the Comma guys were never in this to sell their business
Also, unless I am misunderstanding the situation, since the code is MIT they don't need to acquire Comma to take advantage of the situation. I'd strongly suspect they all want to roll their own implementation for liability reasons, not strictly technical ones
I see that the Dodge Ram is not listed as a compatible vehicle. Could you explain how you managed to make it work?
Edit. I'm dumb. It's listed under "Ram" not "Dodge"
I can’t wrap my head around the fact that 275 car models include all the actuators needed for self driving driving and there’s some kind of port third party software can hook into.
Level 2 driving assistance is commonplace on many new vehicles, often as a standard feature. They are just significantly more conservative in their functionality compared to the level 2 offerings from Tesla, and marketed as safety features rather than "self driving" features.
It's important to note that nothing we're talking about here is actually "self driving" per SAE standards. Openpilot, Tesla's Autopilot/FSD, Honda Sensing, Toyota Safety Sense, Hyundai SmartSense, etc are all level 2 driving assistance features.
This turns level 2 driver assistance features into ... nicer level 2 driver assistance features.
Really need some decimal points in the SAE list. Fully self driving but a human has to watch, and lane keeping are both level 2
"Fully self driving but a human has to watch" is nonsensical. Responsibility for observing the road and making decisions is a very important and inseparable part of driving. Mechanistically manipulating the steering, brakes, and accelerator is not the hard part.
https://www.sae.org/blog/sae-j3016-update
Vehicles with levels 0-2 of driver support are never self-driving at any moment of operation. These systems might be able to manipulate the controls of the vehicle, but they are not good enough to make decisions without constant supervision at all times.
Level 3 is partial self-driving, where the vehicle can assume responsibility for driving under some narrow circumstances.
Full self driving doesn't happen until level 4.
I think the reason that there's no decimal points is because J3016 is really a road-map of the milestones on the way to full automation, not a buyers guide.
In which case the number should be multiplied by the percentage of cases it can be activated in, if you can Level 3 self drive in, eg "up to 37 mph but only on highways" so you multiply 3 by 0.001 because it's only useful in 0.1% of conditions, and then you can add that to the times level 2 works.
That's not possible to calculate because the total denominator of possible "cases" approaches infinity. Level 3 doesn't define what the cases for operation are, that is up to each individual implementor of such a system to define. The criteria is likely to be more much more complicated, like "the output of all 12 sensors when applied to this algorithm, and referenced against this mapping data, delivers a confidence greater than x"
Things don't really get that simple until level 5.
Even at level 4, we're going to have vehicles that pull over when they can't figure out how to navigate successfully.
I’ve been saying this since Tesla released autopilot because I personally think teslas implementation is the best but there isn’t an easy way to compare it to something like radar cruise with lane keeping other than saying they’re both level 2.
Exactly, Toyota, Honda, Hyundai / Kia, etc have systems, they're just nowhere near Open pilot.
I don't know if there's a physical port in most cars, but it uses the CAN bus which has been around since the 1980s.
Also, most cars that have distance assist and lane keeping probably have the required hardware to control speed and steering to some extent.
Nevertheless, it's still impressive that so many cars are supported... and that it can be retrofitted like this at all!
They also lie about models that are supported and won't assist you when you run into that. Had to return one myself. Found no evidence anywhere that my model/year was ever actually supported and anyone was using it either.
There's a very active community on Discord that would have been happy to help you add support for your car assuming it has the right hardware.
https://comma.ai/vehicles has also been improved quite a bit in the past year.
Sounds like they refunded you?
Yeah, that process at least was painless
Could you say what car ?
2023 Forte
https://www.reddit.com/r/Comma_ai/comments/197k04q/2023_kia_...
https://github.com/commaai/openpilot/issues/30936
You can find a few attempts of people trying to get it to work in their discord with no clear positive outcome, discord is unfortunately not search indexed.
> discord is unfortunately not search indexed.
Using discord, etc for any open source project discussions is really unproductive.
Get it that people like the immediacy of chat style communication. But it seems to encourage way more noise and less thoughtful dialogue. The worst aspect is that it locks away a lot of useful information. Trying to dig through chat histories for information is a horrible experience.
> But it seems to encourage way more noise and less thoughtful dialogue.
But compare it to something like Microsoft support forums and you immediately realize why people are using it. It might not be the best implementation of chat, but the immediacy and lack of formality are undeniably valuable.
It looks like support was added in February
https://github.com/commaai/openpilot/pull/30761
Yeah it was on the supported list when I bought it in October of 2023. The website asks your make/model/year when you buy if I recall correctly, that's how it tells you which harness to get.
I found out I needed to update some files in the firmware, followed the guide for that. Asked for help in the Discord. I could never get it working though and returned it after a week. I was going to hang on to it and harass people on Discord, but didn't want to lose track of time and go beyond the return window. Believe me, I really wanted it to work.
Looks like it was added to the supported list before the explicit support I guess
https://github.com/commaai/openpilot/commit/e3275e918354945d...
I suspect Openpilot sends packages to CAN bus to do steering but what makes the physical wheel rotates? I'm confused. Or when a human makes a correction to auto steering, perhaps there could be some kind of full duplex synchronization mechanism
I beleive in the US OBDII ports are required in all vehicles 1996 and later.
I suspect it may be even more than that theoretically. A lot of VAG cars based on the same platform are missing.
Somebody should make an adapter so I can plug a PS2 Controller into it.
I am not surprised that comma is still around.
Minimal VC funding, less than 100 employees, not outrageously increasing headcount each month, profitable and sells a product with good margins.
Not many startups do this anymore, they are just chasing funding every 3 months using OpenAI’s API, comma has their own models before the AI hype.
It’s cool to see that it’s possible to innovate sustainably, in a niche. So refreshing to see something stick around rather than become a bubble with a billion dollar valuation that either takes over the world and enshittifies or implodes and takes the product with it.
Make business chill.
That said: The larger they get, the more regulator attention they’ll attract. If some government entity wanted to, they could probably easily kill them.
kill the company, probably, but the source is MIT and there doesn't appear to be any copyright assignment, so no rug pull
https://github.com/commaai/openpilot/blob/v0.9.7/LICENSE
https://github.com/commaai/panda/blob/32eecd721129b9215030c5...
IIRC the actual model is private and only downloads once you activate the device so presumably if the company was shutdown you couldn’t use the device anymore unless someone shelled out for training
Makes sense. Given what I have seen of geohot's personality, I would bet dollars to donuts that he'd post a link to it in such an outcome
I'm inclined to say the model is open because I've seen PRs that just update the various models
For those similarly curious, I went sniffing around the PRs and found that, yes, there are two .onnx (via git-lfs) in https://github.com/commaai/openpilot/tree/v0.9.7/selfdrive/m... - one for the forward camera and one for watching the driver. The driver one had a lot more features than I expected, but I'd guess they are related to the NTSB certification I saw mentioned elsewhere in the thread here. Regrettably, because I don't have a Panda-compatible car, I actually can't try out the self-hosting setup but it certainly seems plausible
Also, while poking around, I saw they use a "CTF" for on-boarding new contributors, which I thought was neat: https://github.com/commaai/openpilot/blob/v0.9.7/tools/CTF.m...
I chose the Hyundai Ioniq 5 as my current car specifically because it’s compatible with OpenPilot. It’s been a total game-changer for my driving experience. Just like their tagline says, “make driving chill,” and for me, it truly delivers on that promise.
This reminds me of Waymo's approach to self-driving cars. Paraphrasing, but basically they found that progressively adding self-driving to help human drivers is bad, because it leads to the humans becoming complacent and not paying enough attention. Therefore they decided on an all or nothing approach, where their cars would be only and entirely self-driven.
This always seemed like a bit of bull from waymo. It's not an easy problem to work with existing manufacturers to give a better and or cheaper solution... Especially when there are established competitors with efficient verification and validation processes (that every manufacturer requires).
They decided it wasn't worth explaining that their techniques don't generalise to a driver assist. It would not be good or cheap enough to be worth developing the compliance and integration frameworks.
The first thing Waymo tried building (way back when, circa 2010 or so) was highway based driver assist in the style of Autopilot. They did a ton of testing with it and didn’t like how quickly their testers stopped paying attention despite a ton of instruction not to. I’ve seen clips from these tests.
It’s also possible they shifted direction because the long term vision of robotaxis is much more lucrative.
The thing is, you can tune driver assist to whatever form you like; but then you have a true comparison of function with established competitors.
I just don't believe the approaches for high autonomy (especially at the time) actually could make a cost effective assist system.
And for whatever reason they decided to push the we didn't like the driver behaviour message, rather than actually talking about what was actually plausible to achieve in the driver assist space.
Maybe I'm being ignorant about something here but isn't paying less attention the whole point?
Unless you assume that self-driving software is perfect, no, it really isn't. That's the whole problem - the drivers would get complacent, so when there's an issue, they'd be caught by surprise and wouldn't be able to react.
Isn't the point to pay _no_ attention? The difference is when an accident occurs, was the person in the car at fault for not vigilantly watching everything.
You want the curve of total attention to be always above a baseline human in an unassisted car. The car can do some attention and the human can do some. But if the sum of the two falls below the threshold, you’re in trouble.
You can already do that by just closing your eyes and letting Jesus take the wheel. No, the point is doing so while maintaining safety.
It is materially less safe to operate a ADAS while distracted than driving manually. Humans are exceptionally good drivers on average, only encountering minor crashes on timeframes measured in years to decades. As such, if safety critical ADAS errors occur more frequently than every ~100,000 miles and you are attentive in less than 100% of all such occurrences, you are operating your vehicle multiple times more dangerously than the average driver (which is a number that includes drunks and distracted drivers).
That is why it is critical to deliberately downplay the capabilities, to avoid wishful over-reliance, and enforce strict driver awareness (through techniques such as driver monitoring) to avoid operating multi-ton killing machines in ways that are multiple times more dangerous to the occupants, other drivers, and pedestrians. Without that, people are prone to over-generalization of safety capabilities, extrapolating that a single success means robust, continued success thousands to tens of thousands of times in a row.
That statement from Waymo always struck me as deeply uninsightful because it was really just a more complicated way of saying “self driving systems need to be good enough to drive the car on their own” which isn’t just obvious, but tautological.
I'm confused:
"THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY. THIS IS NOT A PRODUCT. YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS. NO WARRANTY EXPRESSED OR IMPLIED."
Where can this be used? In a private parking lot?
The driver takes liability, of course, and this can be used wherever the driver deems it safe and useful.
> this can be used wherever the driver deems it safe and useful
With the disclaimer that this depends on the location of course. For example, I think in Spain (and probably EU wide?) modifications that affect steering and throttle control would need to undergo local homologation before you're legally allowed to drive with that on public roads at all.
Which, to be honest, makes a lot of sense. I don't think anyone would be happy if cars start using software MVPs automatically controlling throttle and steering while in real traffic.
I mean let’s be frank here. The sort of people who are enthusiastic about this also aren’t going to be stopped by the law especially with the low risk of getting caught.
If I were to personally install this and use it anyways, my biggest worry would be the additional punishment if I cause an accident. Not sure exactly what the punishment is for driving a vehicle you know isn't road worthy, but I bet it aint pretty.
While the chances of getting caught are low, I wouldn't want to risk this for insurance. If you are involved in a crash for any reason I don't think they are going to cover it if you modified this stuff on purpose.
Two thoughts.
Suppose I deem it safe and useful for my 6yo to drive for a while, using his Xbox controller from the passenger seat.
It is illegal in many countries for a device (or anything else) to obscure any part of the driver's forward view (area swept by wipers). So even without actually controlling the car, we have an unlawful vehicle.
Aka, this will drive your car into a ditch and it will be your fault.
It really bothers me when people make statements like this. I propose a bet:
If you’re so confident this will drive my car into a ditch, then front the money for me to buy a compatible car & this kit. If it drives my car into a ditch, I’ll pay you back double that money.
That’s an easy bet for you, right?
If a piece of software that has the capacity to kill people is provided without any kind of assurances or warranty with the sole responsibility placed solely on the user, then that already tells me all I need to know, no bet needed to settle it.
Would you buy an angle grinder that's specifically been designed to not have a guard so it can be more useful, made by random contributors on the internet without any official certification? I'll stick with the ones that needed to pass the CE mark and you can sue the company responsible if it chops your arm off thank you very much. They at least have a required level of anxiety needed to patch anything serious knowing the level of responsibility they carry and what's coming if they mess up.
They’re just trying to scare away people who thing they can chuck this on their car and suddenly have a self-driving robot that they don’t have to pay attention to.
You pay a thousand bones for a device you have to babysit. What's confusing about that?
Is it like an app you install on the car or is it a custom integration?
It's a dashcam that you put on the windshield with 2 cameras pointing forward and one inward (filming the driver).
It’s my understanding that in addition to the cameras it also uses the sensors already built in to the car which would include blind-spot detection, no?
Some cars that have BSD it will work with. My car uses it, but don't forget the lane changes are not automated by default. A user must turn on the blinker and nudge the wheel by default. Positive BSD sensors read on CAN-BUS will be read by OP and it will not perform the rest of the lane change. This is how it works on my car (albeit, I don't run default so I just need the blinker).
It says my car is supported and my car doesn't have any blind-spot detection, nor does the requirements list that as needed, so maybe it's optional but not required?
What is filming the driver used for? Can you disable that?
To ensure the driver has eyes on the road all time. It detects if you're not paying attention and beeps at you. I don't think it can be disabled.
Ah, thanks, I just found more detail here - https://github.com/commaai/openpilot/blob/master/docs%2FSAFE...
They say it's required for ISO compliance with Level 2 autonomous driving
There's an element of using these systems that people don't usually discuss. One of the forks, Sunnypilot, enables a mode where you control the pedals but don't have to do the steering. What I've found is you do pay attention in this mode, and since you're controlling the brakes you can easily avoid most issues that you would get with self driving. But, not having to have your hands on the steering wheel makes the experience a lot more pleasant. Also, with traditional lane keeping systems, the so-called "longitudinal" control (accelerator and brakes) is where in my experience the system makes the most mistakes.
I think this mode is something car manufacturers should enable in general. I actually suspect it's significantly safer than completely hands- and feet-free driving modes, and you get most of the benefit of lane-keeping assist.
This is super cool, but:
1. Is this street legal? If so, how?
2. They discuss functional safety, and lots of testing, which is great. But I’d want to see some data on the test results — maybe this exists and I just didn’t see it?
3. It makes me uncomfortable that the anecdotal videos are easily findable, but bulk data/statistics are not. Anecdotes can easily be cherry picked. I get that they’re necessary for marketing, but I don’t feel they’re sufficient given the product’s purpose
This is a modern ADAS system, but a lot more stable. The driver is always liable for what their vehicle does. It's not claiming to be FSD and it's very apparent when the system will need additional inputs from the driver.
I have 15k miles on it. Was able to retrofit a friend's 2015 car as well with a bit of additional hardware, and he likes it. He also has FSD on a model3. But OP or FSD, driver always has to pay attention and add their inputs.
If I was in law enforcement, I’d be rubbing my hands in glee to get ahold of that saved video.
I'd be even more gleeful if I was in car insurance. "Openpilot's not covered, your insurance is invalidated."
Most likely not legal to invalidate in EU. There is laws that’s say that ev everything you can do manually, you are allowed to automate. Any rules against that is null and void.
The “horse winning race” case is a known one where they go into this.
Geico will cover you. You can disclose ahead of time that you have an aftermarket ADAS if you want. If it drives you off the road, it will be as if you drove off the road and you will be declared at-fault, of course.
Insurance is a non-issue with openpilot.
Has that been tested in court? In my jurisdiction?
Many OP users have asked insurance about it and gotten the a-okay. Believe it or not insurance likes safety devices.
Been following Openpilot for awhile, would totally use it if I had a supported car. Though these days most cars come with 'pretty good' ADAS, even 'hands free' in some situations, so I wonder how much it's "worth it" to DIY compared to factory default.
I do think that within the next 5-10 years most cars will be able to hands free highway as reliably as OP. However, cars keep getting more and more expensive. One can buy a 2015-2024 used car for much cheaper and get some very good highway cruising out of it. That's what I did and am very happy about it.
I wonder why the website doesn't say "operating system" but instead calls it an "advanced driver assistance system"
for cars that aren't compatible, can you install additional hardware to make them compatible?
Possibly. You can check out retropilot. Of course depending on the car it may be extremely difficult.
This one <https://github.com/RetroPilot/retropilot> last updated 2 years ago?
for a second, I though this was the ROS alternative I've been forever waiting for smh
Without taking away anything from the substance or achievement of this release, I find phrases like "openpilot is an operating system for robotics." always quite fishy.
No, it's not an OS for robotics. You can't do actual robotics stuff with it, like drive actuators to control limbs or grippers, do motion control or SLAM or perception or any of the usual robotics stack.
Their website correctly says openpilot is an open source advanced driver assistance system that works on 275+ car models of Toyota, Hyundai, Honda, and many other brands. Should've stuck to that.
Thinking about it some more, it's probably just another engagement baiting strategy to get attention and I'm their gullible puppet. Well played.
George Hotz says: "we developed a proper successor to ROS. openpilot has serialization (with capnp) and IPC (with zmq + a custom zero copy msgq). It uses a constellation of processes to coordinate to drive a car."[1] And Comma sells a robot that runs Openpilot: https://comma.ai/shop/body
> You can't do actual robotics stuff with it, like drive actuators to control limbs or grippers, do motion control or SLAM or perception or any of the usual robotics stack.
A lot of the "usual robotics stack" is not going to be relevant for the new wave of consumer robotics that is coming soon. It will be enabled by end-to-end machine learning and stuff like traditional SLAM methods will not be a part of that. The Bitter Lesson[2] is coming for robotics.
[1] https://x.com/__tinygrad__/status/1834792473081815259
[2] For those unfamiliar: http://www.incompleteideas.net/IncIdeas/BitterLesson.html
I enjoy Hotz as a hacker, but I'm really allergic to this kind of oversold language. "[W]e developed a proper successor to ROS" is a past tense statement, as if they've already done this thing. In reality, at best they have presented a roadmap for a thing that could approximate ROS one day.
In the robotics community, the stuff coming out of George Hotz has always been considered a kludgy mess, and unsuitable for serious work. Dude is a talented hacker, but the idea that this will replace ROS is kind of a joke.
To be fair, even ROS I would consider a hobby one.
The point of the bitter lesson is "leverage compute as best you can" not "use DNNs everywhere just because". Oftentimes your available compute is still a crappy ARM machine with no real parallel compute where the best DNN you can run is still not large nor fast enough to even be viable, much less a good fit.
And well some classical algorithms like A* are mathematically optimal. You literally cannot train a more efficient DNN if your problem needs grid search. It will just waste more compute for the same result.
Besides, the nav stack is not really the point of ROS. It's the standardization. Standard IPC, types, messages, package building, deployment, etc. Interoperability where you can grab literally any sensor or actuator known to man and a driver will already exist and output/require the data in the exact format you need/have, standard visualizers and controllers to plug into the mix and debug. This is something we'll need as long as new hardware keeps getting built even if the rest of your process is end to end. It doesn't have to be the best, it just needs to work and it needs to be widely used for the concept to make sense.
The future of consumer robotics will not be built on "a crappy ARM machine with no real parallel compute". Traditional robotics has failed to produce machines that can operate in the real world outside of strictly controlled environments, and more of the same isn't going to change that. Fast hardware for running DNNs will be a hard requirement for useful general purpose robots.
I agree that it'll be needed, but that hardware that can provide enough compute at acceptable wattage is yet to materialize. Only once that changes the equation will change. Today you'd be surprised how many production UGVs run off an actual Pi 4 or something in a comparable compute ballpark.
With due respect, this has to be one of the most ignorant takes on robotics I have read in a while. Yes, you can always slap serialization and ZMQ on your framework. That doesn't make it an OS.
And no, the usual robotics stack is not going away anytime soon. Maybe develop some actual useful robots before posting like an expert on robotics topics.
I believe the idea is that openpilot replaces the usual robotics stack with an end to end neural net.
While I agree operating system is usually a marketing term, it does feel correct in this case as it is the operating system for the Comma Three, which can operate cars but also this thing: https://www.comma.ai/shop/body
I definitely thought it was a ROS clone based on that first line.
ROS doesn't need a clone, it needs a successor.
Took the bait as well.
ROS2? I'll see myself out...
Could someone explain the joke? I've been dabbling with learning robotics and I've been confused by how ROS and ROS2 both appear to be actively developed/used. Is ROS2 a slow-moving successor version (like Python 3 was) or a complete fork?
Slow-moving successor, which the community isn't exactly going wild over. It offers modest improvements in exchange for a painful upgrade process, and many of the original issues with ROS1 remaining unsolved.
The other half of the joke is that ROS was never an operating system either.
Well there is one thing that ROS 2 does better, you can declare params directly inside nodes and reconfigure them all without building extra config files. And it doesn't stop working if your local IP changes.
But the rest are firmly downgrades all around. It's slower (rclpy is catastrophically bad), more demanding (CPU usage is through the roof to do DDS packet conversions), less reliable (the RMWs are a mess), less compatible (armhf is kill). The QoS might count as an improvement for edge cases where you need UDP for point clouds, but what it mostly does on a day to day basis is create a shit ton of failure cases where there's QoS incompatibility between topics and things just refuse to connect. It's lot more hassle for no real gain.
Config generally feels more complex though, since there isn't a central parameter server anymore. The colcon build system also just feels more complex now, which I thought was already impressively complex with catkin.
Yep it takes super long to get parameters from all nodes cause you need to query each one instead of the DDS caching it or something.
And yeah I forgot, there's the added annoying bit where you can't build custom messages/services with python packages, only ament_cmake can do it so you often need metapackages for no practical reason. And the whole deal with the default build mode being "copy all" so you need to rebuild every single time if you don't symlink, and even that often doesn't work. The defaults are all around impressively terrible, adding extra pitfalls in places where there were none in ROS 1.
It does a lot of things better but it also does a lot of things worse and also doesn't fix a lot of the real problems with ROS as a system.
ROS2 has been pushed a the successor to ROS for like a decade, and people still prefer ROS for various reasons. So yeah like Python 2/3 kinda.
No it’s much worse, python3 was all round better, it just took a while to get all your dependencies ported which made the transition hard. Judging by the comments it doesn’t seem like people agree that ROS2 is even all round better from ROS.
It's funny this topic came up today because I have a group of students working on a ROS2 project and at our meeting this afternoon they had a laundry list of problems they've been having related to ROS2. I'm thinking our best option is to use ROS1...
You're right ROS2 isn't all round better than ROS so the transition will never happen fully.
FWIW I'm working on an actual replacement for ROS, I'll post it to ShowHN one day soonish :P
Isn’t the software for training end-to-end NN to be used in automation? Just a first version that it’s used for cars, and they have been using it for their own robot.
So the claim still stands?
The docs (https://docs.comma.ai/) begin with a more honest - and useful - description:
openpilot is an open source driver assistance system.
Yeah came to say the same, I thought a new big player is in the market. It looks great nonetheless.
This seems to be a mix of C++ and Python, including a script called "realtime.py" (oxymoron?). So am I now exposed to other people using Python on the roads to operate heavy machinery?
Well NIST says don't use C++ either: https://www.nist.gov/itl/ssd/software-quality-group/safer-la...
So what, you want everything written in RUST on a linux kernel with hard real-time patches? It uses machine vision anyway, which has no hard guarantees at all. The software it uses to detect lanes or cars is probabilistic by it's very nature.
Python does pretty good at soft real time if you manage your own event loop and disable the garbage collector, and you're a lot less likely to get "crash the entire stack" style memory allocation bugs. Sure, GO or RUST would be better, I think CPP could be worse if handled inexpertly.
exception handling in GO makes it unusable IMO
Python has segfault issues, surprising exceptions and version incompatibility.
I've been using Linux/BSD for over a decade now. No C or C++ application has ever crashed, I cannot say the same about Python applications. Outright segfaults are rare but happen. Rogue exceptions are much more common and could basically have the same detrimental effect on a self-driving system as a segfault. And let's not talk about logic bugs due to version incompatibility and the obsessive rewriting of those who took control over CPython.
Ahh, you've been running some grad students first python project as if it was a serious project like curl with 20 years of history, and expecting it to have the same quality. But you've somehow avoided the tons of grad-student CPP programs with similar quality issues, or the broken code pushed by companies like crowdstrike or IBM.
Fair enough, your experience may vary. I'd suggest not judging the language by the standards of some hobbyist code that just so happened to end up on github. I've had tons of bugs in c/cpp programs over the years, some more critical than others.
I've seen a lot of shitty and unreliable python code, and a lot of good and mature C/CPP projects. I've also seen really bad security issues and crashes with bad C code, heartbleed, crowdstrike, etc.
For what it's worth I've never had youtube-dl hard crash on me, and I could argue that it's a more complicated problem to solve than what curl is solving. In an apples-to-apples comparison I think it does pretty well.
No matter what language you use for this you're going to be relying on an AI vision model with no hard guarantees.
Actually Python was insufficient for the sort of grad student bugs I wanted to write, I was able to just wrap everything up in giant try blocks and then,
(Where I might be an index. Or an element).Fortran is able to generate better bugs, because it has allocate/free.
You have much more control over a pure C/C++ application because it does not involve the Python runtime. Crowdstrike etc. are exploits that don't really matter here: If you are on the CAN bus it's game over already.
That said, I'm pretty sure CPython has exploits, too. They'll be harder to find and trigger though.
Sure, runtimes exist and have engineering trade-offs. You avoid a whole class of memory related bugs but you lose a lot of control over memory allocation. You can do soft real-time as long as you manually manage the garbage collection and accept that there will be some (bounded) jitter on memory allocations.
The first rule of the tautology club is the first rule of the tautology club. Things have trade-offs. Python removes (or at least significantly reduces) a whole class of bugs that appear when using lower-level languages, that's part of why it's a good glue language.
The interface between the openpilot and the car is a standalone device (the panda) that provides and enforces the safety model. All code is written in C to automotive safety standards including ISO26262, ISO11270, ISO15622, and MISRA-C. 100% line coverage for all safety unt tests.
They also run pretty extensive tests (regression, unit, hardware/software-in-the-loop, mutation, and vehicle specific) on every commit and have actual hardware devices continually running real routes looking for regressions.
https://github.com/commaai/openpilot?tab=readme-ov-file#safe...
https://github.com/commaai/panda?tab=readme-ov-file#code-rig...
Just so we're all clear here, there is a lot of gobbly-gook in this answer which is either off target or irrelevant.
> a standalone device (the panda) that provides and enforces the safety model
What the actual safety model is that is being enforced is far more important here. The safety model could be "there is no safety guarantee whatsoever" and this sentence would still be true.
> All code is written in C to automotive safety standards including ISO26262, ISO11270, ISO15622, and MISRA-C.
26262 says practically nothing about software, what you really want is 21448. And 11270 and 15622 are super low targets for the amount of control authority available here.
MISRA-C is mostly a waste of time when it comes to safety. It gives software developers the warm blanket of having a checklist they can tick items off of, but does little to prevent unsafe systems from being built. Programmers have gotten pretty good about at least using tests and other analysis tools to make sure they're not doing the wildly stupid things that MISRA tries to prevent.
> 100% line coverage for all safety unt tests
100% like coverage is also rather trivial to achieve and doesn't say much. Branch coverage would be better, but being able to make some claims about state space coverage with exposure numbers would be what I'm expecting here.
> So am I now exposed to other people using Python on the roads to operate heavy machinery?
yes in the sense that python is running the ML models and deciding what the vehicle should do, but it is heavily bounded in what it can do by the safety model which is implemented in bare-metal MISRA C running on the microcontroller that interfaces between openpilot and the CAN bus (panda). It enforces things like accel/braking limits and steering rate limits along with consistency checks, heartbeats, vehicle status checks, etc.
Level 2 self driving is already only a best effort system so if python caused an issue it would just fall back to the safety model on the panda and ultimately the driver to operate the vehicle safely.
Sort of. It operates after your vehicle safety systems, so yes, python is controlling the car to an extent, but only within the limits of the built in safety systems.
Is this legal in the US?
I’m surprised to see so many people disliking Open Pilot on HackerNews. I have one of these, and it’s a total game changer on long trips. I drove from Texas to California using my Comma 3 and I didn’t have to overtake it a single time on the interstate.
Sure you have to actively be alert your entire drive, but it’s still significantly better than actually doing the work of driving.
I've only driven a driver-assist system once and kind of hated it, but I've always assumed that the mental work of remaining attentive as a passive participant would actually be more mentally taxing than just driving. It seems like you're inevitably going to lose attention.
[dead]
[dead]
I always wondered who would pay $1000 for a device you have to babysit, just so that you can become an unpaid beta tester and data drone for SV millionaires. I guess all you have to do is tell people they're a part of a revolution?
Or want your car to drive itself. I only drive long distance and my Comma has completely changed how often I visit family and improved my safety coming home from exhausting weekends.