Reviews: Tesla Cars Have A Memory Problem That May Cost You A Lot To Repair - - PressFrom - US
  •   
  •   
  •   

Reviews Tesla Cars Have A Memory Problem That May Cost You A Lot To Repair

02:11  18 october  2019
02:11  18 october  2019 Source:   insideevs.com

Barn Find Hunter Digs Up World-Class Collection Of Muscle Cars

Barn Find Hunter Digs Up World-Class Collection Of Muscle Cars This just goes to show what kind of cars some people have sitting around. As car lovers, we've probably all driven past a field of fenced-in cars and thought about stopping to talk to the owner. Hagerty's latest episode of The Barn Find Hunter did just that, and in doing so, stumbled upon an incredible collection of cars classic cars, trucks and even memorabilia. While making a 700-mile drive from Midland, Texas, to McPherson, Kansas, a brief stop in Lubbock, Texas turned into an afternoon of classic car spotting.

Anyway, his “ repair method costs a lot less for the consumer.” And he remembers another important aspect of how Tesla is currently dealing with this. Fortunately, they have a larger flash memory size, which should mitigate the issue for the time being. Tesla will still have to eliminate or curb this logging

Tesla Cars Have A Memory Problem That May Cost You Much To Repair .

It is currently appearing on MCUv1, but the MCUv2 will also have issues.

When Elon Musk said Tesla cars are computers on wheels, he forgot to mention they run on Linux. They also do a lot of logging. According to Jason Hughes, from 057 Technology, more than they should:

New York Hiking Trails Double As Classic Auto Graveyards

New York Hiking Trails Double As Classic Auto Graveyards These rusty relics are a welcoming sight for trail-blazing locals. © Motorious New York Hiking Trails Double As Classic Auto Graveyards Multiple vintage abandoned cars from the 1950s and ‘60s sit along the hiking trails at a few different parks in Westchester County, New York. One would think that cars littered along a nature park path would be an eyesore, but locals love the old cars and use them as landmarks when hitting the trails. Overrun with foliage and ridden with bullet holes, these classic autos have been sitting for decades and have become part of the landscape.

Tesla Cars Have A Memory Problem That May Cost You Much To Repair .

Tesla Cars Have A Memory Problem That May Cost You Much To Repair .

“The information logged here is pretty much useless on production vehicles. Unless a developer has a specific reason for enabling it, it does the customer no good. These logs are also rarely downloaded by Tesla.”

Why are we telling this? Because you may end up paying more than $1,800 to repair your Tesla because of a cheap eMMC flash memory card.

The reports about the problem come from three different shops in three very different places. 057 Technology is from Hickory, North Carolina. Robert Cotran and his partner Jean-Claude Thibault work on the issue at the Cotran Consulting in Candiac, Québec. He kindly provided us the MCU pictures for this article. Pete Gruber, from Gruber Motors, deserved a whole series of videos from Out of Spec Motoring that we talked about here. He is in Phoenix, Arizona.

Jaguar woos Tesla owners with $3,000 I-Pace EV discount

Jaguar woos Tesla owners with $3,000 I-Pace EV discount Jaguar woos Tesla owners with $3,000 I-Pace EV discount originally appeared on Autoblog on Wed, 14 Aug 2019 Read More

Tesla Cars Have A Memory Problem That May Cost You Much To Repair .

Tesla Cars Have A Memory Problem That May Cost You Much To Repair .

All three helped us explain the failure. More than that, they aim to warn Tesla owners that the clock is ticking for all of them. Regardless of your car, the logging will require replacing your MCU sooner or later.

Hughes told InsideEVs:

Porsche Taycan Debut: First Impressions From the 2019 Frankfurt Auto Show Floor

Porsche Taycan Debut: First Impressions From the 2019 Frankfurt Auto Show Floor Porsche Taycan Debut: First Impressions From the 2019 Frankfurt Auto Show Floor

Tesla Cars Have A Memory Problem That May Cost You Much To Repair .

Tesla Cars Have A Memory Problem That May Cost You Much To Repair .

“The main issue is that this excessive log file writing causes eMMC flash wear. Flash memory is generally only rated for some tens of thousands of write cycles. What happens is that the flash memory starts to fail when writings can no longer be completed. When one block fails, parts of the firmware may also become unreadable, leading to poor operation or failure of the MCU completely.”

Ask Cotran what the problem is: You’ll get the same answer.

“The filesystem in MCUv1 is handled on a NAND-based eMMC flash chip. Although these are solid-state and great for automotive use, there is one pretty serious drawback. Each memory bit on a flash chip can only be written to a limited number of times before data gets corrupt – and that bit can no longer reliably store a 0 or a 1.”

Mercedes EQS Concept: How the S-Class of Electric Cars Will Battle Tesla

Mercedes EQS Concept: How the S-Class of Electric Cars Will Battle Tesla Frankfurt Auto Show 2019: Mercedes EQS Concept first impressions

Some early Tesla models are spontaneously malfunctioning and could cost owners more than ,800 to repair . The issue stems from a worn-out flash memory chip embedded in the Multi-Media Controller board that UPDATE: Tesla Cars Have A Memory Problem That May Cost You A Lot To Repair .

UPDATE: Tesla Cars Have A Memory Problem That May Cost You A Lot To Repair . If the car showed 100 percent of charge and still had a lot of regenerative braking available, that would mean the 4.9 kWh turned into a hidden buffer closer to full charge.

a circuit board: Tesla Cars Have A Memory Problem That May Cost You Much To Repair © InsideEVs Tesla Cars Have A Memory Problem That May Cost You Much To Repair

Gruber is even more direct with the diagnostics:

“Tesla selected a flash chip that is unable to handle the constant read/write functions.  These chips have since been replaced with a more robust version.”

If you still did not understand what happens, every Tesla has an MCU, or main control unit. Version 1, also called MCUv1, equipped Tesla Model S and Model X units up to 2018. When it fails, the car simply stops, as these threads on TeslaMotorsClub.com and on Tesla's official forum show.

Among its many components was an nVidia Tegra ARM-based CPU. Tesla soldered the 8 GB eMMC flash memory chip to the same board of the CPU.

When the cars started to be sold, at the beginning of the 2010s, logging was not an issue. “However, since the initial release, Tesla's firmware image size has gone from about 300MB to the full 1GB maximum size,” said Hughes.

a circuit board: Tesla Cars Have A Memory Problem That May Cost You Much To Repair © InsideEVs Tesla Cars Have A Memory Problem That May Cost You Much To Repair

In other words, the firmware is now competing with logs for space on the eMMC chip. When the log writing wears a sector of the chip, it uses a mechanism called wear leveling. Cotran explains the process:

New cars and trucks: Vehicle ownership costs reach new high, AAA finds

New cars and trucks: Vehicle ownership costs reach new high, AAA finds The average cost of owning a new vehicle reached an all-time high this year as financing costs jumped, according to a new AAA study.

Tesla ’s website no longer includes a cost calculator that breaks down the expenses — such as Fueling a gas-powered car that gets 30 miles to the gallon would cost you significantly more at Installing a home charger can cost a lot . Tesla previously provided free charging at its nationwide

The maintenance and repair categories are other areas where Tesla takes the cake, as it comes with a vehicle maintenance package and everything else is This is the data I used to come up with the true cost of ownership for the Tesla Model S. While it’s not perfect, and costs may vary, overall the Tesla

“The eMMC flash chip architecture attempts to mitigate this problem using a wear-leveling technique. It spreads out write operations over the entire chip to ensure that specific bits are not written to very often, essentially avoiding the write limitation.”

Check what Hughes has to say about this:

“The flash controller transparently and seamlessly spreads the wear across the chip utilizing unused sections of the flash memory to extend the effective number of write cycles available. With Tesla utilizing near 100 percent of the flash memory today, there is no free space left for additional wear leveling to compensate for the excessive log writing.”

a circuit board: Tesla Cars Have A Memory Problem That May Cost You Much To Repair © InsideEVs Tesla Cars Have A Memory Problem That May Cost You Much To Repair

Simply put, there comes a time in which the eMMC flash memory fails. “If data is changed on the chip too often and in large quantities, wear-leveling can only do so much and at one point data starts to get corrupt. You can either lose data or core functionality can start to fail depending on where the corruption occurs,” Cotran told InsideEVs.

When that takes place, Tesla just replaces the whole MCU. If your car is still under its warranty period, that is for free. If coverage is no longer valid, you will have to pay the bill.

Hughes told InsideEVs:

Pre-Owned Kia Stingers Are Already An Awesome Value

  Pre-Owned Kia Stingers Are Already An Awesome Value It’s no secret that we like the Kia Stinger . It’s a fun, rear- or all-wheel drive four-door fastback with driving dynamics on par with a lot of German metal. And while new ones can be had with significant deals if you know where to look, the pre-owned 2018 models may be an even better value.© Photo: Andrew CollinsRelated Story: Reminder: You Can Still Get Awesome Deals On A Kia StingerThe Kia Stinger starts new around $33,000 for a turbocharged four-cylinder model with RWD and tops out around $53,000 for a GT2 AWD with a twin-turbo V6.

“ Tesla ’s got a problem . They create so many logs in the car , they write to [the chip] so fast that it basically burns them out. They have a finite amount of As a result, there are very few independent Tesla repair professionals, acquiring parts to repair Teslas is very difficult, and finding or making

UPDATE: Tesla Cars Have A Memory Problem That May Cost You A Lot To Repair . It is currently appearing on MCUv1, but the MCUv2 may also have issues.

“At $1800 for a replacement, getting this fixed at a Tesla service center out of warranty isn't cheap. Given the nature of the failure, it usually does take years to happen... Although likely less on cars closer to the end of production of MCUv1 – around Q2'17 up to  Q1'18. They would have started life with less wear-leveling ability, to begin with, due to 100 percent flash usage.”

a circuit board: Tesla Cars Have A Memory Problem That May Cost You Much To Repair © InsideEVs Tesla Cars Have A Memory Problem That May Cost You Much To Repair

That would be the price for a replacement in the US. We have asked Tesla if that is correct, but the company did not get back until we published the article. If it does, we will update it. “In Canada, that can cost up to C$4000 for parts and labor. It sometimes requires a wait if the service center is busy or doesn't have spare MCU units to swap with,” Cotran told InsideEVs. That corresponds to a little more than $3,000.

Hughes asks a fraction of that to repair the MCUs that present the issue. More precisely, 13.3 percent. “I currently charge $399 for the repair service, but I need the MCU or vehicle at my shop, 057 Technology, to do so. The turnaround can take some time as we've been super busy lately with this and other projects.”

a circuit board: Tesla Cars Have A Memory Problem That May Cost You Much To Repair © InsideEVs Tesla Cars Have A Memory Problem That May Cost You Much To Repair

Cotran says his price depends on many variables. Anyway, his “repair method costs a lot less for the consumer.” And he remembers another important aspect of how Tesla is currently dealing with this. “They are replacing many units at service centers, and unfortunately, the computer goes to waste when only a replacement chip is necessary.”

Buying a car: How long should you keep your clunker?

  Buying a car: How long should you keep your clunker? My philosophy has always been to pay cash for a lightly used car and then drive it until the doors fall off. But which repair should be the last?In their respective driveways, my neighbors' heads immediately turned toward the chaos to see what was wrong. I took a deep breath, and slowly backed my 15-year-old car, with 200,000 miles on it, out of my garage, fresh off its primal scream.

Posting to the Tesla Forums is now limited to verified Owners. To request verification, contact [email protected] tesla .com from the email associated to your Tesla account.

He tried to prevent the automaker from turning its MCUs into electronic trash. To no avail. “We approached Tesla to offer our services, but they were not interested and claimed that they do not outsource work.”

Gruber charges $165/hour. “That’s for component-level diagnosis, troubleshooting, and repairs. We have sophisticated flash memory chip removal equipment, and ball soldering equipment, and commonly reverse engineer and perform component level repairs.”

Tesla Cars Have A Memory Problem That May Cost You Much To Repair © InsideEVs Tesla Cars Have A Memory Problem That May Cost You Much To Repair

All three of them replace the flash memory chip. “We remove the MCU from the car and dismantle it completely. Then we are able to extract unique identifying authentication keys from the eMMC even though part of it is corrupt.  These keys are necessary for the car to authenticate against the Tesla network and give the user access to firmware updates and the Tesla app,” Cotran told InsideEVs. He and Thibault replace the original Hynix eMMC chip on the board for the Swissbit you can see standing alone in the image right above and in the ones below.

Hughes gives us an idea of how complicated the repair is. “Since this flash chip is a hard-connected unit, there's just no simple way to replace them. It involves specialized tools and equipment. In my case, I pre-repair several units to have on hand for customers. That means I mainly have to try to recover the car-specific information from their unit to copy to a replacement and match the firmware version. It's tedious but doable. In general, it can take several hours of work even with the ready-to-go replacements on hand.”

a sign on the side of a building: Tesla Cars Have A Memory Problem That May Cost You Much To Repair © InsideEVs Tesla Cars Have A Memory Problem That May Cost You Much To Repair

Gruber follows a slightly different procedure. “We are working on installing flash memory chip sockets since the chips fail over time, instead of replacing an entire MCU for a failed flash memory chip.”

Gruber replaces it with a more robust chip, with a larger capacity. All of them do that, in fact. “Once that is done, we de-solder the defective eMMC chip from the processing board. We then have some proprietary scripts which create the filesystem layout necessary for the proper operation of the MCU computer on a brand new eMMC chip. We use industrial-grade chips, and we double the memory capacity from 8GB to 16GB in order to give it more space to perform its wear-leveling operation,” said Cotran.

Gruber confesses he still needs to sort one aspect of the issue: software. “We are still unsuccessful reprogramming the chips once we replace with more robust technology. Our MCU repairs are confined to power supply issues, non-flash chip component failures, wiring failures."

Perhaps Cotran and Hughes can help him with that. There is a lot of work for all of them related to this eMMC failure. “We have helped dozens of customers with this problem,” Gruber told InsideEVs. Cotran says he has seen a dozen of clients with the issue since he started repairing it. “Thibault has come on-board for the Tesla repair side of things.” Hughes has fixed more than a dozen Teslas with MCU failure just in September.

Does replacing the failed hardware solve the problem? Only temporarily. “This ensures that the eMMC chip will last much longer than the original. But we also plant scripts on the computer which write logs to a RAM drive instead of to the eMMC chip,” Cotran told InsideEVs.

Using the RAM was also something Hughes decided to do. “Tesla can do like I do on my cars and move the logging to RAM to trick the system, but this penalizes performance a little since RAM is limited.”

Cotran mentioned other pros and cons: “This has one big benefit and one big drawback. The benefit is that the logs are no longer being written to the eMMC and instead to RAM. It does not suffer from any excessive write problems. The drawback is that logs are now in volatile memory. That means that, if the computer gets powered off or rebooted, the logs are lost. We feel that the trade-off is worth it because nobody wants this to happen again.”

a circuit board: Tesla Cars Have A Memory Problem That May Cost You Much To Repair © InsideEVs Tesla Cars Have A Memory Problem That May Cost You Much To Repair

There is no escape regarding the root cause of the issue: excessive logging. And the only one able to address that is the manufacturer.

“Tesla needs to just disable syslog on all vehicles unless specifically required on a development car or to diagnose an infotainment issue on a specific car. There are absolutely zero reasons to log hundreds of MB per day to a small built-in flash chip,” Hughes told InsideEVs.

Cotran agrees. “In my opinion, this isn't necessarily a manufacturing defect. The various Tesla software teams may not have considered that other teams were logging as much when they were coding their logging functions. However, there was definitely some kind of oversight in general that was missed in the engineering of these units. I do think Tesla should find a way to mitigate this going forward – as we have – to prevent the unnecessary replacement of these computers.”

a red car parked on the side of a mountain: 2. Tesla Model S © InsideEVs 2. Tesla Model S

We have asked the company what it will do to prevent the problem but got no response so far. We did not have to ask if it was aware of it. “Tesla has known about this issue for years now and has done nothing to mitigate it. I've personally reported it on multiple occasions, and I know others have as well, said Hughes. “I've noted this to Tesla on several occasions, starting in late 2015, and several times since.”

Tesla needs to act. Not only because of the older units with MCUv1 but also because its newer cars are at higher risk of MCU problems than the older ones. “Instead of mitigating the issue, it writes even more data to the logs today than ever before. Combined with the max-size firmware images, general caching – map tiles, Autopilot info, music, etc. – this makes every MCUv1 have a high probability of failure,” Hughes told InsideEVs.

a car parked in a parking lot: Tesla Model X © InsideEVs Tesla Model X

Despite having a 32GB eMMC flash chip, MCUv2 deals with much larger software. “There is a lot more room for wear-leveling. Keep in mind, though, that MCUv2 does have some advanced functions – like 3D gaming and Youtube, Netflix, etc. – which does take more room on the filesystem,” Cotran said.

Hughes thinks likewise. “MCUv2 and Model 3 also have an issue with excessive logging. Fortunately, they have a larger flash memory size, which should mitigate the issue for the time being. Tesla will still have to eliminate or curb this logging significantly on these if they want them to last, though.”

For Cotran, having to deal with the MCUv2 is not a matter of "if" but "when". “I haven't done a lot of work on MCUv2 yet simply because the fleet is mainly MCUv1. Those are the units that need repair now. We do have an MCUv2 here where we have filesystem-level access and will be looking around in there at one point soon.”

a car parked in a parking lot: Tesla Model 3 © InsideEVs Tesla Model 3

The sad part of the story is that Tesla is aware of the issue since 2015. And it has apparently done nothing so far to correct it, hence the multiple cars being currently fixed. Why? Hughes has a theory. “The cynic in me looks at this as a planned obsolescence type of thing... However, the reality is probably a lot more benign: laziness.” A successful Occam’s Razor example? Only Tesla can tell.

If you have one of their EVs, knowing about the issue will help you try to avoid it. If your car was made before 2018 and is still under warranty, try to verify if your MCU is working properly. You may still get a replacement for free.

a car parked in a parking lot: Here's The Estimated Range Of Tesla Model S, 3 & X At Highway Speeds © InsideEVs Here's The Estimated Range Of Tesla Model S, 3 & X At Highway Speeds

If you do not have coverage anymore, talk to Cotran, Hughes, or Gruber. They may develop a prophylactic solution to your MCU prior to having to replace it. Such as writing the logs to the RAM before the eMMC card fails.

For all other Tesla clients, ask the company to get this solved. Perhaps a single update can avoid future problems. Burying your head on the sand and calling this article FUD will only help until “the bell tolls for thee.” According to Hughes, Cotran, and Gruber, it will toll for every current or past Tesla MCU. It is just a matter of time.

Buying a car: How long should you keep your clunker? .
My philosophy has always been to pay cash for a lightly used car and then drive it until the doors fall off. But which repair should be the last?In their respective driveways, my neighbors' heads immediately turned toward the chaos to see what was wrong. I took a deep breath, and slowly backed my 15-year-old car, with 200,000 miles on it, out of my garage, fresh off its primal scream.

—   Share news in the SOC. Networks

Topical videos:

usr: 34
This is interesting!