Of ECU's and Electronics

Tuesday, July 2, 2019 - 11:45

Welcome to DiscoTd5.com, a site dedicated to the Land Rover Td5 Engine. The DiscoTd5.com website springs from the interests and experiences of my ownership of a Td5 Discovery which I purchased in May 2011. Since I first set up the site, my interests have become heavily focused on reverse engineering the Td5 ECU firmware, and I'd expect this focus to remain for the foreseeable future.

To thank site donors for their ongoing support, I've decided to give donor account holders "early access" privileges for new tuning and map related content. The content will eventually appear on the site, but there will be delay of around three months. You'll need to be logged in with a donor account to see this "early access" material.

Donated and didn't get an account?

It appears that a couple of webmail services - hotmail.com in particular - are quietly filtering emails from the discotd5.com domain to spam.

Donors will always get a "Thank you for your donation email" from the site within a few minutes of the donation being made, and I try to get donor accounts setup same day, but it can take up to 2-3 days if I'm away, and have limited internet access.

When the account is created an email advising the account has been setup is sent. This contains a one-time login link.

If you have donated and didn't get a "Thank you email", first thing is to check your spam folder.
If 3+ days have passed, and you haven't got an account creation email, first thing to do is check your spam folder.

In particular if you have used a Hotmail account to donate - just check your spam folder for email from @discotd5.com.

Datalogger Project - XDL logging milestone.

Saturday, August 3, 2019 - 16:45

The past week has seen a lot of progress on the Datalogger.

The event-driven non-blocking FAT32 library I posted about earlier in the year was integrated into the logging module and after some initial hiccups is now working well.

CSV logging to card is now running well. Auto detect of MSB // NNN hardware has been implemented and throttle mapping switched on that basis. MSB ECU's report two throttle tracks, throttle %, and supply voltage, while NNN ECU's report three throttle tracks.
A diagnostic request that returns the raw ADC conversion values, and fault codes are also dependent on ECU hardware type.

EGT probe support has been integrated into the CSV logging. The logger checks if to see if there is a probe connected at the start of each log run, and adds the EGT column if required.

The big news - for me anyway - is that logging to Tuner Pro XDL format is working. The code for the XDL log format was written yesterday - based on the reverse engineering done for the Nanocom conversion as posted previously - , and a small bug in a log buffering code which prevented the logs writing to card was fixed this morning.

I've only had a chance to do a few short runs around home - lots of 40kph zones and slow moving traffic. Aside from my laptop going to sleep after 15 minutes and killing the longest of the logs, the logging seems to be pretty stable.

This evening I looked up my test ECU and eft the logger run on the bench recording to XDL for a bit over three hours without any issues.

As a bit of a teaser, this is a screenshot of Tuner Pro RT tracing from a log I recorded today.
The black spots on the tables indicate the current running point.

XDL Datatracing

None of the parameters being used are calculated - it's all logged ECU data.

The to-do list is still pretty long, but I'm now feeling far more optimistic than I was 3 months ago.

Desktop diagnostics...

Tuesday, January 24, 2017 - 08:00

Garmen has posted up a Python script which at the moment allows the user to read out the "Nanocom Fuelling" parameters. github

The project uses a cheap serial programming interface based on the CP2102 USB to TTL converter plus some external circuitry to interface with comms levels. The trick is that the CP2012 needs to be reconfigured to suit the "non-standard" baud rates, and to allow sending a 0x00 byte with a period of exactly 25ms. It's a fairly neat trick.

Anyway Garmen's work prompted me to pull out a "dumb" OBD-II interface that I'd picked up a couple of years ago and thrown in the cupboard. Most OBD-II interfaces are based on the ELM-327 chip which basically does all the grunt work of managing the diagnostic connection and allows users to simply send OBD commands over a serial connection and read back the response.

A "dumb" interface is a USB to serial converter chip (usually a variant or clone of the FTDI FT232) and hardware to interface with the diagnostic port line levels. These are usually sold as something like "VAG-COM KKL USB Interface" - just google it... These require that the user application to handle all of the communications protocol details.

The interface I have has a cloned FT232BL. The complete lack of markings on the chip is the give away. But as the part cost $14.00AUS including postage it's hard to feel ripped off...

The nice thing about the FTDI chips is that they have a "Bit Bang" mode. This allows you to write a byte to the chip and it will hold that state until you write another byte. It's perfect for doing things like sending the 25ms wake-up message or bit-banging the 5 Baud initialisation sequence.

There are a couple of Python modules that allow access to these FTDI specific features. I'm using pyftdi.

As a rough example of how you do it...

#!/usr/bin/env python3
 
from pyftdi.ftdi import Ftdi
import time
 
HIGH = bytearray([0x01])
LOW = bytearray([0x00])
message = bytearray([0x81,0x13,0xF7,0x81])
 
 
uart = Ftdi() 
 
uart.open(0x403, 0x6001)
uart.set_baudrate(10400)
uart.set_line_property(8, 1, 'N')
 
# Enable Bitbang Mode
uart.set_bitmode(0x01, 0x01)
 
# Set TX line high
uart.write_data(HIGH)
 
 
# Short sleeps in the loop seem to improve accuracy...
# Using time.sleep(.025) for the full delay is very jittery.
# time.perf_counter() can be used instead time.monotonic()
# requires Python 3.3 or later
 
start = time.monotonic()
while (time.monotonic() <= start + 0.35):
    time.sleep(0.01)
 
# Set TX line low
uart.write_data(LOW)
 
start = time.monotonic()
while (time.monotonic() <= start + 0.0245):
   time.sleep(0.00025)
 
# Set TX line high
uart.write_data(HIGH)
 
start = time.monotonic()
while (time.monotonic() <= start + 0.0245):
     time.sleep(0.00025)
 
# Disable Bitbang Mode
uart.set_bitmode(0x00, 0x00)
 
# Send serial message
uart.write_data(message)

Which should give you something like:

Wakeup Sequence

The ISO14230 spec states the total time for the wakeup time (Twup) should be 50ms +/- 1ms. This code gives Twup of 50.67ms, so is within spec.
As reference the Nanocom sets the line low for 26.54ms and then idles high for 118.9ms giving a Twup = 145.44ms. The Td5 ECU is pretty forgiving.

Nanocom One log converter

Sunday, May 14, 2017 - 08:00

Updated 2019-7-8. Added link to Python 3 version of scripts on GitHub

Updated: I decided to revisit the Nanocom One log converter code today with view to incorporating it into a larger project. When I made a couple of changes to tidy things up, I discovered that I'd managed to wallpaper over a small problem in the Nanocom One log format, and also missed the bleeding obvious in regard to the record delimiter. The post is getting a rewrite to correct.

While the Nanocom One is viewed as a bit of a dinosaur there is still a body of opinion that it is far more reliable for .map uploads than it's replacement - the Evo. And there are still a lot of them in circulation, and that means there a quite a few logs in the binary .fu1 format.

The usual procedure is to open up the old graphic viewer PC application and use that to check files or to export as a.csv file and then use a spreadsheet app to look at the data. Either way it's not especially flexible and a bit of a PITA.

Anyway last night I sat down with a copy of Synalyze It! (Hexinator on Windows/Linux) and worked out the structure of the .fu1 log files.
Synalyze It! uses "grammar" files to apply structures a binary file which is a fairly nice way of reverse engineering file formats.

This shows how a single fuelling record looks with the fu1 grammar applied:

       Fuel Record [0] (0)
        {
            RecordDelimiter: CR LF: 0D 0A  [0]
            FuelData [0] (2)
            {
                DiagRequest_0x09 [0] (2)
                {
                    RPM: 754 [0]
                }
                DiagRequest_0x0D [0] (4)
                {
                    RoadSpeed: 1 [0]
                }
                DiagRequest_0x1A [0] (5)
                {
                    ECT: 3523 [0]
                    ECT mV: 772 [0]
                    IAT: 3311 [0]
                    IAT mV: 1309 [0]
                    AAT: 3200 [0]
                    AAT mV: 1681 [0]
                    FT: 3325 [0]
                    FT mV: 1260 [0]
                }
                DiagRequest_0x1B [0] (21)
                {
                    TPS1 mV: 571 [0]
                    TPS2 mV: 4460 [0]
                    TPS3 mV: 4677 [0]
                    Throttle %: 0 [0]
                    TPS Supply mV: 5021 [0]
                }
                DiagRequest_0x1C [0] (31)
                {
                    MAP: 10143 [0]
                    MAP raw: 10158 [0]
                    MAF: 575 [0]
                    MAF mV: 1977 [0]
                }
                DiagRequest_0x21 [0] (39)
                {
                    idleErrorRpm: -3 [0]
                }
                DiagRequest_0x40 [0] (41)
                {
                    Cyl 1: 3 [0]
                    Cyl 2: -2 [0]
                    Cyl 3: -1 [0]
                    Cyl 4: -2 [0]
                    Cyl 5: 2 [0]
                }
                DiagRequest_0x23 [0] (51)
                {
                    AAP: 9971 [0]
                    AAP raw: 10013 [0]
                }
                DiagRequest_0x37 [0] (55)
                {
                    EGR Inlet: 0 [0]
                }
                0x38 [0] (57)
                {
                    WGM Duty: 0 [0]
                }
                0x38 [0] (59)
                {
                    WGM Duty: 0 [0]
                }
                0x10 [0] (61)
                {
                    BAT mV: 13807 [0]
                    BAT_MSB_only: 35  [0]
                }
            }
        }

The format is actually pretty simple. It consists of a CRLF ('0x0D0A') pair delimiting each "observation" and then the block of data which makes up the observation. The final byte of the file looks to be a mod256 checksum.

The hex number appended to each DiagRequest_0x refers to a predefined diagnostic request used by the ECU to look up engine parameters. These are common to MSB, and EU2 and EU3 NNN, so if you request 0x10 you'll always get back 4 bytes of data containing corrected and uncorrected Battery Voltage. The problem I discovered when revising the script is that the Nanocom One logs only 3 of the 4 bytes of data for the Battery Voltage. This means the corrected voltage is OK, but only the most significant byte for the uncorrected voltage is available making it useless. I've corrected the script so it ignores the corrupt value.

Note also that until the most recent update (v1.32) the Nanocom was making duplicate requests for the WGM Duty Ratio, and using one read for the EGR Output. This update fixed the duplication in fueling mode, but it remained in Instrument mode when I last looked.

What is interesting is that the full data for each request is saved in the file, so that the Throttle % and AAT reading is present in the data. As can be see from the decoded structure the file also includes sensor voltages for all temperature sensors, and the MAF.

I found that there is also a small error in the Nanocom One's AAP value - there are two values returned - one is the raw reading, and the other is the reading corrected for variation in sensor supply voltage. The corrected value is used by the ECU for fuelling and boost limiting purposes, however the Nanocom (One) reports the raw value.

Converter

I've put together a small python script which is attached.

It's is a bit less basic than previous version, and has only been tested on macOS 10.13 with Python 2 and 3. You'll need to remove the txt extension.

The script is run from command line, with the path to the file to be converted as an argument.

python fuconvert.py file.fu1  

The converted file is saved as filename.csv in the current working directory. The .csv contains the full data from the log saved in the same order the requests appear in the .fu1 file. This is different to the Nanocom CSV order. The converter alters temperatures from Kelvin to Celsius, and adjusts decimal points to match the the Nanocom, with the exception of voltages which are left as millivolts.

Note that the AAT and TPS3 voltages will read 5000mV if the input is not connected.

download file from GitHub

Engine map scripts on GitHub

Tuesday, July 2, 2019 - 09:15

I’ve published a couple of Python scripts on GitHub that may be helpful for users of the BDM scripts.
The scripts are designed to split .map files and .bin/.ori images into component parts to allow programing using BDM.

The .bin/.ori script splits the full flash binary into four parts:

  • boot code
  • variant code
  • VIN and programming information
  • fuel map

Most .bin/.ori files have swapped byte order which needs to be corrected before BDM programming or use in .map files. The script swaps the byte order if necessary.

The .map split script extracts the variant code and fuel map from a supplied .map file, and pads the sections to suit the underlying blocksize.

You can finds the scripts on GitHub

EU2 System Demand Flowchart

Thursday, June 20, 2019 - 12:30

This is the first of what should be a series of posts which provide a little more detail about how the maps fit together.

The individual flowcharts are too large to embed while maintaining any kind of legibility so the full content is attached as a pdf at the end of the post.

EU2 System Demand Flowchart

The current flowchart covers the operation of the EU2 "System Demand" (aka Driver Demand) map and Smoke limiter.
"Driver Demand" is used in Land Rover docs to refer to the throttle position - not the maps. Another reason for describing these maps as System Demand is that the Cruise Control controls speed by using a calculated throttle % that substitutes for the "driver demanded" throttle % as the input to the System Demand maps.

The flow chart includes mention of a "new" Braking IQ map - this is currently marked as Map010 for D2, and Map011 for Defender in the NNN XDFs. Note that these maps are effectively disabled in all tunes - the "limit" is set to 100mg/fire so has no effect - but I have included as they are "active" in the sense that if you modify to a point where the values are lower than the smoke limiter the map will alter engine behavior.

Also note that the Manual maps include Autobox code and vice versa, so the Auto Torque reduce checks are used in Defenders and D2 Manuals.
The maps will have no effect, but the checks are there.

There is a scalar which selects between IAT and ECT as the temperature input to the Smoke air density adjust map. I believe this is set to IAT for all tunes so this hasn't been included to keep things simple.

The calculated airflow is discussed in the Airmass post, as MAP Airmass. As discussed in that post the MAF Airmass is calculated but not used as input into the System Demand, Smoke Limiter, or Torque Limiters in EU2 maps.

Cranking IQ, Autobox Torque Reduce IQ and Idle governor IQ calculations are quite complex so are treated a "black boxes" for clarity.

The three final boxes at lower left are parameters that are used in the Torque Limiting routines.

The EU2 Torque Limiter operation will be covered in the next post.

Attached PDF updated 7 April 2019

AttachmentSize
PDF icon EU2 System Demand Flowchart (rev3)221.6 KB

MSB XDF updates

Saturday, April 27, 2019 - 22:15

I've uploaded a long overdue update to the MSB XDF's which bring them into line with the EU2 NNN versions.

The archive now includes additional D2 Auto XDF's (MSB101260, MSB101261 and MSB101350).
Thanks to Nicky at Rovertech providing bins for these ECUs.

Currently these XDF are only correct for the first map in the .bin.
The layout of second map is usually sufficiently different to require a different XDF.
I'll look at including definitions for the second map in a future release.

Donors can access the archive after logging in.

On Td5 map tables

Tuesday, April 23, 2019 - 08:45

How map tables work…

It’s not entirely obvious how the individual tables are used in the Td5 ECU, and I don’t think I’ve explained this anywhere else.

The Basics

All tables are looked up using linear interpolation. This is done by searching along an axis to find the values on each side of the target value.

If the target value is lower than the minimum or higher than the maximum the nearest “edge” is used.

Using a EU3 torque limiter table as an example this would mean an engine speed value of 500 rpm (which would only occur when cranking) would be clamped to the 600 rpm column, while a value of 5500 rpm (possible but unlikely) would be clamped to the 5000 rpm column.

EU3 Torque Limiter

If the target value was within the bounds of the table, say 2400 rpm, the value would determined by finding the axis values immediately above and below - 2200 rpm and 2500 rpm.

The interpolation between column values is done by finding the fraction of the difference between the column values at which the target value falls.

$$\frac{2400 - 2200}{2500 - 2200} = \frac{200}{300} = 0.66666$$

Then the difference between the table value for 2200rpm and 2500rpm is multiplied by the fraction to determine the value off the fraction.

$$(44.25 - 42.53)0.6666 = 1.15$$

And finally the lower column value is added to the fractional value to give the limter value at 2400rpm.

$$42.53 + 1.15 = 43.68$$

Essentially the method assumes there is a straight line (hence the linear) joining the points represented by the x-axis and table values.

Higher Dimensions

The same basic principles are applied to 2 and 3 dimensional tables. With 2 dimensional tables the ECU interpolates on both the x and y axis, with the column and row values representing the four corners of a cell. If you look at the graph view of a 2d table the intersections of grid lines represent cell corners and the flat surfaces that fill the cells are all the possible interpolated values that occur between the corner values.

Two sets of tables are actually 3d tables. These are the fuel temp compensation tables, and the inject duration tables.

With these tables there is a hidden third dimension - rpm in the case of fuel temp, and advance in the case of inject duration.

Using the inject duration table as an example, the ECU first determines which tables are above or below the current advance/retard value in the same was as described above. Then the inject duration interpolated from inject quantity and rpm for the maps on either side of the adv/ret value, and the inject duration values are interpolated to find the fractional duration amount.

duration tables as a cube

In effect the duration table is a cube with inject quantity, rpm and advance/retard as the three axes (please excuse the wonky illustration!). The first and last duration tables are sides of the cube, and the remaining table(s) divides the space between the sides equally.

MSB map switching

Wednesday, April 17, 2019 - 07:00

I was having a look at the MSB diagnostics a couple of days ago and came across the request for switching between maps.
Some MSB ECU's - most often the Auto versions - come with tunes former than one market and the switching function provides a method for selecting the correct tune.

NOSELECT "Tune"

The NOSELECT tune is default on a new spare ECU. The tune provides just enough functionality to start an engine.
The duration maps are 2 x 2 tables that have rows for 0 and 750 rpm, so should allow the engine to idle.

The remaining tunes are full market specific fuel maps.
In this case the ECU is an MSB1011191 and has tunes for European and Japanese Auto D2s.

European Auto Tune

Japanese Auto Tune

With some minor modifications to the .bin this ECU should support at least one additional fuel map.
The NOSELECT "tune" seems to be referenced in several places during startup so possibly needs to be retained as is.

Even so this opens up the possibility of software switching between two or three maps without resorting to additional hardware to perform address line switching hacks.

Data tracing in Tuner Pro...

Tuesday, April 9, 2019 - 08:00

As I posted recently the data logger project has been taking up a lot of my time over the past few months.

The main intention of the logger is to enable faster, configurable data logging which can be saved into either CSV or Tuner Pro XDL formats.

Over the past few weeks I've been doing a bit of work on reverse engineering Tuner Pro's XDL data log format. I got about 95% worked out before I got in touch with Tuner Pro developer Mark Mansur who very kindly pointed out with the parts I'd got wrong and filled in the bits I couldn't figure out.

As a way of developing and testing the ADX writer code on the desktop, I've written a bunch of code that reads in an ADX file and Nanocom log file (from an Nanocom running current firmware) and uses that information to create an XDL log file.

There are some issues like the lack of Throttle % in the EVO logs that makes the convertor less useful than it should be.

As a teaser to show the possibilities ADX logging opens up this screen capture shows TP data tracing a converted Nanocom EVO log.
The MAF airflow calculation is being done in Tuner Pro RT, and a simple monitor setup shows AAP and MAP.

Pages