Datalogger Project - XDL logging milestone.

Saturday, August 3, 2019 - 16:45

The past week has seen a lot of progress on the Datalogger.

The event-driven non-blocking FAT32 library I posted about earlier in the year was integrated into the logging module and after some initial hiccups is now working well.

CSV logging to card is now running well. Auto detect of MSB // NNN hardware has been implemented and throttle mapping switched on that basis. MSB ECU's report two throttle tracks, throttle %, and supply voltage, while NNN ECU's report three throttle tracks.
A diagnostic request that returns the raw ADC conversion values, and fault codes are also dependent on ECU hardware type.

EGT probe support has been integrated into the CSV logging. The logger checks if to see if there is a probe connected at the start of each log run, and adds the EGT column if required.

The big news - for me anyway - is that logging to Tuner Pro XDL format is working. The code for the XDL log format was written yesterday - based on the reverse engineering done for the Nanocom conversion as posted previously - , and a small bug in a log buffering code which prevented the logs writing to card was fixed this morning.

I've only had a chance to do a few short runs around home - lots of 40kph zones and slow moving traffic. Aside from my laptop going to sleep after 15 minutes and killing the longest of the logs, the logging seems to be pretty stable.

This evening I looked up my test ECU and eft the logger run on the bench recording to XDL for a bit over three hours without any issues.

As a bit of a teaser, this is a screenshot of Tuner Pro RT tracing from a log I recorded today.
The black spots on the tables indicate the current running point.

XDL Datatracing

None of the parameters being used are calculated - it's all logged ECU data.

The to-do list is still pretty long, but I'm now feeling far more optimistic than I was 3 months ago.

On Td5 map tables

Tuesday, April 23, 2019 - 08:45

How map tables work…

It’s not entirely obvious how the individual tables are used in the Td5 ECU, and I don’t think I’ve explained this anywhere else.

The Basics

All tables are looked up using linear interpolation. This is done by searching along an axis to find the values on each side of the target value.

If the target value is lower than the minimum or higher than the maximum the nearest “edge” is used.

Using a EU3 torque limiter table as an example this would mean an engine speed value of 500 rpm (which would only occur when cranking) would be clamped to the 600 rpm column, while a value of 5500 rpm (possible but unlikely) would be clamped to the 5000 rpm column.

EU3 Torque Limiter

If the target value was within the bounds of the table, say 2400 rpm, the value would determined by finding the axis values immediately above and below - 2200 rpm and 2500 rpm.

The interpolation between column values is done by finding the fraction of the difference between the column values at which the target value falls.

$$\frac{2400 - 2200}{2500 - 2200} = \frac{200}{300} = 0.66666$$

Then the difference between the table value for 2200rpm and 2500rpm is multiplied by the fraction to determine the value off the fraction.

$$(44.25 - 42.53)0.6666 = 1.15$$

And finally the lower column value is added to the fractional value to give the limter value at 2400rpm.

$$42.53 + 1.15 = 43.68$$

Essentially the method assumes there is a straight line (hence the linear) joining the points represented by the x-axis and table values.

Higher Dimensions

The same basic principles are applied to 2 and 3 dimensional tables. With 2 dimensional tables the ECU interpolates on both the x and y axis, with the column and row values representing the four corners of a cell. If you look at the graph view of a 2d table the intersections of grid lines represent cell corners and the flat surfaces that fill the cells are all the possible interpolated values that occur between the corner values.

Two sets of tables are actually 3d tables. These are the fuel temp compensation tables, and the inject duration tables.

With these tables there is a hidden third dimension - rpm in the case of fuel temp, and advance in the case of inject duration.

Using the inject duration table as an example, the ECU first determines which tables are above or below the current advance/retard value in the same was as described above. Then the inject duration interpolated from inject quantity and rpm for the maps on either side of the adv/ret value, and the inject duration values are interpolated to find the fractional duration amount.

duration tables as a cube

In effect the duration table is a cube with inject quantity, rpm and advance/retard as the three axes (please excuse the wonky illustration!). The first and last duration tables are sides of the cube, and the remaining table(s) divides the space between the sides equally.

EU2 System Demand Flowchart

Thursday, June 20, 2019 - 12:30

This is the first of what should be a series of posts which provide a little more detail about how the maps fit together.

The individual flowcharts are too large to embed while maintaining any kind of legibility so the full content is attached as a pdf at the end of the post.

EU2 System Demand Flowchart

The current flowchart covers the operation of the EU2 "System Demand" (aka Driver Demand) map and Smoke limiter.
"Driver Demand" is used in Land Rover docs to refer to the throttle position - not the maps. Another reason for describing these maps as System Demand is that the Cruise Control controls speed by using a calculated throttle % that substitutes for the "driver demanded" throttle % as the input to the System Demand maps.

The flow chart includes mention of a "new" Braking IQ map - this is currently marked as Map010 for D2, and Map011 for Defender in the NNN XDFs. Note that these maps are effectively disabled in all tunes - the "limit" is set to 100mg/fire so has no effect - but I have included as they are "active" in the sense that if you modify to a point where the values are lower than the smoke limiter the map will alter engine behavior.

Also note that the Manual maps include Autobox code and vice versa, so the Auto Torque reduce checks are used in Defenders and D2 Manuals.
The maps will have no effect, but the checks are there.

There is a scalar which selects between IAT and ECT as the temperature input to the Smoke air density adjust map. I believe this is set to IAT for all tunes so this hasn't been included to keep things simple.

The calculated airflow is discussed in the Airmass post, as MAP Airmass. As discussed in that post the MAF Airmass is calculated but not used as input into the System Demand, Smoke Limiter, or Torque Limiters in EU2 maps.

Cranking IQ, Autobox Torque Reduce IQ and Idle governor IQ calculations are quite complex so are treated a "black boxes" for clarity.

The three final boxes at lower left are parameters that are used in the Torque Limiting routines.

The EU2 Torque Limiter operation will be covered in the next post.

Attached PDF updated 7 April 2019

PDF icon EU2 System Demand Flowchart (rev3)221.6 KB

EU3 MAF timeout disable

Tuesday, January 8, 2019 - 10:45

I've had a few requests for details on how to disable the timeout on EU3 maps when the MAF sensor is disconnected.
Without this mod the throttle on EU3 maps is unresponsive for 20-30 seconds when the engine is first started.

It's quite simple to do by altering the MAF "out of range" timeout check from a conditional "if timeout branch" to an unconditional "branch".

You'll need to use a HEX editor for this.

First open up you .map file in your favorite hex editor - I use Hex Fiend on macOS for stuff like this.

With the .map open search for a sequence of hex bytes: 66 30 31

Unmodified .map

There should be only one occurrence of this sequence in the .map file at an offset of somewhere around 0xD100 - 0xD400.

Next, change the 66 to 60 - this alters the conditional branch instruction to an unconditional branch, meaning the ECU always executes the "no MAF" code.

Modifed .map file

Save the modified .map file and you are done.

Be aware that this mod “lobotomizes" the EU3 maps to a EU2 style single smoke map configuration.
The ECU will only use the main high range smoke map (map 60).

This should work for all EU3 variants.
If you can't locate the byte sequence let me know which variant you are working with.

Note: The checksum needs to be updated if you are loading the .map using the Nanocom. This can be done by loading the .map file into Tuner Pro with the appropriate XDF and then saving the "bin". The checksum is updated on save. Td5 Map Editor will also update the checksum. If you are using Td5 ME you'll have to either make and the reverse a change to the map so the file is marked as "dirty" then save, or "save as". Thanks to Neil for mentioning this omission.

Site updates

Tuesday, August 7, 2018 - 21:00

While it shouldn't be obvious, I've moved the website to a new host. The process has been fairly problem free but if anything is acting up let me know.

On a related topic Google Chrome and other browsers have started marking sites that aren't secured will SSL certificates as "not secure". It's really a bit of overkill for a site like this, but as the Not Secure warning is kind of off putting I've moved the site across to the secure protocol.

Chrome is currently still complaining because the images in a lot of the posts are hard coded to the insecure address. I'm slowing working my way through and fixing up the links.

For the moment the front page and the donor login pages are now flagged as "Secure".

Tuner Pro XDF's

Tuesday, August 22, 2017 - 16:00

The donor-ware XDF's for NNN ECU's had a bit of a version bump today...

Note: These XDF's and all future updates are available anyone who supports the site with a donation of $5.00 USD or more.

These are changes I could remember:

Version 2.0 20170822
- Add SOI maps
- Add Smoke compensation maps
Note that EU3 maps use an adjust map and scaling map pair.
The scaling map is a multiplier so if there is 0 in the scaling map the adjust output is 0.
It's going to take me a bit of time to document so please be patient.
Information will be posted when it is ready.
- Add Air Conditioning load map
- Add Auto Torque Reduction IQ map
- Add Auto Engine Torque map
- Alter scalar naming
- Add Waste Gate Modulator control maps and scalars
- Add "hidden" Injector Idle Response map to EU2 XDFs
- other stuff...

There were a few tweaks to improve axis scaling, and descriptions added...
As always the XDF's are a work in progress.

Tuner Pro Update

Mark Mansur has just uploaded a new version of Tuner Pro which fixes a bug introduced in the previous version.
The bug prevented the patches included in the Td5 XDF's from applying.
You can download the new version from the Tuner Pro Download Page.

Overboost Fuel Cut Map

Wednesday, August 2, 2017 - 12:30

The donor XDF's identify a map that limits fuelling when overboost occurs, which I've named "Overboost Fuel Cut".
That naming seems to have created a bit of confusion as I've had a few people email asking if they should increase this map to match the torque limiter.

Where the confusion arises comes from common belief that the kangaroo hopping or jerking that occurs when the wastegate fails to operate is a fuel cut caused by an overboost condition. What is actually occurring is quite different.

Back to the ADC

I've posted previously about the signal flow from MAP sensor through the ECU Hardware to the Analog Digital Conversion module.

This is roughly:

  • Sensor converts pressure to voltage
  • Resistor divider drops voltage by approx 10%
  • ADC converts voltage to a value between 0-1023
  • The value is checked against ADC Max and Min

If the value is above ADC Max, a logged high fault is set, and if below ADC Min a logged low fault is set, and the sensor default value is used.

And this is what occurs when the wastegate fails to open. The pressure quickly increases to a value higher than ADC Max, and the sensor default is set.
Limit checking happens at the first stage of processing after AD conversion, and well before any fuelling calculations. While this might appear to be a fuel cut that is simply a side effect of the dramatic drop in calculated airmass.

The real overboost fuel cut

Overboost Fuel Cut Map is used specifically when the current boost exceeds the boost limit value.
The current boost value is calculated as the difference between Manifold Absolute Pressure (MAP) and Ambient Absolute Pressure (AAP). This is obviously influenced by the pressure drop across the air filter and airbox.

Say for example the reading are 240kpa MAP and 100kpa AAP, in this case boost is 140kpa.
However if there is a 3kpa drop caused by the airfilter, that becomes 240kpa MAP and 97kpa AAP which gives 143kpa boost.

The stock ADC Max limit for the MAP is the equivalent of about 242kpa and stock Boost Limit is set at 142kpa. Due to the drop in AAP at high boost you'll generally reach the Boost Limit at least 2-3kpa before the ADC Max limit.

Once boost exceeds the Boost Limit, the ECU limits the amount of fuel injected using the Overboost Fuel Cut map. This limiting remains in place until the current boost drops below the level set by the Boost Limit Recovery parameter.

A few thoughts

My take on this map is that it is used by the ECU to help control boost levels after overboost is detected, and prior to the MAP value exceeding the maximum limit. With stock values this is a small window of perhaps 3kpa // 0.4psi. It is basically a last ditch effort by the ECU to control overboost, and I personally don't' see any legitimate reason to mod this map.

The best approach is to set boost limit and boost limit recovery values at the maximum value you expect to run.
If this is higher than 150kpa/1.5bar boost you will need to replace the MAP sensor with a wider range unit.
Then adjust the ADC Max value to suit - some thing like 250kpa if you have set the boost limit to 150kpa.
Working in this way you will only hit the Overboost Fuel Cut when the boost reading goes above your maximum expected value.

Td5 Datalogger updates

Sunday, June 11, 2017 - 08:00

I've come to something of a Catch 22 where I need detailed logging to properly document and verify the waste gate modulator, and the code that handles torque reduction requests from the auto gear box before I can post details of the maps. On the other hand, if I work on the datalogger nothing much happens on the website.

The combination of changes to the Microchip tool chain which caused issues with my old code, a failed hard drive which lead to the discovery that the Microchip peripheral library installer does not work on current versions of macOS, and the need for a rethink and rewrite has meant the datalogger has been out of action since late December.

When I was playing around with the VAGCOM adapter and Python earlier in the year I had ported//rewritten part of my C code and found that using a publish and subscribe framework solved a few of the issues I'd run into. As a result of that little "ah ha" moment and the hassle of dealing with the Microchip libraries I've been working on writing and unit testing peripheral drivers and a publish & subscribe framework in C.

Over the past few weeks I've been progressively bolting together the code that manages the communications with the Td5 ECU.

The way I've decided to approach this is to look at the ISO14230 docs and build a module that handles the startCommuncations, sendData and stopCommunications commands as the base of the logger/interface. That section is now working quite well and at this stage only requires some additional error handling to deal with cases where the ECU response times out mid-message.

From that base I've been working on adding a command interface and USB interface. The idea is to provide a basic set of ELM-like AT commands to configure the settings from default. The current defaults for "pass through" operation are set to connect to a Td5 ECU in manufacturer specific diagnostic mode - Testbook diagnostics in other words.

In practice this means you can hook up the interface and send requests without worrying about the underlying communications protocols. This is similar to how the ELM327 operates.

This is a short demo of the pass through mode, operating over a serial usb connection. The delay between hitting return on the "21 01" command and the response includes the time taken to connect and authenticate to the ECU.

AAP/AAT sensor swapping

Friday, April 7, 2017 - 12:15

This is a fairly niche modification.

EU2 and EU3 engines are fitted with significantly different airbox sensors.

The EU2 uses a three wire Ambient Air Pressure sensor, while the EU3 uses a four wire Ambient Air Pressure/Ambient Air Temp sensor.

The curve of the AAP portions of the two sensors are different and require different parameters to give the correct reading.

Without adjusting the parameters there is a misread of around 10kPa. You'll get an over-read ( -700 m altitude) with EU2 AAP + EU3 tune, and under read (+700m altitude) with EU3 AAP + EU2 tune.

The problem is not so bad with EU2 AAP + EU3 tune as the engine assumes higher air density in some correction maps and will INCREASE injected fuel and give a 0.1 bar increase in the boost limit. The boost level is MAP - AAP so reducing AAP by 0.1 results in boost levels 0.1 higher than the would be with correct setup. I suspect this is why you often hear the comment that an EU3 tune drives better that the correct EU2 tune.

If you've addressed the issue by installing a 4-wire sensor - replace airbox lid, sensor and run an extra wire back to the ECU - the problem occurs when you want to run an EU2 map on the motor. The under-read means the ECU uses corrections which reduce the fuelling plus the boost limit is reduced by 0.1 bar. It guarantees bad performance.

The fix is in

The way to fix this problem is to use the correct parameters for the AAP you have installed. Search for the values for the base map and replace with values for the sensor you are using.

EU3 - 4 wire sensor
multiplier: 13171 ( 0x3373 )
offset: 267 ( 0x010B )

EU2 - 3 wire sensor
multiplier: 10410 ( 0x28AA )
offset: 1227 ( 0x04CB )

It's not too hard to find these values with a hex editor as I think they are fairly unique. As a rough guide they are somewhere around an offset of 0x6A0 from the start of the fuel map. In a Nanocom .map the fuel map always begins at 0x19010.

The donor-ware XDF's now have a patch that swaps the values. It's a bit rough as it shows a stock EU3 tune as being patched, so installing the EU2 parameters requires "reversing" the patch.

ABS Modulator Option A additional fix

Tuesday, April 4, 2017 - 11:15

I can't say I've had this issue personally, but when my brother owned his D2 he made an interesting find in relation to the Option A fix.

While this information was posted to AULRO at the time it seems to have been ignored and/or forgotten, so I've decided to post the information up here.

One of the potential causes of the Three (or Four) Amigos is an dry or cracked solder joint on the "snout" that the Shuttle Valve Switch plugs into.

Option A (2/3rds the way down this page) provides a way of solving the issue by removing the plastic from behind the snout pins to allow the joint to be resoldered. When my brother went to do this fix he discovered these solder joints were fine. Even after a precautionary resoldering the earth connection was still intermittent.

What Steve noticed was that the "snout" could be moved up and down slightly. After a bit of exploratory digging into the external potting he found a second location with two solder connections where the "snout" joins to the circuit board that holds the external connectors. Apparently these two pins had some kind of corrosion and required a bit of clean up before they could be soldered. After soldering these two joints the play in the "snout" was eliminated and solved the connection remained solid even when the snout was wiggled. So if you are planning on doing the Option A fix it's worthwhile checking that the connection remains solid when moving the "snout".

This photo shows the location of the pins and the kind of excavation required to access. The hole needs to be filled post repair but is hidden behind the mounting so this doesn't need to be especially tidy.

location of snout pins