Scripts and Code

This section contains posts relating to scripts and code for manipulation of files related to Td5 engine maps and diagnostic log files.

Engine map scripts on GitHub

Tuesday, July 2, 2019 - 09:15

I’ve published a couple of Python scripts on GitHub that may be helpful for users of the BDM scripts.
The scripts are designed to split .map files and .bin/.ori images into component parts to allow programing using BDM.

The .bin/.ori script splits the full flash binary into four parts:

  • boot code
  • variant code
  • VIN and programming information
  • fuel map

Most .bin/.ori files have swapped byte order which needs to be corrected before BDM programming or use in .map files. The script swaps the byte order if necessary.

The .map split script extracts the variant code and fuel map from a supplied .map file, and pads the sections to suit the underlying blocksize.

You can finds the scripts on GitHub

Nanocom One log converter

Sunday, May 14, 2017 - 08:00

Updated 2019-7-8. Added link to Python 3 version of scripts on GitHub

Updated: I decided to revisit the Nanocom One log converter code today with view to incorporating it into a larger project. When I made a couple of changes to tidy things up, I discovered that I'd managed to wallpaper over a small problem in the Nanocom One log format, and also missed the bleeding obvious in regard to the record delimiter. The post is getting a rewrite to correct.

While the Nanocom One is viewed as a bit of a dinosaur there is still a body of opinion that it is far more reliable for .map uploads than it's replacement - the Evo. And there are still a lot of them in circulation, and that means there a quite a few logs in the binary .fu1 format.

The usual procedure is to open up the old graphic viewer PC application and use that to check files or to export as a.csv file and then use a spreadsheet app to look at the data. Either way it's not especially flexible and a bit of a PITA.

Anyway last night I sat down with a copy of Synalyze It! (Hexinator on Windows/Linux) and worked out the structure of the .fu1 log files.
Synalyze It! uses "grammar" files to apply structures a binary file which is a fairly nice way of reverse engineering file formats.

This shows how a single fuelling record looks with the fu1 grammar applied:

       Fuel Record [0] (0)
        {
            RecordDelimiter: CR LF: 0D 0A  [0]
            FuelData [0] (2)
            {
                DiagRequest_0x09 [0] (2)
                {
                    RPM: 754 [0]
                }
                DiagRequest_0x0D [0] (4)
                {
                    RoadSpeed: 1 [0]
                }
                DiagRequest_0x1A [0] (5)
                {
                    ECT: 3523 [0]
                    ECT mV: 772 [0]
                    IAT: 3311 [0]
                    IAT mV: 1309 [0]
                    AAT: 3200 [0]
                    AAT mV: 1681 [0]
                    FT: 3325 [0]
                    FT mV: 1260 [0]
                }
                DiagRequest_0x1B [0] (21)
                {
                    TPS1 mV: 571 [0]
                    TPS2 mV: 4460 [0]
                    TPS3 mV: 4677 [0]
                    Throttle %: 0 [0]
                    TPS Supply mV: 5021 [0]
                }
                DiagRequest_0x1C [0] (31)
                {
                    MAP: 10143 [0]
                    MAP raw: 10158 [0]
                    MAF: 575 [0]
                    MAF mV: 1977 [0]
                }
                DiagRequest_0x21 [0] (39)
                {
                    idleErrorRpm: -3 [0]
                }
                DiagRequest_0x40 [0] (41)
                {
                    Cyl 1: 3 [0]
                    Cyl 2: -2 [0]
                    Cyl 3: -1 [0]
                    Cyl 4: -2 [0]
                    Cyl 5: 2 [0]
                }
                DiagRequest_0x23 [0] (51)
                {
                    AAP: 9971 [0]
                    AAP raw: 10013 [0]
                }
                DiagRequest_0x37 [0] (55)
                {
                    EGR Inlet: 0 [0]
                }
                0x38 [0] (57)
                {
                    WGM Duty: 0 [0]
                }
                0x38 [0] (59)
                {
                    WGM Duty: 0 [0]
                }
                0x10 [0] (61)
                {
                    BAT mV: 13807 [0]
                    BAT_MSB_only: 35  [0]
                }
            }
        }

The format is actually pretty simple. It consists of a CRLF ('0x0D0A') pair delimiting each "observation" and then the block of data which makes up the observation. The final byte of the file looks to be a mod256 checksum.

The hex number appended to each DiagRequest_0x refers to a predefined diagnostic request used by the ECU to look up engine parameters. These are common to MSB, and EU2 and EU3 NNN, so if you request 0x10 you'll always get back 4 bytes of data containing corrected and uncorrected Battery Voltage. The problem I discovered when revising the script is that the Nanocom One logs only 3 of the 4 bytes of data for the Battery Voltage. This means the corrected voltage is OK, but only the most significant byte for the uncorrected voltage is available making it useless. I've corrected the script so it ignores the corrupt value.

Note also that until the most recent update (v1.32) the Nanocom was making duplicate requests for the WGM Duty Ratio, and using one read for the EGR Output. This update fixed the duplication in fueling mode, but it remained in Instrument mode when I last looked.

What is interesting is that the full data for each request is saved in the file, so that the Throttle % and AAT reading is present in the data. As can be see from the decoded structure the file also includes sensor voltages for all temperature sensors, and the MAF.

I found that there is also a small error in the Nanocom One's AAP value - there are two values returned - one is the raw reading, and the other is the reading corrected for variation in sensor supply voltage. The corrected value is used by the ECU for fuelling and boost limiting purposes, however the Nanocom (One) reports the raw value.

Converter

I've put together a small python script which is attached.

It's is a bit less basic than previous version, and has only been tested on macOS 10.13 with Python 2 and 3. You'll need to remove the txt extension.

The script is run from command line, with the path to the file to be converted as an argument.

python fuconvert.py file.fu1  

The converted file is saved as filename.csv in the current working directory. The .csv contains the full data from the log saved in the same order the requests appear in the .fu1 file. This is different to the Nanocom CSV order. The converter alters temperatures from Kelvin to Celsius, and adjusts decimal points to match the the Nanocom, with the exception of voltages which are left as millivolts.

Note that the AAT and TPS3 voltages will read 5000mV if the input is not connected.

download file from GitHub

Desktop diagnostics...

Tuesday, January 24, 2017 - 08:00

Garmen has posted up a Python script which at the moment allows the user to read out the "Nanocom Fuelling" parameters. github

The project uses a cheap serial programming interface based on the CP2102 USB to TTL converter plus some external circuitry to interface with comms levels. The trick is that the CP2012 needs to be reconfigured to suit the "non-standard" baud rates, and to allow sending a 0x00 byte with a period of exactly 25ms. It's a fairly neat trick.

Anyway Garmen's work prompted me to pull out a "dumb" OBD-II interface that I'd picked up a couple of years ago and thrown in the cupboard. Most OBD-II interfaces are based on the ELM-327 chip which basically does all the grunt work of managing the diagnostic connection and allows users to simply send OBD commands over a serial connection and read back the response.

A "dumb" interface is a USB to serial converter chip (usually a variant or clone of the FTDI FT232) and hardware to interface with the diagnostic port line levels. These are usually sold as something like "VAG-COM KKL USB Interface" - just google it... These require that the user application to handle all of the communications protocol details.

The interface I have has a cloned FT232BL. The complete lack of markings on the chip is the give away. But as the part cost $14.00AUS including postage it's hard to feel ripped off...

The nice thing about the FTDI chips is that they have a "Bit Bang" mode. This allows you to write a byte to the chip and it will hold that state until you write another byte. It's perfect for doing things like sending the 25ms wake-up message or bit-banging the 5 Baud initialisation sequence.

There are a couple of Python modules that allow access to these FTDI specific features. I'm using pyftdi.

As a rough example of how you do it...

#!/usr/bin/env python3
 
from pyftdi.ftdi import Ftdi
import time
 
HIGH = bytearray([0x01])
LOW = bytearray([0x00])
message = bytearray([0x81,0x13,0xF7,0x81])
 
 
uart = Ftdi() 
 
uart.open(0x403, 0x6001)
uart.set_baudrate(10400)
uart.set_line_property(8, 1, 'N')
 
# Enable Bitbang Mode
uart.set_bitmode(0x01, 0x01)
 
# Set TX line high
uart.write_data(HIGH)
 
 
# Short sleeps in the loop seem to improve accuracy...
# Using time.sleep(.025) for the full delay is very jittery.
# time.perf_counter() can be used instead time.monotonic()
# requires Python 3.3 or later
 
start = time.monotonic()
while (time.monotonic() <= start + 0.35):
    time.sleep(0.01)
 
# Set TX line low
uart.write_data(LOW)
 
start = time.monotonic()
while (time.monotonic() <= start + 0.0245):
   time.sleep(0.00025)
 
# Set TX line high
uart.write_data(HIGH)
 
start = time.monotonic()
while (time.monotonic() <= start + 0.0245):
     time.sleep(0.00025)
 
# Disable Bitbang Mode
uart.set_bitmode(0x00, 0x00)
 
# Send serial message
uart.write_data(message)

Which should give you something like:

Wakeup Sequence

The ISO14230 spec states the total time for the wakeup time (Twup) should be 50ms +/- 1ms. This code gives Twup of 50.67ms, so is within spec.
As reference the Nanocom sets the line low for 26.54ms and then idles high for 118.9ms giving a Twup = 145.44ms. The Td5 ECU is pretty forgiving.

Td5 KeyGen now on github

Tuesday, January 10, 2017 - 15:45

As a copy of the Seed-Key.txt database has recently turned up in a post on the AULRO forum I've decided there isn't much point in keeping my keygen code under wraps.

Td5 Keygen Github repo

I started doing some testing against the Seed-Key database and came up with some anomalies. As a result I've double checked the code against the ECU assembly code, and I also wrote ended up writing a version in MATLAB to verify what I had was ok. When I ran the MATLAB version against the Seed list I discovered that there are about 308 questionable keys in the database. These are mainly blocks of either 0x2020 or 0xF781 keys which are clearly fillers. This is an example...

SEED_KEY.txt:

F0DD 2020
F0DE F781
F0DF F781
F0E0 F781
F0E1 F781
F0E2 F781
F0E3 F781
F0E4 F781
F0E5 2020

KEY_GEN:

F0DD 7D51
F0DE F9A1
F0DF FCD1
F0E0 2607
F0E1 9303
F0E2 2A0F
F0E3 9506
F0E4 321E
F0E5 990E

There are also two additional values which I believe are wrong.

The Keygen generated values match the remaining 65228 "good" keys.

I'm quietly confident the code is OK, but someone has a 0.47% error rate.