Re: [Ovmsdev] New pull requests pending
Hi Mark, Would it possible to have a 'Dev' branch in the main repository for those of us needing assistance to get a feature to the point it can be merged into the 'Master'? If that's not preferred and we use a branch on our own github clone, is it as simple as providing the link to the branch on this email list to invite others to collaborate/review? With the latter approach we may miss out on input from others not actively monitoring this forum I guess. Regards Derek
Hi Derek. I would prefer the first option as discussing source is way easier on a PR in github as here in the mailing list. Soko On 17 August 2020 06:53:40 CEST, Derek Caudwell <d.caudwell@gmail.com> wrote:
Hi Mark,
Would it possible to have a 'Dev' branch in the main repository for those of us needing assistance to get a feature to the point it can be merged into the 'Master'?
If that's not preferred and we use a branch on our own github clone, is it as simple as providing the link to the branch on this email list to invite others to collaborate/review?
With the latter approach we may miss out on input from others not actively monitoring this forum I guess.
Regards Derek
PS: On a second thought I kinda cannot see the advantage of a PR request into a Dev branch in comparison into the master branch. If the PR is changed and improved until it is approved by you guys it should be OK to go into the master. On 17 August 2020 07:00:37 CEST, Soko <ovms@soko.cc> wrote:
Hi Derek.
I would prefer the first option as discussing source is way easier on a PR in github as here in the mailing list.
Soko
On 17 August 2020 06:53:40 CEST, Derek Caudwell <d.caudwell@gmail.com> wrote:
Hi Mark,
Would it possible to have a 'Dev' branch in the main repository for those of us needing assistance to get a feature to the point it can be merged into the 'Master'?
If that's not preferred and we use a branch on our own github clone, is it as simple as providing the link to the branch on this email list to invite others to collaborate/review?
With the latter approach we may miss out on input from others not actively monitoring this forum I guess.
Regards Derek
Yes, ^this. The main repository master should be relatively stable. If we need something potentially breaking, or a major restructuring of how things are done, then we should do that in a main repository branch. That is the reason for the upcoming 3.3 branch (as it restructures the modem driver, which could be potentially very breaking - more on this later, when I am ready to release). Preparing stuff for inclusion in the main repository should in general be done in local clone copies, then pull-requests made when ready. Regards, Mark
On 17 Aug 2020, at 1:08 PM, Soko <ovms@soko.cc> wrote:
PS: On a second thought I kinda cannot see the advantage of a PR request into a Dev branch in comparison into the master branch. If the PR is changed and improved until it is approved by you guys it should be OK to go into the master.
On 17 August 2020 07:00:37 CEST, Soko <ovms@soko.cc> wrote: Hi Derek.
I would prefer the first option as discussing source is way easier on a PR in github as here in the mailing list.
Soko
On 17 August 2020 06:53:40 CEST, Derek Caudwell <d.caudwell@gmail.com> wrote: Hi Mark,
Would it possible to have a 'Dev' branch in the main repository for those of us needing assistance to get a feature to the point it can be merged into the 'Master'?
If that's not preferred and we use a branch on our own github clone, is it as simple as providing the link to the branch on this email list to invite others to collaborate/review?
With the latter approach we may miss out on input from others not actively monitoring this forum I guess.
Regards Derek _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Also, on the general handling for pull requests: It's generally advised to create separate branches for all your pull requests, each concentrating on just a single feature / development topic. You can have local development branches as many as you need. Publish changes ready to be merged into the dedicated pull request branch (i.e. merge them locally into that branch), then create a pull request on that branch. Pull requests track the branch they are created on, so any new commit added during the discussion will automatically update the pull request. Once a pull reqest has been merged into the master, you can simply delete the branch. Regards, Michael Am 17.08.20 um 07:38 schrieb Mark Webb-Johnson:
Yes, ^this.
The main repository master should be relatively stable. If we need something potentially breaking, or a major restructuring of how things are done, then we should do that in a main repository branch. That is the reason for the upcoming 3.3 branch (as it restructures the modem driver, which could be potentially very breaking - more on this later, when I am ready to release).
Preparing stuff for inclusion in the main repository should in general be done in local clone copies, then pull-requests made when ready.
Regards, Mark
On 17 Aug 2020, at 1:08 PM, Soko <ovms@soko.cc <mailto:ovms@soko.cc>> wrote:
PS: On a second thought I kinda cannot see the advantage of a PR request into a Dev branch in comparison into the master branch. If the PR is changed and improved until it is approved by you guys it should be OK to go into the master.
On 17 August 2020 07:00:37 CEST, Soko <ovms@soko.cc <mailto:ovms@soko.cc>> wrote:
Hi Derek.
I would prefer the first option as discussing source is way easier on a PR in github as here in the mailing list.
Soko
On 17 August 2020 06:53:40 CEST, Derek Caudwell <d.caudwell@gmail.com <mailto:d.caudwell@gmail.com>> wrote:
Hi Mark,
Would it possible to have a 'Dev' branch in the main repository for those of us needing assistance to get a feature to the point it can be merged into the 'Master'?
If that's not preferred and we use a branch on our own github clone, is it as simple as providing the link to the branch on this email list to invite others to collaborate/review?
With the latter approach we may miss out on input from others not actively monitoring this forum I guess.
Regards Derek
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
Hi, I'm trying to find the correct value conversions from the CAN-Hex values to the display values in VCDS. I cannot believe the values I'm getting though so I'd like to ask you for help guys as I ran out of ideas... Its about the Ah and kWh charged and discharged. All 4 values are at the same PID and I get 16 bytes as reply which suggests 4 bytes per value. (I've already checked if there are other values in this PID but there are not). Here is what I've done checked so far: 1. Swapping Ah data with kWh data. No luck as the conversion factors are just other weird values 2. Looking for an offset from 0. No luck as the delta-section below proves 3. As they are 4 bytes they may be no integer but float. Some of you might already see by the hex values that it can't be float I'm very grateful for any thoughts as this just can't be right as the conversion factor differs slightly with every day I'm taking new data... It might just be me interpreting the CAN-Hex data in a wrong way, but I can't find the error.
I don’t have much information to be able to help. All I see is a screenshot of numbers. Also not really sure what you mean by ‘I got 16 bytes as reply’, as CAN frames are limited to 8 bytes total. Or are you talking about a multi-frame reply (ISO-TP, ISO15765)? Simple PID request/response frames are not usually complicated. Simple byte ordering, byte boundaries, single value, and scaling by offset and multiplier/divisor. But if structured data is being returned in a multi-frame response, or actively transmitted on the bus as a broadcast, then things can get more difficult - encoding can be big / little endian, and the values don’t have to start on byte boundaries or have lengths divisible by 8 bits. Can you break it down easier? For example, for the AmpHrs Chrg number, provide a textual canlog-monitor of the VCDS request and response, let us know the PID number requested, response received, and value shown in VCDS. You need to start by narrowing it down to a single request/response for a single known VCDS displayed value. On the positive side, once you’ve worked out the encoding for one, the manufacturer normally uses the same style for all the others (at least from that one ECU). Regards, Mark.
On 17 Aug 2020, at 3:45 PM, Soko <ovms@soko.cc> wrote:
Hi,
I'm trying to find the correct value conversions from the CAN-Hex values to the display values in VCDS. I cannot believe the values I'm getting though so I'd like to ask you for help guys as I ran out of ideas...
Its about the Ah and kWh charged and discharged. All 4 values are at the same PID and I get 16 bytes as reply which suggests 4 bytes per value. (I've already checked if there are other values in this PID but there are not).
Here is what I've done checked so far:
Swapping Ah data with kWh data. No luck as the conversion factors are just other weird values Looking for an offset from 0. No luck as the delta-section below proves As they are 4 bytes they may be no integer but float. Some of you might already see by the hex values that it can't be float I'm very grateful for any thoughts as this just can't be right as the conversion factor differs slightly with every day I'm taking new data...
It might just be me interpreting the CAN-Hex data in a wrong way, but I can't find the error.
<gponehmpacgmdjkc.png>
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Morning, Sorry, I reckon I've been at it for too long and did imply a lot of things which are not obvious for someone who didn't do it a million times already (as it feels to me :/ ). Let's do it step by step and (hopefully) my error is somewhere in the early steps... I have OVMS (no polls active) and VCDS connected to the OBD port via a Y cable. In OVMS I turn the canlog on so I can see what VCDS is sending... In VCDS I go into the 8C ECU and choose "Erweiterte Messwerte". There I have a list of all values which I can activate so VCDS polls it from the ECU. VCDS polls/refreshes those chosen values roughly 10 times a second. So I activate AmpHrs charged and discharged, and KwH charged and discharged from the list (4 values in total). VCDS shows me the final converted values on the screen: OVMS logs the following on the bus: As far as I understand it are the red lines the polls by VCDS (as they are 7E5=the header of the ECU) and the green lines are the replies. I don't understand the red "7E5 30..." line but I reckon its just like a "Please give me the rest of the data". All 4 values are in the same PID "1E 32" (red, bold and underlined above). And the green lines, the bold and underlined HEX values are the actual data with the last "aa" being no part of it. This is my procedure to get the data bytes from the canlog-monitor lines...Maybe here is my first error somewhere?! So I have 16 bytes in total to get the 4 values VCDS is showing me: *00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04* Simplicity and the value-Ids (some internal numbers) in the list of VCDS suggest that same order as given above. So the first 4 bytes is AmpHrs Chrg, than AmpHrs Dischrg and so on. This leads us to this relation: I've also confirmed this by waiting (with ignition on) until the last fraction of AmpHrs/kWh discharged values where changing in VCDS to -1154,307 and -362,448 and comparing it with the changed hex values. These hex numbers changed: *00 09 de e1 ff f6 53 2_/0/_/__/ 00 31 62 92 ff d0 88 0_/3/_/__/* Having said that: Not every change in the hex value changes the display value. I have more detail data to this if needed... The way they change suggests 2's complement storage. So let's convert them into integers: Now, for kWh Dischrg as an example, the factor to get from -3110908 to -326,447 is: -3110908/-362,447=8583,07007645256. This is a weird one as all the other factors so far have been nice numbers! If I do this for all 4 values I get these factors: This cannot be right and therefore a waited a few weeks and did everything above again. The result was the same unfortunately: Some other values have an offset. Like the power-efficiency on the charger-ECU. The value=0x00 means 75% and 0x06 means 75,6%. It's a quite clever way of using just one byte for this value! So I thought they use some offset here to. But if I take the delta of both days I logged I get the same factors: For AmpHrs Chrg the first value was 1177,611 and the value week later was 1424,809 which gives the delta of 247,198. So now I'm really out of ideas and thought I ask you guys. As you can see all conversion factors are close to each other but not exactly the same. And given that the VW engineers are quite clever I can't accept they would use such weird factors for a conversion. If you have now another look at the one screenshot I took in my first mail you should understand it as it is just the complete version of the bits I did here... Any ideas are highly welcome. Soko
Soko, sometimes values are encoded in two parts, for example a 32 bit value may consist of a 20 bit part + a 12 bit part, with different scaling on each part (low + high resolution). Scaling factors also can be non-intuitive, even the OBD2 standard includes factors like 100/255. Internal values like these may be multiples of some odd current sensor resolution. If (!) bit chunked parts represent integer and fractional parts of the value, you may get more insight if you catch a change from (x).999 to (x+1).000. Be aware three fractional digits may be a rounded representation. Another option to check is the VCDS software. These systems normally are rule based, so if you can find the rule set, you may be able to find the value conversion scheme. Yet another option is to send crafted fake values to the VCDS to see how they translate. Regards, Michael Am 18.08.20 um 09:36 schrieb Soko:
Morning,
Sorry, I reckon I've been at it for too long and did imply a lot of things which are not obvious for someone who didn't do it a million times already (as it feels to me :/ ). Let's do it step by step and (hopefully) my error is somewhere in the early steps...
I have OVMS (no polls active) and VCDS connected to the OBD port via a Y cable. In OVMS I turn the canlog on so I can see what VCDS is sending...
In VCDS I go into the 8C ECU and choose "Erweiterte Messwerte". There I have a list of all values which I can activate so VCDS polls it from the ECU. VCDS polls/refreshes those chosen values roughly 10 times a second.
So I activate AmpHrs charged and discharged, and KwH charged and discharged from the list (4 values in total). VCDS shows me the final converted values on the screen:
OVMS logs the following on the bus:
As far as I understand it are the red lines the polls by VCDS (as they are 7E5=the header of the ECU) and the green lines are the replies. I don't understand the red "7E5 30..." line but I reckon its just like a "Please give me the rest of the data". All 4 values are in the same PID "1E 32" (red, bold and underlined above). And the green lines, the bold and underlined HEX values are the actual data with the last "aa" being no part of it. This is my procedure to get the data bytes from the canlog-monitor lines...Maybe here is my first error somewhere?!
So I have 16 bytes in total to get the 4 values VCDS is showing me: *00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04*
Simplicity and the value-Ids (some internal numbers) in the list of VCDS suggest that same order as given above. So the first 4 bytes is AmpHrs Chrg, than AmpHrs Dischrg and so on. This leads us to this relation:
I've also confirmed this by waiting (with ignition on) until the last fraction of AmpHrs/kWh discharged values where changing in VCDS to -1154,307 and -362,448 and comparing it with the changed hex values. These hex numbers changed: *00 09 de e1 ff f6 53 2_/0/_ 00 31 62 92 ff d0 88 0_/3/_*
Having said that: Not every change in the hex value changes the display value. I have more detail data to this if needed...
The way they change suggests 2's complement storage. So let's convert them into integers:
Now, for kWh Dischrg as an example, the factor to get from -3110908 to -326,447 is: -3110908/-362,447=8583,07007645256. This is a weird one as all the other factors so far have been nice numbers!
If I do this for all 4 values I get these factors:
This cannot be right and therefore a waited a few weeks and did everything above again. The result was the same unfortunately:
Some other values have an offset. Like the power-efficiency on the charger-ECU. The value=0x00 means 75% and 0x06 means 75,6%. It's a quite clever way of using just one byte for this value! So I thought they use some offset here to. But if I take the delta of both days I logged I get the same factors:
For AmpHrs Chrg the first value was 1177,611 and the value week later was 1424,809 which gives the delta of 247,198.
So now I'm really out of ideas and thought I ask you guys. As you can see all conversion factors are close to each other but not exactly the same. And given that the VW engineers are quite clever I can't accept they would use such weird factors for a conversion.
If you have now another look at the one screenshot I took in my first mail you should understand it as it is just the complete version of the bits I did here...
Any ideas are highly welcome.
Soko
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
Thanks Michael, A lot of options to check out here, great! After reading my email again I had a light bulb moment which relates to your 100/255 factor example. I thought what if they wanted to make the maximum number a (kinda) nice value. So I gave math a try and found that a factor of (2^32/2)/250200 works perfectly for the kWh values. So they may have said 250200 kWh is the maximum value we want to represent with 4 bytes. 250000 would have been more nice but it doesn't give me the VCDS values. Having said that: If VCDS does also reverse engineering maybe they got it wrong ;) Or do they pay VW for the data? I've also made a copy the VCDS Runtime and all files. But I can't believe that they would put such crucial info in an XML file or something similar. I reckon you need a disassembler at least... Soko On 18.08.2020 10:05, Michael Balzer wrote:
Soko,
sometimes values are encoded in two parts, for example a 32 bit value may consist of a 20 bit part + a 12 bit part, with different scaling on each part (low + high resolution).
Scaling factors also can be non-intuitive, even the OBD2 standard includes factors like 100/255. Internal values like these may be multiples of some odd current sensor resolution.
If (!) bit chunked parts represent integer and fractional parts of the value, you may get more insight if you catch a change from (x).999 to (x+1).000. Be aware three fractional digits may be a rounded representation.
Another option to check is the VCDS software. These systems normally are rule based, so if you can find the rule set, you may be able to find the value conversion scheme.
Yet another option is to send crafted fake values to the VCDS to see how they translate.
Regards, Michael
Am 18.08.20 um 09:36 schrieb Soko:
Morning,
Sorry, I reckon I've been at it for too long and did imply a lot of things which are not obvious for someone who didn't do it a million times already (as it feels to me :/ ). Let's do it step by step and (hopefully) my error is somewhere in the early steps...
I have OVMS (no polls active) and VCDS connected to the OBD port via a Y cable. In OVMS I turn the canlog on so I can see what VCDS is sending...
In VCDS I go into the 8C ECU and choose "Erweiterte Messwerte". There I have a list of all values which I can activate so VCDS polls it from the ECU. VCDS polls/refreshes those chosen values roughly 10 times a second.
So I activate AmpHrs charged and discharged, and KwH charged and discharged from the list (4 values in total). VCDS shows me the final converted values on the screen:
OVMS logs the following on the bus:
As far as I understand it are the red lines the polls by VCDS (as they are 7E5=the header of the ECU) and the green lines are the replies. I don't understand the red "7E5 30..." line but I reckon its just like a "Please give me the rest of the data". All 4 values are in the same PID "1E 32" (red, bold and underlined above). And the green lines, the bold and underlined HEX values are the actual data with the last "aa" being no part of it. This is my procedure to get the data bytes from the canlog-monitor lines...Maybe here is my first error somewhere?!
So I have 16 bytes in total to get the 4 values VCDS is showing me: *00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04*
Simplicity and the value-Ids (some internal numbers) in the list of VCDS suggest that same order as given above. So the first 4 bytes is AmpHrs Chrg, than AmpHrs Dischrg and so on. This leads us to this relation:
I've also confirmed this by waiting (with ignition on) until the last fraction of AmpHrs/kWh discharged values where changing in VCDS to -1154,307 and -362,448 and comparing it with the changed hex values. These hex numbers changed: *00 09 de e1 ff f6 53 2_/0/_ 00 31 62 92 ff d0 88 0_/3/_*
Having said that: Not every change in the hex value changes the display value. I have more detail data to this if needed...
The way they change suggests 2's complement storage. So let's convert them into integers:
Now, for kWh Dischrg as an example, the factor to get from -3110908 to -326,447 is: -3110908/-362,447=8583,07007645256. This is a weird one as all the other factors so far have been nice numbers!
If I do this for all 4 values I get these factors:
This cannot be right and therefore a waited a few weeks and did everything above again. The result was the same unfortunately:
Some other values have an offset. Like the power-efficiency on the charger-ECU. The value=0x00 means 75% and 0x06 means 75,6%. It's a quite clever way of using just one byte for this value! So I thought they use some offset here to. But if I take the delta of both days I logged I get the same factors:
For AmpHrs Chrg the first value was 1177,611 and the value week later was 1424,809 which gives the delta of 247,198.
So now I'm really out of ideas and thought I ask you guys. As you can see all conversion factors are close to each other but not exactly the same. And given that the VW engineers are quite clever I can't accept they would use such weird factors for a conversion.
If you have now another look at the one screenshot I took in my first mail you should understand it as it is just the complete version of the bits I did here...
Any ideas are highly welcome.
Soko
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
PS: 0x30 = flow control frame. If you want to fake a host for the VCDS, you'll need to understand ISO TP. I think you should know ISO TP also before working on the OBD poller ;-) The wikipedia article suffices to understand the basics: https://en.wikipedia.org/wiki/ISO_15765-2 Regards, Michael Am 18.08.20 um 09:36 schrieb Soko:
As far as I understand it are the red lines the polls by VCDS (as they are 7E5=the header of the ECU) and the green lines are the replies. I don't understand the red "7E5 30..." line but I reckon its just like a "Please give me the rest of the data".
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
Ok, read through the wiki and the basics are understood. Speaking of the PR: If you guys have time it would be great if you can have another look. I've addressed all points and it's good to go from my point of view. thx Soko On 18.08.2020 10:15, Michael Balzer wrote:
PS: 0x30 = flow control frame. If you want to fake a host for the VCDS, you'll need to understand ISO TP. I think you should know ISO TP also before working on the OBD poller ;-)
The wikipedia article suffices to understand the basics: https://en.wikipedia.org/wiki/ISO_15765-2
Regards, Michael
Am 18.08.20 um 09:36 schrieb Soko:
As far as I understand it are the red lines the polls by VCDS (as they are 7E5=the header of the ECU) and the green lines are the replies. I don't understand the red "7E5 30..." line but I reckon its just like a "Please give me the rest of the data".
I can confirm this: Please just use copy-and-paste of the text, if you can, rather than screenshots. It is much easier for us to work with. This data is in ISO-TP, ISO15765 format: The 7E5 03 22 is a request The 7ED 10 … is the first frame of the reply. It provides length, and some data The 7E5 30 is a request for more frames (with the inter-frame timing specified) The 7ED 21 and 7ED 22 are the two remaining multi-frame replies The data returned is: 00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04 Assuming 4 bytes per data item returned, that would be: 00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04 To try to decode the values, Michael’s advise is good. You will need a few good known values, along with corresponding data from the CAN bus. One thing I can suggest is to disconnect the car so you just have ovms and the tool on the bus. Then you can spoof replies: can can1 tx standard 7ED 10 13 62 1e 32 00 09 de ... You can mess around with the values and see what the tool displays. I also suggest you load a crtd dump into savvy can - as well as an ISO TP decoder, that has some utilities to analyse the data in different ways and show all the different variants of values. The DBC editor is pretty useful for that. Regards, Mark.
On 18 Aug 2020, at 3:36 PM, Soko <ovms@soko.cc> wrote:
Morning,
Sorry, I reckon I've been at it for too long and did imply a lot of things which are not obvious for someone who didn't do it a million times already (as it feels to me :/ ). Let's do it step by step and (hopefully) my error is somewhere in the early steps...
I have OVMS (no polls active) and VCDS connected to the OBD port via a Y cable. In OVMS I turn the canlog on so I can see what VCDS is sending...
In VCDS I go into the 8C ECU and choose "Erweiterte Messwerte". There I have a list of all values which I can activate so VCDS polls it from the ECU. VCDS polls/refreshes those chosen values roughly 10 times a second.
So I activate AmpHrs charged and discharged, and KwH charged and discharged from the list (4 values in total). VCDS shows me the final converted values on the screen:
<mjbnpadlhnalnloe.png>
OVMS logs the following on the bus:
<lejcbcpkglbjjfae.png>
As far as I understand it are the red lines the polls by VCDS (as they are 7E5=the header of the ECU) and the green lines are the replies. I don't understand the red "7E5 30..." line but I reckon its just like a "Please give me the rest of the data". All 4 values are in the same PID "1E 32" (red, bold and underlined above). And the green lines, the bold and underlined HEX values are the actual data with the last "aa" being no part of it. This is my procedure to get the data bytes from the canlog-monitor lines...Maybe here is my first error somewhere?!
So I have 16 bytes in total to get the 4 values VCDS is showing me: 00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04
Simplicity and the value-Ids (some internal numbers) in the list of VCDS suggest that same order as given above. So the first 4 bytes is AmpHrs Chrg, than AmpHrs Dischrg and so on. This leads us to this relation:
<giamhgmdmpcelheb.png>
I've also confirmed this by waiting (with ignition on) until the last fraction of AmpHrs/kWh discharged values where changing in VCDS to -1154,307 and -362,448 and comparing it with the changed hex values. These hex numbers changed: 00 09 de e1 ff f6 53 20 00 31 62 92 ff d0 88 03
Having said that: Not every change in the hex value changes the display value. I have more detail data to this if needed...
The way they change suggests 2's complement storage. So let's convert them into integers:
<badiafagplgohkkj.png>
Now, for kWh Dischrg as an example, the factor to get from -3110908 to -326,447 is: -3110908/-362,447=8583,07007645256. This is a weird one as all the other factors so far have been nice numbers!
If I do this for all 4 values I get these factors:
<dncnhfckigkdlljc.png>
This cannot be right and therefore a waited a few weeks and did everything above again. The result was the same unfortunately:
<fiiimbljgldacoja.png>
Some other values have an offset. Like the power-efficiency on the charger-ECU. The value=0x00 means 75% and 0x06 means 75,6%. It's a quite clever way of using just one byte for this value! So I thought they use some offset here to. But if I take the delta of both days I logged I get the same factors:
<nblbkolnjdkfbhpi.png>
For AmpHrs Chrg the first value was 1177,611 and the value week later was 1424,809 which gives the delta of 247,198.
So now I'm really out of ideas and thought I ask you guys. As you can see all conversion factors are close to each other but not exactly the same. And given that the VW engineers are quite clever I can't accept they would use such weird factors for a conversion.
If you have now another look at the one screenshot I took in my first mail you should understand it as it is just the complete version of the bits I did here...
Any ideas are highly welcome.
Soko
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Hi Mark, Yeah, I've tried that but Thunderbird didn't let me color the lines separately so I did screenshots. So I understood ISO-TP, ISO15765 correctly without even knowing what it was :D ;) So its really up to interpreting the 16 bytes correctly. I think I will give the car fakeing with OVMS a try next. I think though it won't be that easy as VCDS polls the value with ~10Hz and I can't type that fast :) It may goto into error if the replies take too long... I've already tried to slow down the CPU where VCDS is running on. Because the UI of VCDS shows briefly the HEX value before it converts it to the real value. At least that's what I thought... But all it displays before the real value is shown is "Please wait..." :( Soko Your last paragraph contains many many words I don't know (yet). So I have to look into all this stuff as well... On 18.08.2020 10:31, Mark Webb-Johnson wrote:
I can confirm this:
Please just use copy-and-paste of the text, if you can, rather than screenshots. It is much easier for us to work with.
This data is in ISO-TP, ISO15765 format:
* The 7E5 03 22 is a request * The 7ED 10 … is the first frame of the reply. It provides length, and some data * The 7E5 30 is a request for more frames (with the inter-frame timing specified) * The 7ED 21 and 7ED 22 are the two remaining multi-frame replies
The data returned is:
00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04
Assuming 4 bytes per data item returned, that would be:
* 00 09 de e1 * ff f6 53 21 * 00 31 62 92 * ff d0 88 04
To try to decode the values, Michael’s advise is good. You will need a few good known values, along with corresponding data from the CAN bus.
One thing I can suggest is to disconnect the car so you just have ovms and the tool on the bus. Then you can spoof replies:
can can1 tx standard 7ED 10 13 62 1e 32 00 09 de ...
You can mess around with the values and see what the tool displays.
I also suggest you load a crtd dump into savvy can - as well as an ISO TP decoder, that has some utilities to analyse the data in different ways and show all the different variants of values. The DBC editor is pretty useful for that.
Regards, Mark.
On 18 Aug 2020, at 3:36 PM, Soko <ovms@soko.cc <mailto:ovms@soko.cc>> wrote:
Morning,
Sorry, I reckon I've been at it for too long and did imply a lot of things which are not obvious for someone who didn't do it a million times already (as it feels to me :/ ). Let's do it step by step and (hopefully) my error is somewhere in the early steps...
I have OVMS (no polls active) and VCDS connected to the OBD port via a Y cable. In OVMS I turn the canlog on so I can see what VCDS is sending...
In VCDS I go into the 8C ECU and choose "Erweiterte Messwerte". There I have a list of all values which I can activate so VCDS polls it from the ECU. VCDS polls/refreshes those chosen values roughly 10 times a second.
So I activate AmpHrs charged and discharged, and KwH charged and discharged from the list (4 values in total). VCDS shows me the final converted values on the screen:
<mjbnpadlhnalnloe.png>
OVMS logs the following on the bus:
<lejcbcpkglbjjfae.png>
As far as I understand it are the red lines the polls by VCDS (as they are 7E5=the header of the ECU) and the green lines are the replies. I don't understand the red "7E5 30..." line but I reckon its just like a "Please give me the rest of the data". All 4 values are in the same PID "1E 32" (red, bold and underlined above). And the green lines, the bold and underlined HEX values are the actual data with the last "aa" being no part of it. This is my procedure to get the data bytes from the canlog-monitor lines...Maybe here is my first error somewhere?!
So I have 16 bytes in total to get the 4 values VCDS is showing me: *00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04*
Simplicity and the value-Ids (some internal numbers) in the list of VCDS suggest that same order as given above. So the first 4 bytes is AmpHrs Chrg, than AmpHrs Dischrg and so on. This leads us to this relation:
<giamhgmdmpcelheb.png>
I've also confirmed this by waiting (with ignition on) until the last fraction of AmpHrs/kWh discharged values where changing in VCDS to -1154,307 and -362,448 and comparing it with the changed hex values. These hex numbers changed: *00 09 de e1 ff f6 53 2_/0/_ 00 31 62 92 ff d0 88 0_/3/_*
Having said that: Not every change in the hex value changes the display value. I have more detail data to this if needed...
The way they change suggests 2's complement storage. So let's convert them into integers:
<badiafagplgohkkj.png>
Now, for kWh Dischrg as an example, the factor to get from -3110908 to -326,447 is: -3110908/-362,447=8583,07007645256. This is a weird one as all the other factors so far have been nice numbers!
If I do this for all 4 values I get these factors:
<dncnhfckigkdlljc.png>
This cannot be right and therefore a waited a few weeks and did everything above again. The result was the same unfortunately:
<fiiimbljgldacoja.png>
Some other values have an offset. Like the power-efficiency on the charger-ECU. The value=0x00 means 75% and 0x06 means 75,6%. It's a quite clever way of using just one byte for this value! So I thought they use some offset here to. But if I take the delta of both days I logged I get the same factors:
<nblbkolnjdkfbhpi.png>
For AmpHrs Chrg the first value was 1177,611 and the value week later was 1424,809 which gives the delta of 247,198.
So now I'm really out of ideas and thought I ask you guys. As you can see all conversion factors are close to each other but not exactly the same. And given that the VW engineers are quite clever I can't accept they would use such weird factors for a conversion.
If you have now another look at the one screenshot I took in my first mail you should understand it as it is just the complete version of the bits I did here...
Any ideas are highly welcome.
Soko
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
About sending spoof replies to VCDS from OVMS: If I disconnect the obd plug from car I also loose power to OVMS, don't I? Is there an easier way to solve that than buying an OBD extension cable and make the two CAN wires switchable? On 18 August 2020 10:31:54 CEST, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
I can confirm this:
Please just use copy-and-paste of the text, if you can, rather than screenshots. It is much easier for us to work with.
This data is in ISO-TP, ISO15765 format:
The 7E5 03 22 is a request The 7ED 10 … is the first frame of the reply. It provides length, and some data The 7E5 30 is a request for more frames (with the inter-frame timing specified) The 7ED 21 and 7ED 22 are the two remaining multi-frame replies
The data returned is:
00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04
Assuming 4 bytes per data item returned, that would be:
00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04
To try to decode the values, Michael’s advise is good. You will need a few good known values, along with corresponding data from the CAN bus.
One thing I can suggest is to disconnect the car so you just have ovms and the tool on the bus. Then you can spoof replies:
can can1 tx standard 7ED 10 13 62 1e 32 00 09 de ...
You can mess around with the values and see what the tool displays.
I also suggest you load a crtd dump into savvy can - as well as an ISO TP decoder, that has some utilities to analyse the data in different ways and show all the different variants of values. The DBC editor is pretty useful for that.
Regards, Mark.
On 18 Aug 2020, at 3:36 PM, Soko <ovms@soko.cc> wrote:
Morning,
Sorry, I reckon I've been at it for too long and did imply a lot of things which are not obvious for someone who didn't do it a million times already (as it feels to me :/ ). Let's do it step by step and (hopefully) my error is somewhere in the early steps...
I have OVMS (no polls active) and VCDS connected to the OBD port via a Y cable. In OVMS I turn the canlog on so I can see what VCDS is sending...
In VCDS I go into the 8C ECU and choose "Erweiterte Messwerte". There I have a list of all values which I can activate so VCDS polls it from the ECU. VCDS polls/refreshes those chosen values roughly 10 times a second.
So I activate AmpHrs charged and discharged, and KwH charged and discharged from the list (4 values in total). VCDS shows me the final converted values on the screen:
<mjbnpadlhnalnloe.png>
OVMS logs the following on the bus:
<lejcbcpkglbjjfae.png>
As far as I understand it are the red lines the polls by VCDS (as they are 7E5=the header of the ECU) and the green lines are the replies. I don't understand the red "7E5 30..." line but I reckon its just like a "Please give me the rest of the data". All 4 values are in the same PID "1E 32" (red, bold and underlined above). And the green lines, the bold and underlined HEX values are the actual data with the last "aa" being no part of it. This is my procedure to get the data bytes from the canlog-monitor lines...Maybe here is my first error somewhere?!
So I have 16 bytes in total to get the 4 values VCDS is showing me: 00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04
Simplicity and the value-Ids (some internal numbers) in the list of VCDS suggest that same order as given above. So the first 4 bytes is AmpHrs Chrg, than AmpHrs Dischrg and so on. This leads us to this relation:
<giamhgmdmpcelheb.png>
I've also confirmed this by waiting (with ignition on) until the last fraction of AmpHrs/kWh discharged values where changing in VCDS to -1154,307 and -362,448 and comparing it with the changed hex values. These hex numbers changed: 00 09 de e1 ff f6 53 20 00 31 62 92 ff d0 88 03
Having said that: Not every change in the hex value changes the display value. I have more detail data to this if needed...
The way they change suggests 2's complement storage. So let's convert them into integers:
<badiafagplgohkkj.png>
Now, for kWh Dischrg as an example, the factor to get from -3110908 to -326,447 is: -3110908/-362,447=8583,07007645256. This is a weird one as all the other factors so far have been nice numbers!
If I do this for all 4 values I get these factors:
<dncnhfckigkdlljc.png>
This cannot be right and therefore a waited a few weeks and did everything above again. The result was the same unfortunately:
<fiiimbljgldacoja.png>
Some other values have an offset. Like the power-efficiency on the charger-ECU. The value=0x00 means 75% and 0x06 means 75,6%. It's a quite clever way of using just one byte for this value! So I thought they use some offset here to. But if I take the delta of both days I logged I get the same factors:
<nblbkolnjdkfbhpi.png>
For AmpHrs Chrg the first value was 1177,611 and the value week later was 1424,809 which gives the delta of 247,198.
So now I'm really out of ideas and thought I ask you guys. As you can see all conversion factors are close to each other but not exactly the same. And given that the VW engineers are quite clever I can't accept they would use such weird factors for a conversion.
If you have now another look at the one screenshot I took in my first mail you should understand it as it is just the complete version of the bits I did here...
Any ideas are highly welcome.
Soko
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Also plug the OVMS into your laptop usb when you are doing this. When the 12v is unplugged it will switch to 5v usb. Mark
On 19 Aug 2020, at 8:42 AM, Soko <ovms@soko.cc> wrote:
About sending spoof replies to VCDS from OVMS: If I disconnect the obd plug from car I also loose power to OVMS, don't I? Is there an easier way to solve that than buying an OBD extension cable and make the two CAN wires switchable?
On 18 August 2020 10:31:54 CEST, Mark Webb-Johnson <mark@webb-johnson.net> wrote: I can confirm this:
Please just use copy-and-paste of the text, if you can, rather than screenshots. It is much easier for us to work with.
This data is in ISO-TP, ISO15765 format:
The 7E5 03 22 is a request The 7ED 10 … is the first frame of the reply. It provides length, and some data The 7E5 30 is a request for more frames (with the inter-frame timing specified) The 7ED 21 and 7ED 22 are the two remaining multi-frame replies
The data returned is:
00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04
Assuming 4 bytes per data item returned, that would be:
00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04
To try to decode the values, Michael’s advise is good. You will need a few good known values, along with corresponding data from the CAN bus.
One thing I can suggest is to disconnect the car so you just have ovms and the tool on the bus. Then you can spoof replies:
can can1 tx standard 7ED 10 13 62 1e 32 00 09 de ...
You can mess around with the values and see what the tool displays.
I also suggest you load a crtd dump into savvy can - as well as an ISO TP decoder, that has some utilities to analyse the data in different ways and show all the different variants of values. The DBC editor is pretty useful for that.
Regards, Mark.
On 18 Aug 2020, at 3:36 PM, Soko <ovms@soko.cc> wrote:
Morning,
Sorry, I reckon I've been at it for too long and did imply a lot of things which are not obvious for someone who didn't do it a million times already (as it feels to me :/ ). Let's do it step by step and (hopefully) my error is somewhere in the early steps...
I have OVMS (no polls active) and VCDS connected to the OBD port via a Y cable. In OVMS I turn the canlog on so I can see what VCDS is sending...
In VCDS I go into the 8C ECU and choose "Erweiterte Messwerte". There I have a list of all values which I can activate so VCDS polls it from the ECU. VCDS polls/refreshes those chosen values roughly 10 times a second.
So I activate AmpHrs charged and discharged, and KwH charged and discharged from the list (4 values in total). VCDS shows me the final converted values on the screen:
<mjbnpadlhnalnloe.png>
OVMS logs the following on the bus:
<lejcbcpkglbjjfae.png>
As far as I understand it are the red lines the polls by VCDS (as they are 7E5=the header of the ECU) and the green lines are the replies. I don't understand the red "7E5 30..." line but I reckon its just like a "Please give me the rest of the data". All 4 values are in the same PID "1E 32" (red, bold and underlined above). And the green lines, the bold and underlined HEX values are the actual data with the last "aa" being no part of it. This is my procedure to get the data bytes from the canlog-monitor lines...Maybe here is my first error somewhere?!
So I have 16 bytes in total to get the 4 values VCDS is showing me: 00 09 de e1 ff f6 53 21 00 31 62 92 ff d0 88 04
Simplicity and the value-Ids (some internal numbers) in the list of VCDS suggest that same order as given above. So the first 4 bytes is AmpHrs Chrg, than AmpHrs Dischrg and so on. This leads us to this relation:
<giamhgmdmpcelheb.png>
I've also confirmed this by waiting (with ignition on) until the last fraction of AmpHrs/kWh discharged values where changing in VCDS to -1154,307 and -362,448 and comparing it with the changed hex values. These hex numbers changed: 00 09 de e1 ff f6 53 20 00 31 62 92 ff d0 88 03
Having said that: Not every change in the hex value changes the display value. I have more detail data to this if needed...
The way they change suggests 2's complement storage. So let's convert them into integers:
<badiafagplgohkkj.png>
Now, for kWh Dischrg as an example, the factor to get from -3110908 to -326,447 is: -3110908/-362,447=8583,07007645256. This is a weird one as all the other factors so far have been nice numbers!
If I do this for all 4 values I get these factors:
<dncnhfckigkdlljc.png>
This cannot be right and therefore a waited a few weeks and did everything above again. The result was the same unfortunately:
<fiiimbljgldacoja.png>
Some other values have an offset. Like the power-efficiency on the charger-ECU. The value=0x00 means 75% and 0x06 means 75,6%. It's a quite clever way of using just one byte for this value! So I thought they use some offset here to. But if I take the delta of both days I logged I get the same factors:
<nblbkolnjdkfbhpi.png>
For AmpHrs Chrg the first value was 1177,611 and the value week later was 1424,809 which gives the delta of 247,198.
So now I'm really out of ideas and thought I ask you guys. As you can see all conversion factors are close to each other but not exactly the same. And given that the VW engineers are quite clever I can't accept they would use such weird factors for a conversion.
If you have now another look at the one screenshot I took in my first mail you should understand it as it is just the complete version of the bits I did here...
Any ideas are highly welcome.
Soko
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
participants (4)
-
Derek Caudwell -
Mark Webb-Johnson -
Michael Balzer -
Soko