It seems re_stream_changed always sends “[]”, even if no changes. Easy to fix, but perhaps better to address in the Javascript.

I’m going to continue with the live DBC editing commands now. That is the pain point for using this in the real world (as the current approach is onerous: (a) detaching the DBC from the CAN bus, (b) switching to editor, (c) editing the DBC, (d) switch to command and ‘dbc autoload’, then (e) re-attaching the DBC to the CAN bus). It can, however, all be done from the web interface. Also building up DBC files for Tesla Roadster, to see just how much can be done in that format. I think I also need to get my DBC implementation to support strings.

Regards, Mark.

On 24 Feb 2019, at 9:38 PM, Michael Balzer <dexter@expeedo.de> wrote:
Mark,

the crash was no memory corruption, it was a race condition in the RE init (the queue was created after the task).

DataTables integration and the first draft of the RE tools web UI is now in the hub. It's already usable but far from finished, so I didn't include it as a standard page. Install "web/retools.htm" as a plugin to use & work on it.

The layout is following your draft, and I've added some rules for small devices.

I've added some control buttons and status output, the table stream updates and a detail call on row clicks. Btw: it may be better to send detail info to the browser using the stream updates (for example in hidden columns), so the browser doesn't need to request them.

The table is mostly standard DataTable, I've only enabled the responsive plugin, that does dynamic hiding & drilldown of columns. I've included most of the plugins, so there are many additional options:

Included libraries:

DataTables 1.10.18, Buttons 1.5.4, Column visibility 1.5.4, ColReorder 1.5.0, FixedColumns 3.2.5, FixedHeader 3.1.4, KeyTable 2.5.0, Responsive 2.2.2, RowGroup 1.1.0, RowReorder 1.2.4, Scroller 1.5.0, Select 1.2.6

see https://datatables.net/extensions/index

The web UI listens to stream updates for the table and the text displays using this scheme:

Subtype "retools/list/set" → replace list by payload

Subtype "retools/list/update" → update/extend list by payload

Subtype "retools/<boxid>" → set payload as text content in box <boxid>, boxid e.g. "status", "info" or "extra"

Example: notify raise text stream retools/extra "hi there"

Text boxes are currently <samp> with monospace font & text to html conversion. If you'd like to display HTML, replace "$('<div/>').text(msg.value).html()" by just "msg.value" on line 175.

Table updates are sent as JSON arrays with HTML markup for the changed/discovered highlighting. This currently uses <b> & <u>, which are styled accordingly. Command "re stream list" creates a message with all rows (no highlighting), "re stream changed" includes only changed rows (with highlighting).

I've added a first stream sender on the per second ticker. That currently simply sends all changed rows and could be optimized to only send those changed since the last tx. It could also make sense to raise the update frequency then.

The status box monitors some "retools" events I introduced, the stream ticker also updates the box -- if the stream subscription & updating is going to be permanent (as now), the event coupling can be removed.

Regards,
Michael

Am 23.02.19 um 10:18 schrieb Michael Balzer:
Side note: starting the RE server from the web shell currently consistently results in a crash from xQueueReceive() that looks like our "old friend", the memory corruption.

Starting from the async console or ssh or via MP works. No idea yet how that could make a difference for xQueueReceive(), I'll assume it's the PSRAM bug for now.

D (38792) webcommand: HttpCommandStream[0x3fa2c358] init: handler=0x3fa2f6a0 command='re start' verbosity=65535I (38812) webcommand: HttpCommandStream[0x3fa2c358]: 1996204 bytes free, executing: re startassertion "res == coreID || res == portMUX_FREE_VAL" failed: file "/home/balzer/esp/esp-idf/components/freertos/portmux_impl.inc.h", line 105, function: vPortCPUAcquireMutexIntsDisabledExtramabort() was called at PC 0x40112f67 on core 1Backtrace: 0x40091188:0x3ffea650 0x40091383:0x3ffea670 0x40112f67:0x3ffea690 0x40093ce3:0x3ffea6c0 0x40095ac0:0x3ffea6f0 0x4016c08a:0x3ffea730 0x4016c0c1:0x3ffea770Rebooting...

Am 16.02.19 um 14:00 schrieb Mark Webb-Johnson:
Thanks, Michael.

I can handle the RE side of things - just haven’t done a lot of work in JavaScript (too many people better at it than me at my day job, so I leave it to them). I can also probably extend it once the basics are working.

Just need a basic UI framework, and some way to push the data from RE.

No hurry.

Regards, Mark

On 16 Feb 2019, at 8:43 PM, Michael Balzer <dexter@expeedo.de> wrote:
Mark,

I haven't seen the necessity for a complex plugin just for table views up to now. Normally it's pretty simple to make dynamic tables just using HTML5 and jQuery. For example, see the file selector widget I created. That uses a table to display the file entries, it's dynamic, sortable & responsive with just a few lines of code.

DataTables adds a lot of features and simplifies manipulations of the data. I like the automatic responsive drilldown shown here:
https://datatables.net/extensions/responsive/examples/styling/bootstrap.html

It's got some weight of course (about half of the charts lib with just the most basic extensions), so we would add it as an optional asset. On a first look it will need some customization for us, as it uses the glyphicons for the bootstrap 3 variant, which we don't include for size reasons.

What do others think? And which of the extensions should be included?

Overall, the UI as drafted shouldn't be hard to implement. I'll see if I can create a first version, but probably won't have the time before next weekend.

Regards,
Michael

Am 13.02.19 um 13:05 schrieb Mark Webb-Johnson:
Michael, this is probably for you - but if anybody else can help please do…

I am trying to see how hard it would be to do a web based interface to the RETOOLS system. Presumably this would appear in the menu as Tools / Reverse Engineering (or some such abbreviation if that is too long). The basic screen would look something like this:

+----------------------------------------------------------------------------------------------------+

| |

| +----------+ +------+-------+------------------------------------+------+ +--------------------+ |

| | | | hdr1 | hdr2 | hdr3 | hdr4 | | | |
| | | +------+-------+------------------------------------+------+ |         | |

| | Controls | | | | | | | Info | |

| | | | col1 | col2 | col3 | col4 |  | | |

| | | | | | | |  | | |

| | | | | | | |  | | |

| | | | | | | |  | | |

| +----------+ | | | | |  +--------------------+ |

|   | | | | | |

| +----------+ | | | | |  +--------------------+ |

| | | | | | | | | | |

| | | | | | | |  | | |

| | | | | | | |  | | |

| | Status | | | | | |  | | |

| | | | | | TABLE | |  | | |

| | | | | | | |  | Extra | |

| | | | | | | |  | | |

| | | | | | | |  | | |

| | | | | | | |  | | |

| | | | | | | |  | | |

| | | | | | | |  | | |

| | | | | | | |  | | |

| +----------+ | | | | |  +--------------------+ |

|   | | | | | |
|   | | | | | |

|   | | | | | |

|   | | | | | |

|   | | | | | |

|   | | | | | |

|   | | | | | |

|   | | | | | |

|   | | | | | |

|   | | | | | |

| +------+-------+------------------------------------+------+   |

| |

+----------------------------------------------------------------------------------------------------+

The controls are just a set of controls - things that bring up dialogs for more information or perform actions. Like start/stop RE, switch modes, manipulate DBC files, etc. Left justified in the frame, and perhaps 10% of the width. Height can dynamically adjust with the text content.

The Status, Info, and Extra panes are just a dynamically updated text panes. Main code needs to be able to send text here and have it shown in the appropriate pane. Right justified in the frame and perhaps 30% of the width. Height can dynamically adjust with the text content.

The TABLE is a table. Probably 5 columns by many rows (anywhere from 20 to 200 or so). Either paged, scrolling, or just very tall. Paged is probably preferable. Would need a simple mechanism to dynamically add/update a row (identified by ‘key’ column and in sorted order), presumably using the web socket. Also need something to be able to dynamically clear the entire table. The RETOOLS code would use this to update the table in realtime (presumably over a web socket). It would be good if the table could do things like search filtering of displayed rows, column sorting, etc (all client side - something like datatables.net).

For example, look at this output:

OVMS# re dbc list

key records ms last

can1/100:0081 2 22000 01 02 03 04 05 | .....

can1/101 1 44000 01 02 03 04 | ....

dbc/DI_Signal1: 129 km

dbc/DI_Signal2: 1 km

dbc/DI_Signal3: 130 kWh

dbc/DI_Signal4: 122 Degrees

The idea is to be able to show the key, records, ms, last, and ascii dump in the main table. Then, clicking on a row would show the statistics for that message in the Info pane, and DBC signals in the Extra pane.

I can handle the RETOOLS side, but the javascript is beyond me. I also had a look at the ovms web server framework, but it doesn’t seem to be able to do the table part (yet)?

Any idea how complex this is?

Regards, Mark.
_______________________________________________
OvmsDev mailing list
OvmsDev@lists.openvehicles.com
http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________
OvmsDev mailing list
OvmsDev@lists.openvehicles.com
http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________
OvmsDev mailing list
OvmsDev@lists.openvehicles.com
http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________
OvmsDev mailing list
OvmsDev@lists.openvehicles.com
http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
_______________________________________________
OvmsDev mailing list
OvmsDev@lists.openvehicles.com
http://lists.openvehicles.com/mailman/listinfo/ovmsdev